For me it varies somewhat from ship to ship. It definitely doesn't have to mean that I want it to be canon or think it would be a good idea in real life. It does, however, generally mean that there's something I find interesting about the characters and their dynamic that I want to see more of (or in some cases, that there could be if we ever actually saw them interact), that I think something about them as a couple would be interesting/fun to watch, and that I think about the ship and create and/or engage with content for it in a more than incidental way, as you put it.

Finding at least one of the characters attractive makes me more likely to ship something, but it's neither necessary nor (usually) sufficient.