There are so many port scanners out there now, I don't think moving to a different port will protect against anyone but 15 year old kids in their parents' basements.
I have WAN SSH enabled on my routers because it's the easiest way to connect on the fly, but I also disable password auth.