To leave town what you need to do is get about 30 paint brushes (if you drop one, it floats) and make "stairs" along the wall, then you walk up the "stairs" and jump over the wall.

Warning

If you do this then it will be hard to get back in and the ground will look strange and all of the bodies of water in the game will be gone. But you'll still be swimming (weird).