When you are in overgrown near the covered bridge, head to the side of the barn with 3 boxes outside.
Now climb onto the top one and aim down to a lower part of the fence and jump for it (tricky the first couple of times).
Once you are on the fence walk towards the bridge, jump onto it, and you can get onto other buildings from there by going on the other walls.