All the standing stones allow you to learn your relevant skills 20% faster.

The Apprentice Stone
South of Solitude on an island, in a small water marshland. Helps you recover magicka quickly, but makes you susceptible to magicka damage.

The Atronach Stone
Between Riften and Windhel in the marshland. Just outside the mountain range between the two locations, near the Eldergleam Sanctuary and Darkwater Crossing by the river to the east of the two. Gives you a large magicka boost and enables you to absorb spell damage, but you recover it slowly.

The Lady Stone
On a small island on the lake to the west of Riverwood. Allows you to regenerate health and stamina quickly.

The Lord Stone
Southwest of Dawnstar in the mountain range, on the northern peak. Makes you resistant to magicka and physical damage.

The Lover Stone
East of Markarth, about one fifth of the way to Solitude. Helps you improve all your skills faster.

The Ritual Stone
East of Whiterun, south of the river, between Whiterun and Valtheim Towers. Allows you to re-animate nearby corpses to fight for you.

The Serpent Stone
East of Winterhold, north of Windhelm, in the sea, on a floating ice berg. Gives you a ranged paralyzing poison to use once a day.

The Shadow Stone
South of Riften, on a small mountain before the large mountain range to the south. Lets you become invisible once a day for an extended period.

The Steed Stone
On the tip of the highest mountain, northwest of Solitude. Increases your carrying capacity without any movement penalty.

The Tower Stone
On a mountain in the middle of Winterhold and Dawnstar. Allows you to automatically open expert and lower locks.