Château de Menthon-Saint-Bernard, near Annecy. Photograph: Chris Moran

You won't find Annecy in any British ski brochures this year. In fact, for most British skiers, Annecy is that place you zoom through on the transfer from Geneva airport to Les Trois Vallées. It's not exactly ski hotlist material – which is an absolute travesty if culture and romance are as integral to your winter trip as the skiing.

Much of the old town dates back to the 10th century, and the hunched lanes and leaning walls left by those ancient builders now hide quaint cafes, patisseries and cool bars, all intersected by a network of crystal-clear canals. Having a post-ski beer at the Brasserie Le Munich as swans glide past is not something you can do in most resorts. Sit outside and you'll enjoy the best view of the Palais de l'Isle, an 11th-century prison that sits on its own island and trumps all other medieval buildings around the lake.

Rustic-charm advocate Jean Jacques Rousseau formed much of his philosophy in Annecy, and wrote how he wished to "surround this happy place with a gold baluster". Two centuries later artists came to paint scenes of 1920s motorboat racing and swimmers frolicking in the lake. Many original oil-paintings-turned-posters – including some famous Chamonix advertisements – are housed in the museum wing of the imposing Château d'Annecy above the old town (€4.80 entry, musees.agglo-annecy.fr), and Rousseau fans can visit the "gold baluster" he so wished for in the graveyard of Annecy cathedral. It was erected in his honour in 1978.

After exploring the old town, we crossed the Pont des Amours (admiring the pretty boats on the way) to L'Impérial Palace (hotel-imperial-palace.com), a Buckingham Palace-sized hotel that shares an outcrop with La Plage d'Annecy and its art deco diving platform. I (and baby Harry) bought mum Rachel a kir royale there for Mother's Day last year – a gloriously warm day in early spring.

With its deep colour, mountainous backdrop, and almost permanent haze, Lake Annecy has stunned plenty of visitors. Winston Churchill reckoned the lake held "the most beautiful view on earth", and when Mark Twain visited in 1891, he flatly refused to believe that he could do justice to the scenery in writing.

But does the skiing match up to the town? La Clusaz (laclusaz.com) is the big resort near Annecy, and you're likely to meet some Brits as it's featured by tour operators such as Crystal Ski (0871 231 2256, crystalski.co.uk), Ski Arrangements (01773 300288, skiarrangements.com), and Inghams (020-8780 7733, inghams.co.uk).

With its "Famille Plus" status, La Clusaz is marketed in France as a perfect beginners' destination, but local boy Candide Thovex is one of the best freeride skiers on the planet and the resort's funpark is definitely up to his exacting demands.

Next door is Le Grand Bornand (legrandbornand.com). It's not a huge resort by French standards, but with 40 lifts you'd be hard pressed to see everything in a day. Again it's sold as a quiet, family place, but good skiers will enjoy the lack of competition for the more challenging terrain, and there are some awesome north-facing, steeper runs straight off the top chair. From Annecy you can make it to both resorts in under 45 minutes, passing the spectacular Château de Menthon-Saint-Bernard (chateau-de-menthon.com) on the way. It's sadly closed during the winter, but worth a stop and a gander.

On the road from Annecy to La Clusaz and Le Grand Bornand are the two smaller resorts of Manigod (manigod.com) and St Jean de Sixt (saintjeandesixt.com). They are virtually unheard of by the British crowd and, as such, perfect if you want to practise your French. St Jean de Sixt is also home to the world's only bungee ski jump (bun-j-ride.com), which is as crazy as it sounds. Get the Aravis ski pass (aravis.com) and you can ski all of the above resorts for €177 for an adult six‑day pass.

But the best Annecy resort is also the closest – just 20 minutes' drive from the centre of town, Semnoz (semnoz.fr) is little more than a restaurant building and two chairlifts, but it accesses some luscious corduroy pistes and brilliant, hidden tree runs. Snowboard Semnoz in a snowstorm and you could be in Japan: short runs packed with great features and powder stashes. Semnoz also sells its lift-passes by the hour (€6 for two hours or €9 for a full day during the week), meaning, euro for euro, there's no better deal in the Alps.

Resorts like Val Thorens might offer fall-out-of-bed-on-to-the-pistes-convenience, but Annecy offers something much harder to define.

• The Hotel du Château (+33 450 452766, annecy-hotel.com) is in the heart of the old town with great views over the rooftops towards the lake. A large double room costs €70-€75 a night, including parking (€7pp for continental breakfast). You really need a car to get around, so drive down from the UK or hire a car through holidayautos.co.uk