No wonder women are obsessed with their age. They simply can't get it right. In a witheringly misogynistic article about the Liberal Democrat minister Lynne Featherstone last week, Quentin Letts of the Daily Mail wrote: "Though aged 60 (and counting), she teeters up to the despatch box in high heels, grinning girlishly." Having gratuitously mentioned that she is "of Jewish stock", he then admitted she looked good for her age, but only because of "whatever seaweed-yoghurty unguent the intensely fashion-conscious Lynne slaps on her pelt of an evening".

Her appearance has nothing to do with her ability as a minister. Yet, we're told, at 60, she's too old for heels and too old to smile. She's also, according to Letts, an "ageing vamp". And she's covered in animal fur. Nice.

There's a worrying tendency these days for successful women to be derided as either too old or too young, with not much in between. This is reflected in a tricky new career structure which peaks at exactly the age when working mothers are forced to tread water because of their children.

The "too young" derision can cut in surprisingly late in life. At a conference dinner the other day, I was placed next to a banker who asked the dreaded question, "So what do you do?" I dread it because I do lots of different things and find it hard to guess which one my interlocutor will be interested in. I thought it best to list them and let him choose. "Well, I write a weekly column for The Independent , present the odd radio programme, chair a think-tank, sit on a couple of boards and do some not-for-profit stuff," I said.

"You're a busy little girl!" was his reply.

Little girl?! I'm half a century old. I haven't been little since I was about eight, and I found it patronising even then. This man was only a few years older than me; had he been in his eighties, I might have winced and endured it. But he had no excuse for this trivial but irritating chauvinism.

Later that evening, I asked my Twitter followers and Facebook friends what I should have done. "Thump him" was the favoured response, followed by poking him with a fork and pouring wine on his lap. The best came from a man: "You should have stabbed the sod in the eye with a cocktail sausage." What was interesting, though, was how impassioned the debate was.

Some men claimed they wouldn't have minded being dubbed a little boy. But there's no symmetry here. Had I called the banker a little boy, he might have interpreted it as flirtatiousness (by the way, I'm sure his was not a clumsy flirtatious move). A woman saying that to a man has none of the centuries-old baggage of condescension that accompanies a man patronising a woman.

If we're not little girls, though, we turn almost instantly into wrinklies to be dismissed. Sometimes we can be both at once.

In my twenties and thirties, I had loads of offers to present television programmes. I took up some, but soon found that combining a full-time newspaper job with raising children was hard enough without adding TV commitments. Once I hit my mid-forties and our daughters were teenagers, I thought it would be good to do some more TV presenting. The executives I went to see were perfectly pleasant, but nothing came of it. Like other women of my age, in their eyes I was too old. Yet we're the same people we were when these commissioners were falling over themselves to get us in front of a camera – if anything, we're better informed, more confident and more experienced. And if our wrinkles would look bad on TV, what about those of Jeremy Paxman or David Dimbleby, who are far older?

This matters, not just because it is downright discriminatory, but because the job market has changed. Careers used to progress in a linear fashion, with people hitting their peak in their mid- to late-fifties, giving them five or ten years at the top before retirement. Now, men usually become chief executives, director-generals, even prime ministers, in their forties.

Tony Blair entered Downing Street at just 43. A working woman of 43, though, is usually still in the child-rearing stage of life, in which it is pretty much impossible to do an extreme job like prime minister or chief executive unless she has a stay-at-home husband. Most working mothers soft-pedal until their children are old enough not to need them so much. By then, though, they find they have been overtaken by younger men.

Take Ed Balls and Yvette Cooper. In the last Labour leadership election, Cooper said she couldn't run because her children were too young. Yet Balls, who did run, not only had young children too – he had the same young children as her. Next time, she may be allowed a crack at the job, but only because Balls knows he is too unpopular to win the leadership for himself. And by then, it might be too late for her.

Which is why it is important that the BBC Trust doesn't hold the age of the two leading candidates for Director-General against them. Caroline Thomson, the BBC's chief operating officer, is 57; Helen Boaden, director of news, is 56. Both are slightly older than the departing DG, Mark Thompson, and quite a bit older than their male rivals. Twenty years ago, they would have been considered the perfect age for a top job and would have been competing against their male contemporaries. Now they probably worry they have missed their opportunity.

Because of this new phenomenon of early promotion, women are feeling the squeeze at both ends of their working lives. When they are young (and sometimes even middle-aged), they are patronised as little girls; in their mid-life, when their male colleagues are racing ahead, they are held back by family responsibilities; and by the time they reach their peak of intellectual and emotional maturity, they are already deemed to be past it. At all ages, they're judged on their appearance, however irrelevant it is to the job they do.

It doesn't have to be like this. We all, but men in particular, need to question our assumptions. Treat people with respect, value their ability, accept that some take longer to get to the top, don't judge them by their age. Let's not ask what Caroline Thomson or Helen Boaden slap on their pelts. Oh, and put those little girls to bed right now.

m.sieghart@independent.co.uk / twitter.com/MASieghart