Robin doesn't travel by himself on public transport and so she needs to accompany him. This, as every parent and carer reading this will know, is simply a question of time.
An MP's life has little time as it is - even less if you are on the front bench. Mrs Browning is the Opposition spokeswoman for education and disability. Clearly, something had to give.
"My son hasn't been very good since Christmas. We've been struggling on since then and he spends most of his time in Devon with my husband," said Mrs Browning, 51, the MP for Tiverton and Honiton and a special counsellor to the National Autistic Society.
"We've lived with autism for 26 years. It does have its ups and downs, but it's been obvious over the past few weeks that he does need my help and support now. In the end, the decision was easy to make."
Robin has Asperger's Syndrome, a form of autism that leaves sufferers with difficulties in interpreting social signals - generating deep incomprehension and obsessive behaviour.
But, as Mrs Browning explains, it is a form of autism which allows for a fairly high level of functioning. For example, he went to local and private schools and passed several GCSEs. But it was a struggle and, through all of this, no one really knew what was wrong.
This is because Asperger's Syndrome is notoriously difficult to diagnose and Robin was 19 before anyone told him and his family (they have one other son, Philip, 29,) what the problem was.
Mrs Browning says that she has since learnt that the words "infantile autism" do appear in hospital notes from when he was a baby.
"I will always be angry that the people who dealt with him as a child failed to tell us," she said. "If they had, it would have changed his life."
Robin will read this story and his mother did not want to go into details about him or their life as a family. But she did say that both she and her husband, David, a carer who also works as her research assistant, are devoted to helping him build his self-confidence.
This, too, is simply a matter of taking the time. Both of them are "all too aware" that they are not going to be around forever.
Like many with an autistic condition, Robin none the less has a particular talent or a "savant" side. His mathematical ability as a child far outstripped even his brightest classmates.
Mrs Browning said: "Robin has an exceptional head for figures, in particular sequencing numbers.
"His former head teacher told me recently that the scores he set in maths at his old primary school have never been surpassed - even by subsequent pupils with huge Mensa scores."
Her resignation, which will take place during the next reshuffle, means that Gillian Shephard could be the only woman on the Opposition front bench.
But Mrs Browning does not want to do the job if she is not "firing on all cylinders". She talks about being on the telephone at 10.15am trying to cope with a problem at home, knowing that she had to go into committee by 10.30am. "It tears you in two directions. When somebody is not well, you can't say: Please hurry up!"
She did not believe the situation would get any easier. "If you are on the front bench, something can happen in the morning that means you have to be in the House at 2.30pm. What if you had a hospital appointment scheduled for that time?" she asks.
Has this ever actually happened? "No, but I have been in a situation when I have been quite torn and where I felt that I would rather be doing something else."
Would a male MP have made the same decision? "I don't know," she said, adding that Tory party leader William Hague did not once try and talk her out of her decision.
"He said that if we are the party that talks about understanding disabilities, then if there is someone on our front bench with the problem, it is important we understand the position."
Mr Hague released a letter yesterday in which he says he regrets that she will be leaving the front bench but that he sympathises with her reasons.
Mrs Browning does not see this as a sacrifice. She has been an MP for six years and has been on the front bench for four of those and was junior minister at the Ministry of Agriculture, Fisheries and Food before last year's general election defeat.
"I have had a pretty good run. This is not my political obituary. I just have my priorities very clearly focused on my son," she said.
The problem of too little time has become known as the "time squeeze". Most of us - and especially Members of Parliament - try to deal with this by hiring people to do the things that cannot be done if you work long and difficult hours.
But Mrs Browning notes that there are certain things that cannot be bought.
"It is simply not possible. I could hire someone to do his washing and make his bed, but I could never hire somebody to do this," she said. "People with autism have very few people they are close to. He is close to me and I must be there."
Her case may be more high profile but she does not see her or her husband's situation as unusual. "People who are carers make these decisions all the time. I spend a lot of time with carers," she said.
"Recently, I was talking to a young couple who have a young child with autism and they were saying that for the first time in a year they had found someone who could come into their house who could understand their child. This meant that they could go out. So they decided to go out for a meal. They said that, when they actually got to the restaurant, they weren't really sure what to do. That just encapsulates what goes on every day.
"I'm not a special case. There are thousands of us."Reuse content