But, if it makes you feel any better, according to estate agents any money spent on improving the garden will repay itself handsomely when the house is put up for sale.
"If you are selling a traditional stone cottage with wisteria in bloom around the door, you can add up to 20 per cent to the value," says Stephen King, of the estate agency Strutt and Parker. "The next few weeks up to the end of May will be the best time to sell if a property has a good garden. All the plants will be coming into bloom and buyers can see the potential."
In the Halifax Building Society's most recent survey, garden improvements rose to number two in the top 10 must-do list, above installing a fitted kitchen and new bathroom. And a Savills survey of new home buyers showed strong negative reactions to unfinished, sloping, overlooked or badly laid-out gardens.
"Buyers like a good-size garden with not too deep borders," says Stephen King.
"Beech is the favourite for a hedge, as it has leaves all year round and it provides privacy. Weeping willow is the worst."
Strutt and Parker are currently selling Church Cottage, overlooking the village green in Goosey, Oxfordshire, which is a gardener's dream. The gardens are enclosed by yew and beech hedges, and there is a rose-covered pergola, plus four vegetable and herb gardens, an orchard, and eight fully stocked rosebeds. The five-bedroom, 18th-century house is being sold for pounds 400,000.
The rose gardens of Glebe House, a stone-built house in Fortingall - the Scottish village said to be the birthplace of Pontius Pilate - are a major selling feature. The four-bedroom house, on the market through Knight Frank for pounds 180,000, is surrounded by a mass of bloom.
But even the most modest flowerbeds are likely to hold a few hundred pounds' worth of investment. Mature shrubs can be worth up to pounds 30 or pounds 40 each. And it takes a lot just to fill up half an acre. Rye Cottage, south of Basingstoke in Hampshire, is a four-bedroom thatched and beamed Grade II listed house with typical country cottage gardens. Its half-acre has been landscaped, to include stocked flowerbeds, bulbs, rose bushes and mature trees. It is for sale through Hill & Morrison for around pounds 325,000.
It is hardly surprising if vendors try to take as many plants with them as they can. One gardener I know plans to borrow a horse box to transport a couple of much loved small trees to his new home.
It's not uncommon for buyers to roll up with the removal van and find that the garden which so delighted them is now as bare as a wheat field in December.
"When I moved into my former farm worker's cottage in Hampshire," recalls Gillian Morris, "it had been empty for a year. The neighbours had stripped it of every single plant; there wasn't a Brussels sprout to be seen. Typical country folk, though - no point in letting all those vegetables go to waste. On the other hand, I have moved four times with my beautiful Liquidambar styraciflua, which is now about 5ft high. It goes fantastic shades of crimson and amber in autumn, and I wouldn't move home without it."
Anyone intending to dig up their garden and take it with them should spell out the details in the pre-contract enquiry forms which they give to their solicitor.
"This should take place before the property goes on the market," warns Mark Stewart, of Knight Frank. "The lists run through fixtures and fittings such as towel rails and taps. Greenhouses, sheds and plants are usually included. If you want to take a plant, dig it up in advance and put it in a pot. The agent must be aware of the excluded items, because we have to know exactly what we are selling."
As most people choose a house because of its setting, the gardens are part of the package.
"Most people wouldn't mind if it were just one or two plants," says Mark Stewart.
"I did have a client last year who wanted to take everything, but the buyers objected. In the end she was limited to a dozen, all named specifically. If, however, you take something which was included in the sale, you could be made to replace it."
Not everyone is smitten by gorgeous gardens. They can be intimidating, especially if they feature complicated and time-consuming topiary.
Stephen King recently took viewers around a country property with four acres of landscaped and terraced gardens; their owners' pride and joy. "They were dismayed by the comments of some viewers, who said they couldn't consider the property because it would need a full-time gardener to look after it." There is also a danger in buying a property and dramatically altering the garden. One buyer in Leicestershire planted trees in what had been a paddock.
When he came to sell, the county being pretty horsey, everyone said it was no good without the paddock - but by then the local authority had reclassified that part of the property as a wood and wouldn't let him chop down the trees.