What happened to Harry Potter? More than 400 million copies of Potter books have been sold, setting records for the fastest selling books in history. So why was his creator, J K Rowling, absent from the list of books most borrowed from public libraries when it was published the other day?

It was striking that seven out of the 10 top authors in the Most Borrowed list write for children. At the top, as ever, was the potboiler adult thriller writer James Patterson, with a staggering two million loans, compared with a mere 37,422 for the most borrowed non-fiction title, At My Mother's Knee, and Other Low Joints by Paul O'Grady (maiden name, Lily Savage).

But the rest of the Top 10 was dominated by kids' books – Jacqueline Wilson (Tracy Beaker), Francesca Simon (Horrid Henry), Mick Inkpen (Kipper et al), Lauren Child (Charlie and Lola), Terry Deary (Horrible Histories) and Julia Donaldson, whose book The Gruffalo was the most borrowed children's title last year.

So where was Harry Potter? I have a theory. But first a confession...

The Top 10 was compiled by the Public Lending Right scheme, which administers the payment of royalties to authors. It revealed that children's borrowing of library books has been on the rise for the past six years. Almost 80 per cent of five- to 10-year-olds now use our public libraries, the highest figure since records began over 20 years ago. That in itself tells us something about why our libraries are so under threat in the current public spending tsunami. Children, like the elderly at the other end of the vulnerability scale, have no political voice.

I am part of the problem here. As a boy, I was an obsessive user of Whinney Banks Library in my home town of Middlesbrough. I would cycle there three times a week, or more, since you were only allowed to take out three books at a time. You could have a fourth, but it had to be non-fiction, so often I didn't bother. Later, I got my mum to join, since adults could get seven books out. I purloined her tickets, which decreased my cycling and increased my reading time.

Nowadays, I am a book buyer. I write at home, so Amazon, Abebooks and Alibris are frequent ports of call for work-related volumes, and also, just as often, as a displacement activity. So I am a bit out of touch with the library system.

You might say the same about many of the naysayers on the websites of the various Save Our Library campaigns. "Years ago I worked at our local public library," said one. "If you wanted a very recent book, because of their buying practices, it could be from three to six months before it arrived. I can't wait that long! So, I buy my books."

"Years ago" are the operative words here. I recently spoke to an old university friend who rose to a very senior position in the library service. There was a time when some public libraries were a bit sniffy about popular fiction such as Harry Potter, he said. Librarians saw no reason to pander to the hyped-up demands of book publicity campaigns by getting books on the shelves more quickly or in greater quantities.

But that's not been the case for years now. Libraries entered into the Harry Potter hysteria, buying large numbers in advance and having them ready for borrowing at one minute past midnight on the day of publication. It all sounds as dated now as the race for the Beaujolais Nouveau.

But old perceptions clearly linger, as does the apprehension that libraries still look down on popular fiction and that anyone going in and asking for it would be treated like a youth going into Boots in the 1960s and asking to speak to a male assistant.

The truth is that authors such as J K Rowling are in steady demand in libraries – though not, perhaps, in proportion to the numbers of their books sold. There may be several reasons for that. Fashionable books are, by definition, read by many youngsters who don't do an awful lot of other reading, so they are unlikely to be library users or to think of the library as a source.

There is also a touch of consumer fetishism at work. Cult book mania is cleverly stoked by publicists weeks, even months, ahead of publication so that everyone wants to be the first of their friends to read, or have, the latest volume. The deferred gratification of hanging around for a couple of weeks to get it from the library is not part of the game. Possession is nine-tenths of the book.

Many also suspect that fiercely à la mode books – Stephen Hawking's 1988 A Brief History of Time is the classic case – are bought but never read. The veteran newspaper editor Harry Evans used to describe the phenomenon as "the virtue of unread copy". For some people, owning the book and going to see the film is sufficient.

So the other day I went to the local library where I live now and signed up. It was an act which was political as much as bookish. Use it or lose it.

It was like walking into the past, but a past we need to preserve. Amid all the self-scanning technology was a middle-aged lady in walking boots, leafing through a magazine. Old folk were peering myopically at the large print books. People in cheap clothing were at the bank of public computers, all but one of which were in use. Children were sitting on the floor reading the books and picturebooks they had pulled from the shelves. "Absolutely no eating or drinking in the Local Studies area," a sign read with the air of an authority long departed from elsewhere in our lives. It was an oasis of calm, purpose and civility.

You can take risks in a library. I serendipitously selected a volume called New Caribbean Poetry that I wouldn't have shelled out £12.95 to buy – even if Amazon has told me it was a book "You Might Also Like" – but which I'm now rather enjoying.

Having a book collection at home is wonderful, but it does not bring with it the thrill of running your fingers along a shelf of spines – as I saw a boy doing in that library – and pulling out a book at random just because it was, perhaps, where he had found the Lemony Snicket the week before. I watched him as he started to read. Who knows, the book he picked out might be the one to transport him to an unexpected world.

More than a third of children are apparently now leaving primary school without having met the required standard in reading, writing and maths. This is not the time to be closing down libraries where those who cannot afford Amazon and the internet can find the portal to a better place. And that is not just a utilitarian argument; it is about preserving the space in which the world of the imagination can be opened.

Last time I was back home I made a pilgrimage to Whinney Banks Library. It had been pulled down. Don't say you haven't been warned.