Its upper floors apparently abandoned, it rises behind another "blinded" three-storey building with doors and windows boarded up. It takes a second or two to follow the minister's gaze, and then we see it - a balcony, half-way up the tower, festooned with plant pots. Geraniums and spider plants in profusion indicate, not just a human presence, but a human imagination pitted against bleak reality.
"That shows the strength of the human spirit," observes Ms Jowell. There is a respectful pause, a murmur of assent, and then the entourage moves on.
This sort of thing impresses the natives. Ms Jowell is warm, committed and transparently sincere - and she charms everyone she meets. A health visitor whose project she has come to inspect is almost speechless with delight. A solemn GP expresses frank surprise at the seriousness with which his views are sought and, apparently, heard.
The minister is in Birmingham to begin the process of putting flesh on the bones of the Government's pledge to tackle health inequalities. Last week, Peter Mandelson announced that he would be chairing a new cabinet committee set up to examine ways of overcoming social exclusion. Ms Jowell is already out gathering the material that will, in time, shape the new policy.
Labour urgently needs this material to prove that its pledge is more than just warm words. As the honeymoon with the electorate comes to an end, critics are questioning how much the Minister for Public Health, Britain's first, can achieve without a budget of her own.
When challenged on the lack of money, the minister's normally sweet demeanour acquires a steely edge. She brusquely dismisses any suggestion of a threat to her credibility. "We start by saying we are going to do it [tackle inequalities] and then we set about doing it. That is a defining aspect of this government. Today is about charting that journey. There are no short-term solutions. It will take a long time and require sustained and determined commitment."
So how is it to be done? The journey today begins in a leisure centre on the fringes of a deprived area of Birmingham where seriously overweight, unfit people who would never have contemplated visiting a gym are being enticed into twice-weekly sessions on an exercise bike by an innovative scheme known as "exercise on prescription". Instead of writing out prescriptions for drugs, local GPs can now prescribe a series of sessions at the centre, and patients get personal attention from trained experts who devise individual programmes to suit them.
Ms Jowell, wearing a powder blue shift dress, poses gamely for photographers astride an exercise bike, trying to preserve her dignity while chatting to Cath Hughes, a large, cheerful lady of 48 who has suffered from arthritis for 10 years, and Ann Clough, 52, who has reduced her weight from 17 stones to 11 stones since coming to the centre. Both are now enthusiastic and regular attenders. Ms Hughes said: "I had never been in a gym before, I hadn't the confidence. I thought you needed to get into a leotard first."
The minister was impressed. Here was an example of needy people being helped to use a normal public facility that they would never otherwise have visited. Overcoming social exclusion was not about providing some expensive, specialist facility which perpetuated the exclusion. Nor was it just about poverty. It was about helping the 30 per cent of people who felt excluded from society by creating routes back into the mainstream.
Outside the centre, she said: "These are women for whom exercise and getting into exercise kit are as alien as paragliding. They are people who are as unfit as it is possible to measure - who can't walk 50 yards on the flat without getting out of breath - and who, without help, will become chronically disabled in later life. The task for government is understanding how to clear the pathways that will take people into facilities that already exist."
"Creating routes into the mainstream" could become Tessa Jowell's mantra. She repeated the phrase several times during the day. It is also, helpfully, low cost. Spending should be required only at the margin, because all the facilities are already there.
All the Government's efforts, if you look at how they develop in practice, are about finding routes into the mainstream, she says.
The idea fits with the personality. Ms Jowell is a naturally inclusive politician. She has an uncanny knack of making whoever she is talking to feel that they are the only person in the world for her at that moment. It is a quality that can be misinterpreted, especially by men. She is a tactile (more than a tactical) politician, which is what makes her so effective on these walkabouts, greeting strangers with a squeeze of the arm, hugging former colleagues and listening with the practised care of a psychiatric social worker (she qualified as one in the early Seventies, and was later assistant director of the mental health charity Mind).
But she is also hard-working, determined and ambitious; and that gives her an edge that can surprise the unwary. The next stop on our tour is a meeting at the Royal Orthopaedic Hospital where a posse of the local great and good, including the Bishop of Birmingham, have turned out to present their plan for a pounds 1m gymnasium for the hospital. They hope to win a lottery grant and run the gym as a commercial venture, open to the public, to defray the cost of providing it as a service to patients.
Ms Jowell is attentive throughout the presentation and asks polite questions afterwards. Most of those at the meeting appear to feel that their plans have been well received. Only those familiar with the minister's normal effusiveness will have suspected her of lacking enthusiasm. The drawback with the scheme, and its proponents are quite open about this, is that it is opportunistic - geared first to tapping into an available source of money and secondly to meet a perceived need, rather than the other way about.
The acid test, in the minister's eyes, is that a project should be focused on assisting a specific group of the disadvantaged to gain some benefit in a way that reduces, rather than increases, their separateness from the rest of society. Merely opening the doors of a hospital fitness centre to the general public - which will attract the leotard-clad already-fit because it is cheaper than the private gym down the road - is not the way to tackle social exclusion. It is the sort of woolly do-goodism that leaves Ms Jowell unimpressed.
The virtue of the hospital's scheme is that there is an identifiable source of money - the lottery - to pay for it. Ms Jowell displayed a curious lack of interest in the cost of the schemes she visited in Birmingham, not once asking how much money was involved or how it was raised. Yet officials privately conceded that in the case of the exercise-on-prescription project, which involved building an extension to the leisure centre eventually paid for by the health authority, there were fierce local battles about how the bill would be met. A second project - providing interpreters to work with Bangladeshi first-time mothers in a part of the city where the perinatal death rate is twice the national average - cost pounds 80,000. The organiser of that scheme disclosed this only after the minister had left the room - wiping her hand across her brow as she did so to indicate the trouble she had had funding it (by not filling health visitor posts as they fell vacant).
In a tower block next to the Bellevue Medical Centre, the next stop on the tour, a flat formerly occupied by a crack cocaine addict has been taken over and converted into an improvised community centre. After receiving a lick of paint, the centre now offers a thriving after-school club for local children, plus basic health services such as a post-natal support group.
Here, too, however, the funding is a problem. For what appears to be principally a social service is in fact paid for by the NHS. Bryan Stoten, chairman of the health authority, decided to step in out of despair at the local authority's failure to act. "It seemed to me someone had to do something. I have a slush fund, and I threw pixie dust over it. There had to be a health gain in there somewhere."
Those who met Ms Jowell were impressed by her preparedness to recognise that health is broader than the health service. As Dr Stephen Field, a GP in the Bellevue centre, whose practice area is home to drug dealers and prostitutes, told her: "The NHS is scratching the surface. Only by working across boundaries will we achieve anything."
This will be the real test for Ms Jowell - whether she can get departments, with their jealously guarded separate fiefdoms, to join forces and share budgets, both nationally and locally. Without a budget of her own, providing the funds to drive change will depend on borrowing from Peter to pay Paul.
The challenge will also be to extend local successes into the national arena. She says: "There is no one answer. Some will be Government-led, some local partnerships, and some about individual responsibility. We have to get beyond the extraordinary examples of projects led by a few shining individuals and make them commonplace."
In Birmingham, as elsewhere, the best projects are run by a single powerful personality with the vision and drive to carry it through. Repeating their successes elsewhere is never going to be simple.