Over at London's Pizza on the Park, Richard Rodney Bennett is singing them sweet and low-down. A little Gershwin, a little Arlen, Porter, Kern, Rodgers and Hart... On a good night, he says, "it's like having dinner with your best friend". A great song is a conversation piece, a confidence, a word or two in someone's ear. Bennett knows the form, he knows how to share a lyric. No Englishman - and certainly not one from Broadstairs, Kent - should inhabit these American classics as he does. The voice has been lived with, lived in - more low-down than sweet - but it's a real voice, a well-marinated voice: melodies invite its embellishments. That's instinct, that's jazz for you. You've either got it or you ain't. And like vocal cords, like fingers. It's the way they just happen to wander into the most fabulously unlikely chords. Harmony. That's a composer's ear for you.

And when he isn't being a cabaret performer (you might call it a pastime, if he weren't so damn good at it), Richard Rodney Bennett is a composer. Now, even here, he's hard to pin down. The one-time pupil of Pierre Boulez has written in every conceivable genre. If he were an athlete, he'd be a decathlete. From opera to the 16-bar popular song, from jazz piano to abstract piano, from orchestral atonalism to Murder on the Orient Express. That lush, chromium-plated, insidiously catchy little waltz tune has done all right by him, thank you. Movie music (and the credit list is enormous - from Indiscreet in 1958 to Four Weddings and a Funeral) has bought him freedom and no small fortune. He is, in every facet of the term, "a professional musician". So actually there are a number of Richard Rodney Bennetts, each as talented, as wry and as dry, as genial and as plain-speaking as the other. And I've just met them all.

Richard Rodney Bennett, the composer of "concert music" (as he chooses to refer to his "classical" pursuits), has a world premiere imminent. His Partita for Orchestra is the commissioned work for the 1995/96 BT Celebration Series. In an age when contemporary pieces still have to look hard for that second performance, this exceptional venture - co-ordinated by the Association of British Orchestras - guarantees no less than 17 nationwide. And more than that, 17 different performances from 17 very distinctive ensembles: from chamber groups like the City of London Sinfonia or the Scottish Chamber Orchestra to big bands like the Halle or the Royal Liverpool Philharmonic or the capital's own - the Philharmonia and London Symphony orchestras. Major international conductors line up at the helm. Christoph von Dohnanyi conducts the premiere in London on 19 October. In Leicester, on 29 March next year, Sir Simon Rattle and the CBSO play it to celebrate the composer's 60th birthday. Some birthday present.

A funny thing happened on the way to this piece. And it all began when Bennett gave up smoking. Call it nicotine withdrawal, call it coincidence, but the notes simply stopped coming. For a full eight months he was dry. Mid-commission. It shook him up. Then along came this BT project. Now, Bennett generally prefers writing specifically for players and friends: the personal pragmatist. And if, in the process, he can create repertoire where it's really needed, so much the better. How many harpsichord or marimba concertos are there? If Stan Getz asks for a Saxophone Concerto, you don't say no.

The Partita was for the late Sheila MacCrindle of Chester Music. And because it was for her, a playmate and soulmate who shared Bennett's love of popular music, it began taking shape in D major. "Now, of course, I've written tonal music before - a great deal of it. But it was always what I thought of as 'light music' - you know, film music, incidental music. So I started on this piece thinking, 'Oh, well, so it's in D major and full of tunes - that's fine, Sheila would have loved it.' And then I realised that it wasn't a 'light music' piece at all. It was a 'proper' piece - but in D major and full of tunes. And because I suddenly wanted to write this way - more lyrically, more romantically - it was scary. I suppose that, 30 years ago, we felt that way about going out on a limb to write violent and dissonant music. And now..."

And now... liberation? Isn't it encouraging that composers would seem to be free to write what they want to write and not what they are expected to write? Bennett is cautiously optimistic. He regrets - no, detests - the advent of so-called "neo-romanticism", particularly prevalent in the US, where he now resides. "So many of these people just don't know how to write tonal music - that's the problem. A few harp arpeggios in D flat major and everybody's cooing, 'Oooh, it's so romantic' - I mean, really... And because of this freedom, this 'liberation' that you so rightly point to, I think it's actually terribly difficult for young composers now - because every door is open. So much of what I hear doesn't sound genuine for me. Worst of all is that sense of a composer not having listened to and really loved and absorbed the whole spectrum of music. When you're in your twenties, that's the time to go through huge influences. Heaven knows, I did... But I also knew what was mine and what wasn't."

Bennett was a child of the "Darmstadt years". This historic town in West Germany became a Mecca for new music when Wolfgang Steinecke initiated his summer courses there in 1946. In the 1950s it was the place in the world to hear and share and experience new music, to be at the cutting edge of it. This was no velvet underground. Soft options were no options at all. Bennett remembers the passion of it all: working through the night to copy unpublished Stockhausen manuscripts; he and Cornelius Cardew playing Boulez's Structures on two upright pianos; Boulez, Henze and Bruno Maderna teaching in one room.

Boulez was both teacher and hero to him - "the greatest composer in the world, bar none" - the first teacher to stop him in his tracks and ask, "Why? Why do you do this?" He maintains a diplomatic silence when I ask him where Boulez stands now in his pantheon of composers - but then... New music wasn't in the least "fashionable" then. His generation of composers (and it was a prime vintage: Alexander Goehr, Peter Maxwell Davies, Nicholas Maw, Harrison Birtwistle) made their mistakes "in Arts Council drawing- rooms, not at the Proms".

Bennett has been a professional composer for some 40 years now. It all began with a documentary film about insurance in which the celebrated Leon Goossens was his oboist. That impressed him, made him feel "legitimate" (what was good enough for Elgar and Vaughan Williams was good enough for him). To this day, though, he still equates film music with journalism - "you're not composing the canvas, you're colouring it". This journalist concedes. But films have been good to him, and he'd be the first to acknowledge that there is no better way of exercising the technique, keeping it supple. It shocks him to discover how ill-tutored some young composers are in the rudimentaries of being a composer - "laying out a page of manuscript, getting the music on to the page, understanding orchestration".

Throughout a colourful career Bennett has chosen to keep his various musical activities strictly compartmentalised. Until 1990, he'd never contemplated bridging the gap between his jazz and concert music skills, for fear of diluting the one with the other and "ending up with a kind of nondescript mud-colour". But after decades of keeping their distance, Richard Rodney Bennett, the jazz musician, and Richard Rodney Bennett, the concert composer, finally found each other in the Saxophone Concerto for Stan Getz.

The great man never did play it (John Harle recently recorded it for Argo) - he died in the year of its completion just as Bennett was faxing page after page of score to his bedside. But that piece really spun Bennett around. Writing it was the culmination of a process which began when Bennett moved to New York City in 1979. He left his inhibitions - or some of them - back in Blighty. His technique began to loosen up. The slow movement of the Getz Concerto sounds like a release, a sequence of swooning violin exhalations enticing that nighthawk saxophone into a slow waltz through a smoky-blue urban landscape - structured and yet so free, like a Gershwin tune. When Leonard Bernstein said, "It's all jazz", isn't that what he was really saying: free the spirit, liberate the expression? It's only taken Bennett 40 years.

n Richard Rodney Bennett plays at the Pizza on the Park, 11 Knightsbridge, London SW1, twice nightly, at 9.15 and 11.15pm, to Saturday. Booking: 0171-235 5273

n Christoph von Dohnanyi conducts the world premiere of Bennett's 'Partita' at 7.30pm on Thursday 19 October at the RFH, London SE1. Booking: 0171- 960 4242