The greatest ball gowns ever made: New York Met is honouring designer Charles James

James is the most influential designer you’ve never heard of – uncredited creator of the puffer jacket, and inspiration for everyone from Christian Dior to Rick Owens

The name Charles James means nothing to most people, inside and outside fashion. Mention him and they shrug, look blank – even when New York's Metropolitan Museum of Art announced that the inaugural show at the Anna Wintour Costume Centre (formerly the Costume Institute; it was rechristened on Monday) would celebrate James's work, many were none the wiser. Why? Because James is a hazy, slightly crazy figure in the annals of fashion. A perfectionist, an obsessive, an irascible, somewhat misanthropic genius. An exhibition at the Brooklyn Museum in 1982, four years after his death, was called, simply, The Genius of Charles James. The Metropolitan Museum chose the title, Beyond Fashion. Both are statements of fact.

Charles James was born in Sandhurst: his father a highly disapproving British army officer, his mother a Chicago grandee. Chicago is where James moved after leaving Harrow (expelled for a "sexual escapade"); he began to create millinery designs at the age of 19, under the name Boucheron, and then moved to New York in 1928, adding clothing to his oeuvre.

James had no formal training, perhaps explaining why he approached his craft in such an unconventional manner. He attacked the task of dressing a body with chutzpah, his early creations tightly trussing it in fabric and scissoring spiral seams and zips (an early innovation James pounced upon); later he made ball gowns, utilising materials more commonly used for hats to hold out skirts in expansive volume reminiscent of 19th-century crinolines. Many garments were true one-offs, experiments in fabric to solve problems James held in his head: the dress with a single seam, trousers with a sloping waistband to accommodate a variety of sizes, his famous down-padded jacket of 1937, the first ever 'puffer', which became a cult item in the Seventies with the popularity of the duvet coat. Harold Koda, the curator of the Metropolitan Museum exhibition, and a James fan, calls it "the one that got away" – referencing the fact that, although the Met's archive of James's work is the most complete museum collection of any single designer's work, the jacket resides at London's Victoria & Albert Museum. Salvador Dalí called the jacket the world's first soft-sculpture.

However, that jacket – like so many of James's seemingly robust pieces – is remarkably fragile. James's innovative and experimental construction techniques have condemned many of his garments to a lifetime of delicate conservation efforts – almost like his mercurial personality condemned his career, pulling the legacy of his genius apart at the seams. "He's like Frank Lloyd Wright and Wright's early buildings," says Harold Koda. "If Wright had the opportunity to work with structural engineers today, his buildings wouldn't leak. But he insisted on doing what he wanted at a time where the technology hadn't caught up with his ideas. With James, it's very much a similar thing."

There is no house of Charles James today, and no chance of one ever being revived. James went bankrupt numerous times, ran through prodigious fortunes with remarkable speed, and ended his days in the Seventies at New York's Chelsea Hotel – albeit still designing. He worked with his friend, Halston, in 1970: Halston told New York magazine in 1978 that he had asked James for help when his garments "fell apart at [the American department store] Marshall Field". James agreed, but with typical tenacity added the caveat that the dresses be labelled 'Shaped by Charles James'. That didn't happen. Nevertheless, Halston supported James during his final years – although on the wall of James's studio was a selection of garments he alleged Halston had stolen from him.

James's story is no rags-to-riches. It's the exact opposite, a tale eerily similar to that of the hugely influential pre-war couturier Paul Poiret, who died in abject poverty in 1944. In the 1930s, when James first showed in Paris, Poiret is reputed to have declared, "I pass you my crown. Wear it well".

But like Poiret, at one point Charles James was the greatest fashion designer in the world. His clothing, painstakingly wrought, was sought after by some of the world's wealthiest and most discerning women: the likes of Austine Hearst, Millicent Rogers, and Dominique de Menil, who collected James's work in the same way they amassed fine 18th-century furniture or works of art.

That was very much how James looked on his own work. "I think when you study James you realise that even if you have that reservation [of calling fashion art], you can't deny that James was an artist," states Harold Koda. "That's how he approached his craft and that's what precluded him from being the kind of commercial designer we think of as a fashion designer." For James, the clients he created dresses for were a combination of custodian and canvas: they provided shelter, and a body on which to build his dreams. 'Dreams', however, gives the wrong impression. For all their effervescent femininity, the tactile layers of satin, tulle and velvet, James's ball gowns are wrought with mathematical precision. Despite their retrograde allusions, they are breathtakingly modern. The famous Cecil Beaton photograph of icy James ball gowns in a Second Empire interior has become an image of idealised mid-century female perfection, but there's something frozen about the figures, something monumental and sculptural, rather than sensual.

James found most inspiration in the natural world: he created gowns with obfuscating titles such as 'Petal', 'Rose' or 'Tree'. James abstracted nature at every step – the latter 'Tree' gown, for instance, has a corrugated bodice of pleated taffeta to mimic bark, and a skirt expanding like branches. It's inverted, perversely. Not as perverse as the uncredited gown he created for a Modess sanitary-towel campaign (photographed by an equally uncredited Cecil Beaton), where gathered fabrics recall a cross-section of female genitalia. "He's creating a body," says Koda of James's clothing. He doesn't just mean corsetry: the variation of the padding in his down-jacket, for instance, mimics human musculature to allow the wearer to move inside its sculptural form. It's corporeal couture.

Despite his obscurity, James's work has inspired generations of designers. John Galliano created a Dior haute couture collection in 2010 that paid homage to James – Christian Dior himself had stated that James's ball gowns influenced his world-famous and game-changing New Look of 1947. Rick Owens, the American sportswear designer, has created collections that consciously reference James's work, while his unusual approaches to fabric, shape and proportion have a root in the latter's thought processes.

The designer most evidently influenced by his work is the New York-based Zac Posen, whose sinuous evening dresses, architectonic tailoring and, indeed, wide-skirted gowns are modern bearers of James's aesthetic imprint. Posen himself declares, "It's the aura of tailoring, rigour, patience, and trust towards the cut and the fabric that makes Charles James's work perennial. It's the dedication to the creative process that should be a lesson for every designer in the industry. Style is extremely personal and ever evolving; but how style is rendered into physical creations is through techniques that need to be mastered."

The technique and dresses are one side of the James story. That is the side the Metropolitan Museum of Art focuses on, extracting them from the designer himself, about whom apocryphal tales abound. They can distract from the work, but never overshadow it – as was the case in the late Forties, when James decreed that the model he had engaged to show his dresses to department-store buyers was inadequate, and chose, instead, to model the creations himself. Unconventional, certainly. But the department stores still bought them.