Photography and painting have always looked over their shoulder at the other. There is barely a show of a modern artist – viz David Hockney, Lucian Freud and Gerhard Richter – which hasn't a description of how the artist has set out to prove that painting can reach the parts that photography does not dare to go.
But then photographers have, since the beginning a century and a half ago, just as consciously sought to set themselves up to challenge and to surpass traditional painting. It's a theme currently explored by the Holburne Museum in Bath in an intriguing show of photographic still life. Right at the start, Henry Fox Talbot, the founder of the new medium numbered it as a genre where photography could and should show its worth. It's still going strong.
The first two pictures you are faced with in the Holburne show are a pairing of a dense but vibrant study of fruits and a spilled tankard from 1834 by Roger Fenton, the man who did more than any other to give the nascent technology the status of art, and an equally concentrated composition of a bunch of grapes between Javanese figures by the finest contemporary British photographer, Don McCullin.
For McCullin, still life and landscape were a deliberate sanctuary from the violence and pain of the war reportage for which he is best known. He tends to shrug off this side of his work as a "bit of a joke". He shouldn't. The fascination of this genre for photographers as artists is not just the sense of order but the concentration on craft and composition.
Fenton is clearly fascinated by surface and light and he arranges the objects with great care to elaborate the way that light falls on the rounded surfaces of grape and ivory. Switching from a lifetime of the shot taken in a fraction of a second to an arrangement taken with an exposure of several seconds, McCullin uses the dusty light of the shed in which the objects are placed on a table to give a certain dimness to the texture of the wall exposed behind his objects. The Dutch masters of still painting in the 17th century did the same, seeking precision with the fineness of the brush and the care of the varnish. .
It is this pursuit of technique that is so intriguing as you go round the show. Once photographers had moved beyond the painterly still life they soon found that, in photography, the closer you got the more majestic became the compositions. Ansel Adams, the great recorder of the American wilderness, takes the lens up against a bark with an almost architectural picture, Moth and Stump, Interglacial Forest, National Monument, Alaska (1949). Edward Weston is represented by an extraordinarily intricate and bold close-up of corroding paint in Abandoned Car, Mojave Desert from 1937 that makes it seem like a river pattern seen from the air. The results are even more spectacular when photographers started to explore the possibilities opened up by the microscope. Talbot did it very early on in Insect Wings, As Seen in a Solar Microscope from 1840, while the largely unknown William Henry James Coombs prefigures Mapplethorpe in a breathtaking Negative Radiograph of Tulip of unknown date.
Down that avenue painting found it hard to go. Where photography found it more difficult on the other hand was in imitating the graphic arts in playing around with perspective, time and viewpoint. Surrealism had a profound effect on photography, of course, not least in the real-life shots capturing the absurdity or pathos of objects caught in contradiction or commentary on one another. Yet when it came to playing tricks with the camera in the manner of Dalí or the Cubists the results were rather strained. Clarence White managed to use glass globes with surprising effect as bubbles in a symbolist Still Life of 1907 and there is a wonderful homage by photography to art in André Kertész's picture of Mondrian's studio, Chez Mondrian of 1926, the geometric lines of the spiral staircase cutting the composition as Mondrian did with straight lines. But it took 28 throws of three cats and a pail of water for Philippe Halsman to get the Salvador Dalí-inspired image of suspended animation, Dali Atomicus (1948) and still it looks staged. Photography is too realistic a medium to make fantastical at least until the coming of digital.
The Holburne titles this show, taken from photographs held in the National Media Museum in Bradford in Art of Arrangement, as a way of casting its net as widely as possible to include everything from portraits to photo-finishes. You can see their point. Too strict a definition would lose some of the spontaneity and adventurousness that has always accompanied this art form. But including the self-portraiture of Helen Chadwick and Madame Yevonde is too great a tangent. They might have been better to have kept to the subtitle: Photography and the Still Life Tradition.
Spreading the net this wide also misses one of the most interesting aspects of modern photography: the reinvention of the long-exposure shot in monumental form. If the show had been bigger one could have proceeded, with a side gallery on the erotic attractions of flower close-ups, to the massive pictures of factories, urban sites and horizons taken and magnified by contemporary artists. Looking at pictures by, say, the German photographer Thomas Struth and you are on a line that goes straight back to Fenton and, indeed, to the Dutch masters of the 17th century who made still lives into a genre of their own.
Art of Arrangement: Photography and the Still Life Tradition, Holburne Museum, Bath (01225 388569; holburne.org) to 7 May