Fast and easy way to cut a long story short

Summarising a text is a key human skill. But can software do it? Roderick Neil Kay looks at progress

Automatic summarising has long been considered one of the most prized goals in artificial intelligence, or AI. Working summarisers have now finally appeared, although they are based on far more superficial techniques.

The ability to summarise a text is founded on core intellectual skills, so much so that the trials in the task still appear in numerous IQ and recruitment tests, including those run for the elite of the Civil Service.

Like many other products involving natural language, the new summarising technology doesn't quite live up to its billing. Summarisers have now been developed by a host of companies, including Oracle, InXight and BT, all based on similar statistical techniques. The simplest such method runs as follows. First, find the most frequently occurring words in a text, excluding trivial words. Second, locate the sentences in which groups of these words occur. Third, extract these sentences and compile them in chronological order. Then describe the compilation as a summary, without blinking.

One of the most surprising things about the statistical techniques at the centre of the new software is that the ideas have been around for a long time, almost before the dawn of AI, in fact. As early as 1958, while working for IBM, Luhn ran some experiments on a corpus of technical articles, using the algorithm just described. He was enthusiastic about the results: "The auto-abstract is perhaps the first example of a machine- generated equivalent of a completely intellectual task in the field of literature evaluation."

But at the time, text retrieval wasn't the hot topic it is today, and his idea was never marketed in the form of software. The view within the AI community, which has always aimed at getting computers to understand language, has been that statistical techniques are OK as far they go, but if you consider what a human can do, that isn't very far. The emergence at this point of the new summarisers probably owes as much to bumped-up demand as it does to advancement in the field.

While the recent crop of summarisers fall reassuringly short of human performance, their wide availability should generate the kind of interest which leads to improvement. And anyway, enough of human condescension; let us allow the computer to speak - or rather to summarise - for itself. The following extract is a 20 per cent summary of this article, produced by BT's Netsumm.

"Automatic summarising has long been considered one of the most prized goals in artificial intelligence, but working summarisers have now finally appeared based on far more superficial techniques. Summarisers have now been developed by a host of companies not normally associated with text processing: Oracle, InXight and British Telecom, all based on similar statistical heuristics. The view within the AI community, which has always aimed at getting computers to understand language, has been that statistical techniques are OK as far they go, but if you consider what a human can do, it isn't very far"n

Arts and Entertainment
Lou Reed distorted the truth about his upbringing, and since his death in 2013, biographers and memoirists have added to the myths
musicThe truth about Lou Reed's upbringing beyond the biographers' and memoirists' myths
News
people
News
Ed Miliband received a warm welcome in Chester
election 2015
Life and Style
Apple CEO Tim Cook announces the Apple Watch during an Apple special even
fashionIs the Apple Watch for you? Well, it depends if you want it for the fitness tech, or for the style
Have you tried new the Independent Digital Edition apps?
Life and Style
ebookNow available in paperback
ebooks
ebookPart of The Independent’s new eBook series The Great Composers
  • Get to the point
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

ES Rentals

    iJobs Job Widget
    iJobs General

    Recruitment Genius: Project Implementation Executive

    £18000 - £23000 per annum: Recruitment Genius: They work with major vehicle ma...

    Recruitment Genius: Chiropractic Assistant

    £16500 per annum: Recruitment Genius: A Chiropractic Assistant is needed in a ...

    Recruitment Genius: Digital Account Executive - Midlands

    £18000 - £26000 per annum: Recruitment Genius: They work with major vehicle ma...

    Recruitment Genius: Web Developer

    £28000 - £30000 per annum: Recruitment Genius: This company provides coaching ...

    Day In a Page

    NHS struggling to monitor the safety and efficacy of its services outsourced to private providers

    Who's monitoring the outsourced NHS services?

    A report finds that private firms are not being properly assessed for their quality of care
    Zac Goldsmith: 'I'll trigger a by-election over Heathrow'

    Zac Goldsmith: 'I'll trigger a by-election over Heathrow'

    The Tory MP said he did not want to stand again unless his party's manifesto ruled out a third runway. But he's doing so. Watch this space
    How do Greek voters feel about Syriza's backtracking on its anti-austerity pledge?

    How do Greeks feel about Syriza?

    Five voters from different backgrounds tell us what they expect from Syriza's charismatic leader Alexis Tsipras
    From Iraq to Libya and Syria: The wars that come back to haunt us

    The wars that come back to haunt us

    David Cameron should not escape blame for his role in conflicts that are still raging, argues Patrick Cockburn
    Sam Baker and Lauren Laverne: Too busy to surf? Head to The Pool

    Too busy to surf? Head to The Pool

    A new website is trying to declutter the internet to help busy women. Holly Williams meets the founders
    Heston Blumenthal to cook up a spice odyssey for British astronaut manning the International Space Station

    UK's Major Tum to blast off on a spice odyssey

    Nothing but the best for British astronaut as chef Heston Blumenthal cooks up his rations
    John Harrison's 'longitude' clock sets new record - 300 years on

    ‘Longitude’ clock sets new record - 300 years on

    Greenwich horologists celebrate as it keeps to within a second of real time over a 100-day test
    Fears in the US of being outgunned in the vital propaganda wars by Russia, China - and even Isis - have prompted a rethink on overseas broadcasters

    Let the propaganda wars begin - again

    'Accurate, objective, comprehensive': that was Voice of America's creed, but now its masters want it to promote US policy, reports Rupert Cornwell
    Why Japan's incredible long-distance runners will never win the London Marathon

    Japan's incredible long-distance runners

    Every year, Japanese long-distance runners post some of the world's fastest times – yet, come next weekend, not a single elite competitor from the country will be at the London Marathon
    Why does Tom Drury remain the greatest writer you've never heard of?

    Tom Drury: The quiet American

    His debut was considered one of the finest novels of the past 50 years, and he is every bit the equal of his contemporaries, Jonathan Franzen, Dave Eggers and David Foster Wallace
    You should judge a person by how they peel a potato

    You should judge a person by how they peel a potato

    Dave Hax's domestic tips are reminiscent of George Orwell's tea routine. The world might need revolution, but we like to sweat the small stuff, says DJ Taylor
    Beige is back: The drab car colours of the 1970s are proving popular again

    Beige to the future

    Flares and flounce are back on catwalks but a revival in ’70s car paintjobs was a stack-heeled step too far – until now
    Bill Granger recipes: Our chef's dishes highlight the delicate essence of fresh cheeses

    Bill Granger cooks with fresh cheeses

    More delicate on the palate, milder, fresh cheeses can also be kinder to the waistline
    Aston Villa vs Liverpool: 'This FA Cup run has been wonderful,' says veteran Shay Given

    Shay Given: 'This FA Cup run has been wonderful'

    The Villa keeper has been overlooked for a long time and has unhappy memories of the national stadium – but he is savouring his chance to play at Wembley
    Timeless drama of Championship race in league of its own - Michael Calvin

    Michael Calvin's Last Word

    Timeless drama of Championship race in league of its own