Fast and easy way to cut a long story short

Summarising a text is a key human skill. But can software do it? Roderick Neil Kay looks at progress

Automatic summarising has long been considered one of the most prized goals in artificial intelligence, or AI. Working summarisers have now finally appeared, although they are based on far more superficial techniques.

The ability to summarise a text is founded on core intellectual skills, so much so that the trials in the task still appear in numerous IQ and recruitment tests, including those run for the elite of the Civil Service.

Like many other products involving natural language, the new summarising technology doesn't quite live up to its billing. Summarisers have now been developed by a host of companies, including Oracle, InXight and BT, all based on similar statistical techniques. The simplest such method runs as follows. First, find the most frequently occurring words in a text, excluding trivial words. Second, locate the sentences in which groups of these words occur. Third, extract these sentences and compile them in chronological order. Then describe the compilation as a summary, without blinking.

One of the most surprising things about the statistical techniques at the centre of the new software is that the ideas have been around for a long time, almost before the dawn of AI, in fact. As early as 1958, while working for IBM, Luhn ran some experiments on a corpus of technical articles, using the algorithm just described. He was enthusiastic about the results: "The auto-abstract is perhaps the first example of a machine- generated equivalent of a completely intellectual task in the field of literature evaluation."

But at the time, text retrieval wasn't the hot topic it is today, and his idea was never marketed in the form of software. The view within the AI community, which has always aimed at getting computers to understand language, has been that statistical techniques are OK as far they go, but if you consider what a human can do, that isn't very far. The emergence at this point of the new summarisers probably owes as much to bumped-up demand as it does to advancement in the field.

While the recent crop of summarisers fall reassuringly short of human performance, their wide availability should generate the kind of interest which leads to improvement. And anyway, enough of human condescension; let us allow the computer to speak - or rather to summarise - for itself. The following extract is a 20 per cent summary of this article, produced by BT's Netsumm.

"Automatic summarising has long been considered one of the most prized goals in artificial intelligence, but working summarisers have now finally appeared based on far more superficial techniques. Summarisers have now been developed by a host of companies not normally associated with text processing: Oracle, InXight and British Telecom, all based on similar statistical heuristics. The view within the AI community, which has always aimed at getting computers to understand language, has been that statistical techniques are OK as far they go, but if you consider what a human can do, it isn't very far"n

Have you tried new the Independent Digital Edition apps?
Life and Style
ebookNow available in paperback
ebooks
ebookA delicious collection of 50 meaty main courses
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

ES Rentals

    iJobs Job Widget
    iJobs General

    Recruitment Genius: Business Development Executive / Digital Marketing Executive

    £26000 - £35000 per annum: Recruitment Genius: A luxury beauty house with a nu...

    Recruitment Genius: Housekeepers - Immediate Start

    £8 per hour: Recruitment Genius: This company are currently recruiting new exp...

    Recruitment Genius: Head Concierge

    £25000 - £28000 per annum: Recruitment Genius: This award winning Property Man...

    Recruitment Genius: Content, SEO and PPC Executive

    £20000 - £25000 per annum: Recruitment Genius: A fantastic opportunity has ari...

    Day In a Page

    On your feet! Spending at least two hours a day standing reduces the risk of heart attacks, cancer and diabetes, according to new research

    On your feet!

    Spending half the day standing 'reduces risk of heart attacks and cancer'
    Liverpool close in on Milner signing

    Liverpool close in on Milner signing

    Reds baulk at Christian Benteke £32.5m release clause
    With scores of surgeries closing, what hope is there for the David Cameron's promise of 5,000 more GPs and a 24/7 NHS?

    The big NHS question

    Why are there so few new GPs when so many want to study medicine?
    Big knickers are back: Thongs ain't what they used to be

    Thongs ain't what they used to be

    Big knickers are back
    Thurston Moore interview

    Thurston Moore interview

    On living in London, Sonic Youth and musical memoirs
    In full bloom

    In full bloom

    Floral print womenswear
    From leading man to Elephant Man, Bradley Cooper is terrific

    From leading man to Elephant Man

    Bradley Cooper is terrific
    In this the person to restore our trust in the banks?

    In this the person to restore our trust in the banks?

    Dame Colette Bowe - interview
    When do the creative juices dry up?

    When do the creative juices dry up?

    David Lodge thinks he knows
    The 'Cher moment' happening across fashion just now

    Fashion's Cher moment

    Ageing beauty will always be more classy than all that booty
    Thousands of teenage girls enduring debilitating illnesses after routine school cancer vaccination

    Health fears over school cancer jab

    Shock new Freedom of Information figures show how thousands of girls have suffered serious symptoms after routine HPV injection
    Fifa President Sepp Blatter warns his opponents: 'I forgive everyone, but I don't forget'

    'I forgive everyone, but I don't forget'

    Fifa president Sepp Blatter issues defiant warning to opponents
    Extreme summer temperatures will soon cause deaths of up to 1,700 more Britons a year, says government report

    Weather warning

    Extreme summer temperatures will soon cause deaths of up to 1,700 more Britons a year, says government report
    LSD: Speaking to volunteer users of the drug as trials get underway to see if it cures depression and addiction

    High hopes for LSD

    Meet the volunteer users helping to see if it cures depression and addiction
    German soldier who died fighting for UK in Battle of Waterloo should be removed from museum display and given dignified funeral, say historians

    Saving Private Brandt

    A Belgian museum's display of the skeleton of a soldier killed at Waterloo prompts calls for him to be given a dignified funeral