Automatic Keywording And Robots Which Do The Can Can - How Long Will We Wait?

A couple of years ago I went to an industries fair to see the Honda Asimo robot that was the envy of the world, the product of millions of dollars and millions of hours of research and development. Imagine my surprise when I discovered its greatest party trick was to walk up stairs without hanging onto the rail. One of my teenage sons summed up the experience with the words "big deal" as we filed out of the auditorium.

I've been taken back to that day by some recent musings by colleagues about the future of automatic keywording, where computers can read an image and keyword it.

Here are some examples of state-of-the-art automatic keywording from ALIPR as selected by John Harrington in an excellent article in Photo Business Forum late last year:

In the first example the automatic system had a good stab, although the keywords are far more limited than the sort of thing which comes across my computer screen every day.

In the second example the result is more comedic than keywording. Anyone working at Keedup who regularly added words like "dinosaur" and "firearm" to a picture of a boat wouldn't be kept on very long. Admittedly the vocabulary being used in this development project was limited to a few hundred words, but coming up with completely wrong words shows this technology is in a similar league to that Honda robot, or worse.

It's been suggested to me that a more immediate and successful application of this technology is in the use of image recognition software to help researchers find similar or related objects, thus adding a new system of researching which avoids or supplements keyword searching. I went to the ALIPR site and got the following "similar" images which the system found:

I don't know about you, but a boat and a cow don't really seem that similar. There were some images which were more alike for sure - including at least one with a boat in it - but this is a little less impressive than I'd expected.

So at present everyone agrees this technology has a way to go.

The questions to be asked are how long will that journey take, and will they get there at all?

I would also like to know in the context of automatic keywording just what "there" means? If the ALIPR project had produced perfect results in the boat image above, that would be progress, but what about concepts, sayings, colour of meaning, working out which object is most important (most prominent in the frame isn't the same thing), prioritising keywords and interpreting emotions in context with other objects in the frame.... The list goes on and on.

What researchers will need to bear in mind is that for automatic keywording to be really useful it will have to be as good as a human being, which is an incredibly tall order. It's hard to imagine why anyone would spend the sort of money that will require, except as a byproduct of some other, bigger endeavour.

Honda may have spent millions on getting a robot to walk up stairs, but at least we can see the huge market potential of a robot even if it doesn't dance the Can Can.