Deep Content: Advances in Indexing Visual Media
/You know that movie...the one with the explosions...and Tom Cruise...lots of action...What is that movie?!
If you asked that question a decade ago, you'd have a few sites and a Leonard Maltin book as your resources. Now, of course, you can just google it. But if the search engine fails you there's another option. A beta-stage site offering "Deep Content" searching might be able to help you out. The site is called, fittingly, "What is my movie?"
It prompts you to use conversational language, your own wording, to describe what you remember about the film. By way of extremely complicated algorithms, it produces an answer. I tested it with a number of vague phrases and so far it's been eerily spot-on. For example, I typed "a movie in which James Spader plays a rich jerk." The site gave me my answer: Pretty in Pink. I thought that might be too easy so I upped the game with this: "show me the movie about conquistadors that ends with monkeys on a raft." Much to my surprise, the site correctly gave me Aguirre, The Wrath of God. Amazing!
How does this work? Well, the engine was created by computer science researchers and engineers from a university in Finland. Here's what the site says about all the fancy tech involved:
“Whatismymovie.com has been developed by the tech team of Valossa that has its roots in the Computer Science and Engineering research conducted at the University of Oulu. We have an extensive research background on automatic content recognition and video data analysis. The demonstrations on this site have been developed for research purposes and Proof of Concept for the industry. Deep Content technology has also been piloted with the broadcasters for TV content.”
"Video data analysis" is what piques my interest. I know that programs have become able to recognize certain shapes with startling accuracy (this goes beyond Facebook recognizing faces).
The site says a little more about "Deep Content" as well.
"Deep Content technology can be used to discover and recommend semantically related content from large video collections."
This explains what it does, but not how. What is certain, however, is that we are entering a new stage in terms of indexing visual data. I can see great potential for this technology, particularly for students who want to search for visual details across an entire film, or an auteur's oeuvre.