The major technological challenge is that today's computer technology mostly regards video as a black box whose semantic contents is inaccessible. Algorithmic processing of video content thus relies on auxiliary metadata, rather than the video itself.
The challenge is to extract and combine data from complementary sources into a high-level representation, and to cope with large video archives. This makes it necessary to find suitable forms of representation, and corresponding algorithms that are both efficient and highly accurate.