Video Summarization is the Biggest Problem with Internet Cameras

What happens when you record every frame emitted by an IP-camera? You end up with too much data to make sense of. I now have nearly unlimited data storage, but have little interest in reviewing everything stored. Watching recorded footage in real-time is too time-consuming to be enjoyable, or even reasonable. While playing it back at 2x or 4x speed might sound like a good idea, that’s still a lot of video to look at.

Please Summarize!

Most video editing software offers a “scrubbing” operation for rapidly finding a point in time. Scrubbing is the act of manually moving the transport control backwards and forwards through the images. If you’ve ever scrubbed looking for a single frame you remember seeing, you’ll have noticed that it’s sometimes hard to find the frame. Your monitor is displaying no more than 60 or 75 frames per second: if you scrub over a time period with a resulting rate faster than this, you are not seeing everything.

“Video summarization” is a field of study aimed at developing algorithms and methods to help abstract and identify interesting features in a segment of video to help direct viewer’s attention. Here is a great quote describing video summarization [ 1 ].

Video summarization methods attempt to abstract the main occurrences, scenes, or objects in a clip in order to provide an easily interpreted synopsis.

At Sensr.net, we consider video summarization to be an important part of our technology, recognizing that keeping a collection of all the frames your camera emits is just too much data to use. Our summarization techniques are straightfoward: we use motion-detection¬†algorithms and save only those frames. We also offer a simple form of “hierarchical video-summarization.” When you look at a shot of all of the hours-of-the-day you are presented with the most important 24 frames of that day. Similarly, the days-of-the-month are summarized by the most important frame of each day.

Sensr.net has been hard at work laying the “pipes” for moving the frames emitted by internet cameras through our processors and into the cloud. You can expect to see more from us in the video summarization arena. Until then, take a look at this excellent slide presentation and think about what sorts of summarization you would like to see for your internet camera application.

[1] http://www.cs.utexas.edu/~grauman/courses/spring2008/slides/Video_Summarization.pdf