Jinja to Visualize Shot Threads and Scenes

Jinja2 (docs) – a template rendering system for Python – is quite useful to visualize results in a quick and consistent way. In this post we see some images (frames of a video) which need to be presented in a repetitive structure. In case you do your research work in Matlab this is also easy since you can export your data as a mat file and load it up in Python with scipy.io.loadmat() (follow these instructions to make it easier).

The main idea is
import jinja2
template = jinja2.Template(open("path_template").read())
data = loadmat("path_to_matfile")

In my case of working with TV series and movies, one popular problem is shot threading. TV series and movies are typically shot from multiple cameras, and are heavily edited. Reverse engineering the editing can help put together shots from the same camera, which then goes on to help tasks such as person identification (they might be the same person!).

Threaded Shots from different cameras capturing a scene

Shots from different cameras capturing a scene

Visual analysis of the threading results can be quite hard due to the massive scale of the problem (imagine 700-1000 shots per 40min episode, and 20+ such episodes in a season). In the example above, we use Jinja to generate an HTML file with tables and links to the images. We see that the shot number below the image is staggered and indicates the editing pattern (click to view larger image and actually see the numbers).

Another popular problem is to divde the video into scenes, typically defined as a set of shots at the same location with the same set of people. Again visual analysis of the scene break detection algorithm yields itself very well to a template rendering system.

Scene break detection (5 shots per row)

Scene break detection (5 shots per row)

We see a correctly inserted scene break here (yaay!) as the characters in the story have changed.

Just as a final note, Jinja can do any form of template rendering. It is not restricted to HTML. Automatically generated LaTeX and then PDF files are certainly something to try 🙂


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s