Kitten videos: an engineering approach
29 Oct 2015
Python Rust WebOriginally posted at https://tech.labs.oliverwyman.com/blog/2015/10/29/kitten-videos-an-engineering-approach/
A little while ago one of my colleagues put up a kitten video feed on one of the spare monitors in the office. This was liked, but as we rapidly found out there was a few problems. Biggest of which is that kittens have a fairly well known failure state: they fall asleep. Sleeping kittens are adorable, but kinda boring, especially from a distance, and even more so if they fall asleep out of camera. I changed over the feed a few times to a more active bunch of kittens, but a more inventive approach was really needed. Also, apparently our blog is lacking in kittens, which is reportedly one of those popular things on the internet, and so this needed rectifying.
Enter kitling aka “Moving Kittens as a Service”. It’s a horribly over engineered approach to noticing moving kittens (or any other video feeds you want to hand it). It consists of the following architecture
All of the boxes (other than “the internet” of course) are Docker containers (which are autobuilt onto Docker Hub incidentally). The frontend lets you edit lists of kitten videos, and display the most moving video of any of the lists. The “imager” has a headless Firefox instance which lies through it’s teeth claiming to be an iPhone (so we can get the Flash-free interface), and given a URL of a video page gets a snapshot of the current video. The “comparer” sits between the two, checking the database for videos we haven’t checked recently, asking the “imager” to get a snapshot of them, and then generates a perceptual hash that we can use to get a “motion” value for indicating the difference between the last snapshot and the current one.
I built the frontend with Django mainly because the admin gets me a nice interface for editing data (which given I haven’t gotten around to building any other editing tools yet is very useful); the imager is built with Falcon because I’d normally write this sort of thing with Flask, but apparently Falcon is 8x faster, so wanted to test it out; and the comparer is written in Rust because I wanted an excuse to actually write some Rust (which turns out to be pretty good to work with). So, all in all a typical LShift side-project, with 3 times as many frameworks and twice the number of languages most people would use (and that’s before I count the Javascript changing the kitten feed on the frontend).
Currently our current favourite feeds are probably the TinyKittens ones. I’m on the lookout for tortoise feeds, but that’s mostly because I’m a fan of Clyde. At the moment, this isn’t running on the public internet, partially because I haven’t written an editing frontend, but also because it scrapes public websites and locking it down sensibly would take a bunch of work (also there’s an annoying bug where the video feeds sometimes just fall over and detecting that within the Flash interface is hard). However, if you want your own, either clone kitling and run “docker-compose up -d” to rebuild everything from scratch, or use the below docker-compose file to use the images from Docker Hub.