Archive for May 2008

Rockband – with Computer Vision


Rockband Vision from njoubert on Vimeo.

http://www.vimeo.com/1045326

We built this computer vision system that can play Rockband as our final project for CS184 at UC Berkeley.

In the space of two weekends we designed and built a system that uses computer vision to monitor the xbox display through a camera and play the game. More details can be found at inst.eecs.berkeley.edu/~njoubert/cs184/Rockband.pdf

We were, well, sleep deprived for a good section of this work, which explains the craziness in some parts of the video.

Our system is similar to Slashbot and AutoGuitarHero, but we do not take a video feed from the console – no, we’re doing it through a panasonic handycam pointed at the screen. We’re interested in machine vision, and this was a fun project to get into the field!

Building OpenCV in Ubuntu 8.04

I’m using OpenCV for my current computer graphics project – hacking Harmonix’ Rock Band – so naturally I have to build it from source in Ubuntu. I downloaded the source from Sourceforge.

The procedure was fairly simple – the most important part was the packages needed to satisfy all the requirements. OpenCV depends on several other libraries to really get the full potential of our system (although simple installs are possible).

Since I wanted to do image input/output I apt-get’ted the following packages:

  • libpng-dev
  • libjpeg-dev

To do ffmpeg development – which is the library OpenCV uses for video campture:

  • libavcodec-dev – development files for libavcodec
  • libavformat-dev – development files for libavformat
  • libavutil-dev – development files for libavutil
  • libpostproc-dev – development files for libpostproc
  • libswscale-dev – development files for libswscale
  • libdlna-dev – development files for libdlna
  • libmpeg4ip-dev – end-to-end system to explore streaming multimedia

For all the funky GUI development:

  • libgtk2.0-0
  • libgtk2.0-dev

You can install all of this using audo apt-get install

Once this is done, I unpacked the TAR file, cd’d to the directory and ran the good old standard set of building commands:
<br />sudo ./configure<br />sudo make<br />sudo make install<br />

That’s it!