Feb 24, 2013

RGBDtoolkit, wide angle lens and automatic calibration

I finally got a wide angle lens that let me match the FOV (Field Of View) of the Kinect, a Canon EF-S 10-22mm f/3.5-4.5 USM, borowed from the Fine Arts College facilities where I work, in the Universitat Politécnica de Valencia.
I made a small application as a proof of concept towards an automated Kinect - DSLR calibration process, and also for a possible further automated recording-download system controlling remotelly the DSLR via USB.

The top left picture is the live Kinect 1280x1024 IR stream, with an overlay reference to ease the framing of the calibration pattern. The top right picture is the DSLR (EOS 600D) video mode live preview, cropped to match the 16:9 aspect ratio of the video/pictures.

I made the overlay as best I could to show in just one picture the 18 optimal positions/sizes for the callibration board. A further automated callibrator development would detect the pattern in real time from the live streams and take the picture once it detects it on one of the positions/sizes not already stored, and also detect with frame differencing technique that the camera is still enough to ensure the quality of the picture to be taken.

On the bottom of the video you can see both pictures blended at 50% transparency to help with the manual parallax calibration and matching the Kinect's FOV. Note that both images are vertically displaced, as are the cameras. Measuring the real displacement could be used to maximize the phisical parallax between both cameras, if the software would be able to tell you the difference detected between the pattern positions in the images. You would have to input the size of the calibration board squares, and the application would calculate the distance between the centers of the boards of both images, wich would have to be matched adjusting physically the cameras' parallax.
The DSLR is triggered remotely by the application via USB / GPhoto, and the picture is transferred directly to the computer and downscaled to 1920x1080.


  1. Hey David! This is a great post! Thank you so much for the info. I found my way to this site searching for the best lens to match the Kinect.'s. You say the 10-22mm is the best lens, but what exact length are you shoot at? 10mm? 22mm? Which is the length that matches the Kinect?

    Thanks again for your hard work!

    1. Thanks Andrew!
      That is a good question. I digged into my data and found that the pics I captured with my app didn't save the metadata info about the lens length. Fortunately, I was able to retrieve the exif data from the MOV file with the help of exiftool (http://www.sno.phy.queensu.ca/~phil/exiftool/).

      The lens was at a focal length of 14.0 mm.

      This is the whole metadata: http://pastebin.com/DDzK47fb

    2. PS: Have you seen this other post?: http://therandomlab.blogspot.com.es/2013/03/logitech-c920-and-c910-fields-of-view.html

      You need a wider FOV for the video camera because of the reduction of field due to the vertical displacement of both cameras.

    3. PPS: The Pro-Optic 14mm f/2.8 Aspherical Wide Angle Lens looks as a good and affordable lens alternative (note 14mm is for APS-C sensor size).

    4. David! Thank you so much for this info! I guess that explains why my guess of 24mm failed miserably :) I will now try renting the 14 mm!! Thanks for all your research and meta data! I greatly appreciate it!

  2. P.S. Sorry for the late reply. I too thought google was going to notify me you had replied. But now I've subscribed!

    I'm currently battling my skills of shopwork and metal cutting to create the perfect mount. I've given up on my angled aluminum like the rgbd tutorial gives due to unstableness. I'm now trying to get a 3d mount printed. I hope this works. If it doesn't I'll try and build your version :)

    I've also noticed your mount doesn't have a way to angle the kinect. Has this not been a problem for you? Have you updated your mount since your original post?


    1. I've done some sketches to add tilting capabilities, but nothing definitive.

      Yes, I had problems to adjust the parallax... for the tests I just loosened the screws a bit and held the proper angle with a piece of cardboard :)

    2. PPS (We have had many of these by now)

      Love the cardboard trick :)

      Also, after re-reading your post about the sensor size, I realized that my sensor is different from your Canon T3i's. I'm currently using a 5D Mk III. I've been a photographer for many years, but I've yet to dive into the specifics and technology of sensors, which you seem to excel at :)

      To my understanding I have a full framed sensor which differs from yours. From what I hear, my lens will seem wider then yours, but in reality, your sensor just performs a cropping of the image, yes? Or am I wrong? Does this mean the 14mm may not be my match or ideal lens after all? Or will my cropping just be different?

      Gracias otra vez!

    3. Your 5D Mk III full size sensor has a crop factor 1.0x. You dont need a lens as wide as 14mm (see http://www.the-digital-picture.com/Canon-Lenses/Field-of-View-Crop-Factor.aspx)
      Canon's APS-C like mine have a 1.6 crop factor. So for your full size (35mm) sensor the equivalent is: 1.6x14mm = 22.4mm. That is what you need (or wider).

      Here is a good explanation:


      De nada!

  3. Perfecto!! Estas un genio. Muchas gracias!