Feb 1, 2013

Kinect realtime hand & finger joints tracking

This just a quick performance/accuracy test of the HandTrackingLib provided by FORTH. The test was done by just running the demo that you can download here: http://cvrlcode.ics.forth.gr/handtracking/




The first thing we noticed is that optimum tracking depth range seems to be quite narrow, as you can notice at the calibration process before the tracking starts (you have to match the initial pose quite well).
The second one is that the modest specs of the hardware used heavily affect the effectiveness of the tracking, probably because the time that it takes to make the calculations some frames get lost, thus making bigger differences/distance between consecutive analysed frames.


The library is available in the OpenNI middleware section, unfortunately only for the 2.0 version of OpenNI's library, that only works with licensed SDK devices (Asus Xtion and Kinect for Windows, but not regular XBox Kinect).
We will try to make it work modifying the provided examples, feeding the tracking library with data from freenect, and letting it use OpenNI 2.0 just for the calculations.

You can get more info about the library here: http://www.ics.forth.gr/~argyros/research/kinecthandtracking.htm

9 comments:

  1. Hi David,

    I am doing my final year project, and I tried this library once. The demo ran well, but when I tried the library with sample code, it was stuck at the initializing instance forever. Have you tried the library? if yes, did you encounter anything like that?

    Cheers,

    T.

    ReplyDelete
    Replies
    1. I encountered problems my self because it is ment to be used with OpenNI 2.0, which works only with licensed sensors like the Xtion or Kinect for Windows. However, the people from FORTH that developed the library told me it should work with the XBox Kinect modifying their example to use OpenNI 1.5 for capturing and pass the captured frames to the HantTracking library, which would then use OpenNI internally for the tracking calculations. I started to do that but, after running the demo and seeing that our workstation's CPU/GPU wasn't enough to get a fast tracking response, I forgot about it.

      Regards,

      David

      Delete
  2. Dear David,

    I use a Kinect for windows with OpenNi2 myself, but still stuck there. Our lab's CPU/GPU is just so-so too, but I hope to get some results before ordering a better GPU. I think I'll try it later. Thanks very much for your reply.

    Best regards,

    T.

    ReplyDelete
    Replies
    1. Dear Toan,

      For your interest I copy the email kindly sent from the FORTH team, in case it may help:

      "Dear David,

      You can still use the library available for OpenNI2.0 independently. The
      API of the library requires you to just supply a depth and RGB image at VGA
      resolution (640x480) with the depth mapped on the RGB.

      The sample and build system provided are for the OpenNI2.0 framework but
      you could adapt them to work with OpenNI1.5.

      You will have to open your camera using OpenNI1.5 grab the depth and rgb
      image from the Image and depth generators and supply them to the Hand
      Tracker library.

      The sensor parameters like zpd and zpps can be set using the
      setSensorParameters method of the HandTracker class.

      kind regards,
      --
      The 3D Hand tracking Library team."

      Delete
  3. Hello
    I am having the same issue, having copied the OpenNI2/Drivers into the project directory, I am now receiving the error 'DeviceDriver: The library handle is invalid for file Kinect.dll". This occurs for all of the driver files. Do you have any idea how I can resolve this? I realise this article is two years old!

    Many thanks

    ReplyDelete
    Replies
    1. Hmmm... sorry, as you mentioned, it has been a long time and I just tested the library's functionality once. Microsoft recently published a related article:http://research.microsoft.com/en-us/projects/handpose/, don't know if there is any library though.

      Delete
  4. Hello
    I am having the same issue, having copied the OpenNI2/Drivers into the project directory, I am now receiving the error 'DeviceDriver: The library handle is invalid for file Kinect.dll". This occurs for all of the driver files. Do you have any idea how I might resolve this problem? I realise this article is two years old!

    Many thanks

    ReplyDelete
  5. I had the same problem. This is how I solved it:
    I copied all the files located in "C:\Program Files\OpenNI2\Samples\Bin"(you may have OpenNI2 installed in a different directory) and placed them inside the folder where my executable was located. Then I ran the executable and everything worked out just fine. Of course, you don't need all the file inside the Bin folder. For instance the files relevant to the examples located there are not needed.
    Note: I did this in Windows 8.1 and I'm working with vc10

    ReplyDelete