Use Kinect in TouchDesigner
TOP and Chop
There are generally two different ways of utilizing Kinect in TouchDesigner, with Kinect TOP and Kinect Chop. Kinect TOP offers all image sensor data in a TOP. Kinect CHOP provides skeletal tracking data through different channels in a CHOP.
TOP
The image data can be accessed through Kinect TOP or Kinect Azure TOP respectively.
Below is an example of using creating a colored point cloud from the depth and color sensing image from Kinect. The project file for this example could be found here.
CHOP
Kinect CHOP offers the skeletal tracking data for each joint and its x/y/z position in different coordinate space through different channels. Use Select CHOP to select the channel you need, Pattern Matching is helpful in filtering and selecting multiple channels with conditions.
Multi-player Tracking
A common problem of using Kinect skeletal tracking in an installation is about finding the right person to track. When you need to track only one person, setting Max Player to 1 will pass the handling of player selection to Kinect, and most of the time it will lock on to the closest person and there is no way to manually switch player. When there are more people in the Kinect tracking space in an installation, this could be an problem.
A good approach is to keep the Max Player at Max, and create cuntom logic to filter and select the player you wanna track. Every time Kinect detects a new player in the frame, they will be assigned a p*/id. You can use id to keep tracking locked on the same person, no matter the position and how many players are in frame.
For each player, you can use the x/y/z positions from the head joint (or any other joint) p*/head:* to caculate its reletive distance to any point in space. And use math to draw a boundary or create sorting logic, you can map a physical point in real-world to a point in Kinect coordinate space, so Kinect only use the tracking from the person standing right on the point.
Below is an example of selecting a player based on the realtive position to the vertical center of the frame x = 0 and a point x = 0, z = 3. The project file could be found here
