Use Microsoft Kinect to send OSC Messages

We can use the Microsoft Kinect to send OSC messages to the LF1. In this tutorial, we'll be using an open source solution that allows you to control OSC values using Kinect Body Tracking.

KinectV2-OSC sends OSC messages for each body detected by the Kinect. We edited the code to add a feature that detects right-to-left hand movement then sends an OSC message to the LF1 to trigger the next slide.

Any other input device can be used instead of a Kinect (e.g., RealSense, Leap Motion, etc.). Here are some open source solutions we found:

Note

This requires a bit of coding!

Requirements:

Windows 8+ (we tested on Windows 10)
USB3 port
V2 Kinect sensor
Visual Studio 2013+ (we tested on VS 2015)

Step 1 Download and Run the KinectV2-OSC app

Clone the KinectV2-OSC repository:

  1. cd into the directory you want the files to live in
  2. git clonehttps://github.com/microcosm/KinectV2-OSC.git

Open KinectV2OSC.sln in Visual Studio. Run the application by pressing the green Start button. Verify that the application runs and you successfully see skeleton data. The screen should look like this:

kinect-osc.png

Step 2 Create Gesture Detection

In this example, we detect a right hand moving from right to left and send an OSC message to trigger the next slide.

Here is the Kinect Body class: https://docs.microsoft.com/en-us/previous-versions/windows/kinect/dn772824(v%3dieb.10)

Step 3 Change IP Address and Port to your LF1's IP Address

We'll need to make sure that the OSC message is sent to your LF1, so we'll configure the right IP address and port in the KinectV2-OSC properties file. Locate the IP address of your LF1 in the Control Page in Lightform Creator:

ip.png

Go to Properties -> Resources to change the IP and port number in the KinectV2-OSC resources file. The port number is 8000:

kinect-osc-ip2.png

Step 4 Send OSC Message

KinectV2-OSC uses Rug.OSC to send OSC messages. You can find this usage in BodySender.cs and MessageBuilder.cs. Here is the OSC message we send to the LF1:

OSCMessage message = OscMessage.Parse("/next, 1.0");

this.Broadcast(message);

This sends the value 1.0f to the OSC address"/next", which triggers the next slide. You can find the full list of OSC mappings for the LF1 in the Control Page:

Screen_Shot_2018-11-13_at_10.38.02_PM.png

Conclusion

This tutorial just scratches the surface of how you can use the Kinect with the LF1. We'll be adding more examples of this (e.g., body pose estimates with Live Effect parameters...). Maybe you'll beat us to it ;)

Was this article helpful?
5 out of 5 found this helpful

Make a suggestion to improve this article

  • Comment actions Permalink
    Avatar
    Brahim Abouzaid

    Hi

     

    i try to use my kinect to send osc message but the step 4 is not clear for me , can you  explain more clearly this step please .i'm not familiar with coding .

     

    thanks 

  • Comment actions Permalink
    Avatar
    Brittany at Lightform

    Hi Brahim,

    This tutorial and specifically steps 2 and 4 will require some coding.

    Step 4 outlines how to send an OSC message to the LF1 upon detecting a gesture. 

    After detecting a gesture, we want to tell the LF1 to advance the playing project to the next slide. To do this, we can send an OSC message to the LF1. We send the value 1.0 to the OSC address '/next'. The following line of code:

    OSCMessage message = OscMessage.Parse("/next, 1.0");

    sets up an OSC message to do so. Then, we use this.Broadcast(message)to broadcast the message to the LF1.

    You can see an example of sending an OSC Message in BodySender.cs. For example, lines 77 and 78 in BodySender.cs:

    77. message = messageBuilder.BuildJointMessage(body, joint);

    78. this.Broadcast(message);

    sets up an OSC message containing the joint positions of a detected body (see MessageBuilder.cs for details) then broadcasts the message [to an IP address]. So for our case, we set up a message for triggering the next slide then send the message using Broadcast(message).

    If you take a look at line 181 in MainWindow.xaml.cs, you can see how this is all brought back together:

    this.bodySender.Send(this.bodies);

    This is where the message gets sent.

    I know this all does require coding and reading through/understanding code. Really sorry for confusion. Hope this makes sense/helps.

    Best,

    Brittany