We do not recommend using Labs for installations or productions where dependability is required. These features are experimental and unsupported.

In this tutorial, we'll be using an open-source solution that allows you to control OSC values using Kinect Body Tracking.

KinectV2-OSC sends OSC messages for each body detected by the Kinect. We edited the code to add a feature that detects right-to-left hand movement and then sends an OSC message to the Lightform device to trigger the next slide.

Any other input device can be used instead of a Kinect (e.g., RealSense, Leap Motion, etc.). Here are some open-source solutions we found:

Note: This requires a bit of coding!

Requirements

Windows 8+ (we tested on Windows 10)

USB3 port

V2 Kinect sensor

Visual Studio 2013+ (we tested on VS 2015)

Step 1: Download and Run the KinectV2-OSC app

Clone the KinectV2-OSC repository:

cd into the directory you want the files to live in
git clonehttps://github.com/microcosm/KinectV2-OSC.git

Open KinectV2OSC.sln in Visual Studio. Run the application by pressing the green Start button. Verify that the application runs and you successfully see skeleton data. The screen should look like this:

Step 2: Create Gesture Detection

In this example, we detect a right hand moving from right to left and send an OSC message to trigger the next slide.

Here is the Kinect Body class: https://docs.microsoft.com/en-us/previous-versions/windows/kinect/dn772824(v%3dieb.10)

Step 3: Change IP Address and Port to your Lightform Device's IP Address

We'll need to make sure that the OSC message is sent to your Lightform device, so we'll configure the right IP address and port in the KinectV2-OSC properties file. Locate the IP address of your Lightform device in the Control Page in Lightform Creator:

Go to Properties -> Resources to change the IP and port number in the KinectV2-OSC resources file. The port number is 8000:

Step 4: Send OSC Message

KinectV2-OSC uses Rug.OSC to send OSC messages. You can find this usage in BodySender.cs and MessageBuilder.cs. Here is the OSC message we send to the Lightform device:

OSCMessage message = OscMessage.Parse("/next, 1.0");

this.Broadcast(message);

This sends the value 1.0f to the OSC address"/next", which triggers the next slide. You can find the full list of OSC mappings for the Lightform device in the Control Page:

Screen_Shot_2018-11-13_at_10.38.02_PM.png

Conclusion

This tutorial only just scratches the surface of how you can use the Kinect with the Lightform device. We're curious to see the ways you'll add more examples of this (e.g., body pose estimates with Live Effect parameters, etc.).

Comments

6 comments

Brahim Abouzaid

April 09, 2019 16:37
Hi

i try to use my kinect to send osc message but the step 4 is not clear for me , can you explain more clearly this step please .i'm not familiar with coding .

thanks
0
Brittany Factura

April 09, 2019 20:48
Hi Brahim,

This tutorial and specifically steps 2 and 4 will require some coding.

Step 4 outlines how to send an OSC message to the LF1 upon detecting a gesture.

After detecting a gesture, we want to tell the LF1 to advance the playing project to the next slide. To do this, we can send an OSC message to the LF1. We send the value 1.0 to the OSC address '/next'. The following line of code:

OSCMessage message = OscMessage.Parse("/next, 1.0");

sets up an OSC message to do so. Then, we use this.Broadcast(message)to broadcast the message to the LF1.

You can see an example of sending an OSC Message in BodySender.cs. For example, lines 77 and 78 in BodySender.cs:

77. message = messageBuilder.BuildJointMessage(body, joint);

78. this.Broadcast(message);

sets up an OSC message containing the joint positions of a detected body (see MessageBuilder.cs for details) then broadcasts the message [to an IP address]. So for our case, we set up a message for triggering the next slide then send the message using Broadcast(message).

If you take a look at line 181 in MainWindow.xaml.cs, you can see how this is all brought back together:

this.bodySender.Send(this.bodies);

This is where the message gets sent.

I know this all does require coding and reading through/understanding code. Really sorry for confusion. Hope this makes sense/helps.

Best,

Brittany
0
sanYuan Fu

July 08, 2020 02:27
Are there teaching videos

i'm not familiar with coding
0
Tyrone Castillo

July 30, 2020 14:36
Hi, I've bought a Kinect V2 second hand. How do I physically connect the Kinect to the LF2? The plug is a proprietary fro Microsoft.
0
Anum

July 30, 2020 16:21

Edited
Hi Tyrone Castillo, the Kinect will be connected to your computer using an adapter (easy to find online).
0
KyungHwan Kim

December 07, 2020 02:37

Edited
Hi. Could you make a tutorial on realsense?

I'm trying to do tutorial with realsense, but I'm not sure where to start.
0