Hands on Review of the Leap Motion Sensor with Developer Links

Posted on by

By David Bradway – Guest Writer

I was sent the Leap Motion device from the Average Guy Tech Scholarship Fund (Thanks Jim!). I thought I could test the device both as an Average Guy and an average coder. First up was unboxing the nice, Apple-like packaging materials.

Unboxing (Click and an photo to enlarge)

IMG_20151029_202016971
IMG_20151029_201916397
IMG_20151029_202159472
IMG_20151029_201957650

IMG_20151029_202215835

 

Software Installation

It was a simple Plug and Play setup, software download and install.

The Leap Motion software installs a desktop program called the “Leap Motion App Home” which contains some starter apps:

  • Leap Motion App Store
  • Playground, which is a set of training games. Put heads on the robots, Pluck flower petals, etc
  • Collider, an app to use with the Oculist Rift headset
  • Form & Function, a 3D anatomy browser
  • Google Earth gesture navigation integration

I tested out some of the apps, including Google Earth. After enabling joystick navigation in Google Earth preferences, I was able to fly around by moving, tilting and rotating my hand. I made a flyover video of trying to find Central Park.

 

 

Leap Visualizer

Leap Motion also provides a debugging tool which gives an inside look at the skeletal model for hand and finger tracking. There was a bit of lag on my system, and the frame rate I observed wasn’t that impressive. I image with a better CPU or GPU and USB3.0, this would look a lot better.

 

LeapVisualizer

 

Developer Tools and Application Programming Interface (API)

Leap Motion has a lot of good documentation for software developers and makers. There are example scripts and programs in several languages including Python. I modified the Python sample tutorial to do something different when the gesture ‘Screen Tap’ was detected. I then added an extra snippet to the Python code that enabled me to receive mobile notifications using Pushover. Pushover is a great multi-platform notification service which I purchased originally because of its IFTTT compatibility.

 

a9b6a404-0a52-41ab-b6a4-f998c7c8d6c1

Pushover has a nice, open API, and I registered an app I called ‘LeapEvent’ which listens for notifications from my Python program. Below is the screenshot from my phone of the detected gesture notification from my Python script. The “screen tap” Leap Motion gesture is a one-fingered poke forward in the tracked field of view.Pushover was used here an an endpoint my detected gesture. In the future, I might like my program to control some device like my Nest Thermostat or another smart device like lights.

 

Screenshot_2015-10-29-21-23-30

 

Leap Motion + 3D Slicer

My day job is in medical imaging research, so I wanted to try to replicate some YouTubes videos (1, 2) that used the Leap Motion device for navigating medical images in 3D Slicer. The project had a website with useful info so I downloaded the code and dug in. To use the Leap Motion device in 3D Slicer, I also needed to download the Visualization Toolkit, or VTK. I forked the project repository and updated it with the new version of the API and code. As shown in the image below, I displayed the tracked finger positions (F_1, F_2, etc) on some example Magnetic Resonance Imaging (MRI) 3-D data in Slicer, but I haven’t figured out how to control the visualization planes as in the YouTube video. If anyone has 3D Slicer experience, please send me a message!

 

SlicerScreenshot

 

Future Work

Some of the coolest uses for the Leap Motion sensor seem to be in conjunction with the Oculist Rift virtual realty hardware headset. The Leap Motion sensor can be mounted on the VR goggles, looking out into the room. It can then be used to detect hands, fingers, tools, etc and bring them into the virtual reality environment. I don’t have an Oculist headset, but I did pick up some super-cheap Google Cardboard-compatible viewers and heads straps from AliExpress in China. Some people have had success displaying tracked images using the phone display, so I might see if I can try that out.

 

Wrap Up

All in all, I think it is an interesting device, and when support is fully enabled in software programs, such as the Google Earth example, it can really enable unique and useful interaction. Without deep integration or a software development skill-set, people may be limited to the kinds of games and programs they find in the appstore. I didn’t do much exploration of the available paid apps, but there appear to some reviews available online. For someone with the skills to integrate this device into their own scripts and apps, I would enthusiastically recommend this device! For someone who just wants to play motion games, there’s a chance it could end up gathering dust after a few uses.