Behind the Scenes with the Kinect Train Build

It’s been a little while since the last Christmas project, so it was time to do something again.

This year, I started out with the following criteria for the Christmas for the City display:

  • Something interactive
  • Something that gets more fun as more people join in
  • Something that lets groups of people compete

Concept

The weekend before, my wife came home with a Nestle model train she got at a toy sale for $20:

NestleTrainSet

ABBButtonThis was great - trains and Christmas go together, and if the train could be driven by some input, we have the beginning of an interactive display. My plan at this point was to mount a series of industrial buttons, and wire them to a Netduino. The faster more kids pushed buttons, the faster the train would go.

However, I was running a little low on time at this point, so I needed something a little simpler. I was thinking of what might be possible while I was watching TV, and noticed my Kinect sitting under the TV. That was perfect. The more motion in front of the depth camera, the faster the train goes.

Measuring Motion

Using the Kinect library, I wrote a small app that grabbed each camera frame and passed it into an AForge.NET motion detection library. This gave me a motion analysis similar to this picture, where red areas represent “things that have moved in the last second or so”.

AForgeMotion

By counting the number of “motion pixels” each second, I ended up with what I coined to be the “motion number”. This could be 0 to 150,000 to 600,000 depending on how much motion was happening. I translated this into a percentage based on the max value seen since application start. So now we have a “motion percentage” from 0-100%. This will more or less serve as the value we need for the train throttle.

Mounting the Train

Trains, sensitive electronics, and anything else not bolted down or secured don’t last very long in front of kids, so in order to keep the train from being kicked, shoved, punched, or stolen, it needed some kind of enclosure.

LHShippingBox

For this I used some Bosch Aluminum Profiles – this stuff is amazing, we use it for just about everything. Our awesome distributor we use at work fabbed/cut pieces for me in a day and then it was off to the assembly area – the area between the couch and the dining room table. Winking smile

TrainFrame

This is also the point where I found out our new puppy hates portable drills..

PixelDeWalt

OK, so one problem.

I set up the Nestle train and… it didn’t work. Either the engine or the controller didn’t work. Still not sure what the issue was, but I had come too far now to abandon this one.

This was Saturday. I had the rest of the night and Sunday to finish this thing. I called The Train Loft in Winston-Salem Saturday evening. They were closing in 15 mins but Jeff stayed late so I could come get another train. They had an amazing display – not something I was going to attempt:

TrainLoftDisplay

Anyway, Jeff hooked me up with a Polar Express train set, and tricked up the track a little bit with a figure-8 instead of just the oval. Project officially over-budget at that point, but hey, Polar Express and Christmas go together, right?

Controlling the Train Throttle

What I thought was the easy part actually became the hardest. Turns out that train voltage is weird in a lot of ways (this train used AC, not DC), and controlling it via a computer is not that easy. I looked at various options for this, even trying to use an AC dimmer limited in software as to not overload the train. This got risky quickly, so I opted for a more simple mechanical control on the actual train controller. I pulled out the Lego Mindstorm NXT programmable controller and modified the train controller handle to attach to NXT…

The before:

LionelController

The after:

LegoController

While it looked cool, the “conveyor track” had a lot of slippage in it. Had to modify it further to couple the servo motor directly to the controller, and then I was in business.

Controller2

Making it into a Game

So, now I had a “motion percentage” to use as a throttle value, a way to control the train via software, and a mounting rig.

Now, I just had to make it into a game. So I wrote an app with this workflow:

  • Plays a Polar Express intro clip with the instructions (“move your body to make the train go”)
  • Gives a 3, 2, 1 countdown
  • For 30 seconds, enables Kinect, plays a Polar Express theme song while the train runs, displays the “motion percentage” in 0-400mph (yes, not really to scale, but hey…), and displays the “distance traveled” by the train
  • After 30 seconds, stop train and display high scores (how far the train went in the 30 seconds)
  • Repeat

Deploy!

I wrapped all that up, and our awesome Christmas for the City volunteers helped set up the rig at the convention center. Here are some pics of what it looked like:

TrainRig1

TrainRig2

TrainRig3

TrainRig5

Overall, this worked out great. It was a self-contained rig most of the time running one “round” every minute. Kids participated in nearly every round in the 6-hour period during the event. We had groups of kids competing, groups of adults competing, and a few “die-hards” with sweat pouring down their faces, and most of all a ton of smiles.

One of my favorite parts of the night was when one kid came over and played a few rounds, and then went to see Santa across the room and asked him for “a Polar Express dance game just like that one”.

All in all, I think I achieved the initial objectives, and combined some great hardware components together in a short amount of time!