For the last couple of years (!) we have been discussing on and off about automatically flying the Crazyflie from a PC using OpenCV and a camera. We did a try a while ago using a PS3 Eye camera that wasn’t very successful. One of the issues we were having was detecting the distance from the Crazyflie to the camera. Another issue was a lag of about 1 second which made it impossible to control the Crazyflie using the video as input. So last week this discussion came up again and we finally decided to buy a Xbox360 Kinect. The image resolution is higher than the PS3 Eye and of course it has the ability to detect the distance from the camera to the objects it’s seeing.
The goal was to create a proof-of-concept application that shows that it’s possible as well as providing a stub for anyone that’s interested in doing more development. The application uses the normal image as well as the depth image from the Kinect to estimate where the Crazyflie is. This can easily be done by attaching a small colored ball to the Crazyflie and using a white background for flying. The images are processed and the current X/Y/Z co-ordinates given to a control loop. The control loop consists of thee PID regulators that will correct the roll/pitch/thrust that is sent to the Crazyflie to reach a X/Y/Z set-point in the image. The X/Y set-point can be selected by clicking the image or you can hold down the mouse button and drag it around. A cool feature that we would like in the future is to draw a geometric shape that the Crazyflie could follow. But there’s still a lot of work to be done with the control loop before we can achieve that.
After seeing the video you might be asking yourself where the James Bond theme music or the inverted pendulum is. Well, we are’t quite there yet and we will probably not even get near it. But still, this shows that the concept works. And for anyone that’s interested, it’s possible to do some basic trajectory planning algorithms at home with a Crazyflie and a Kinect.
For a more technical details have a look at the Kinect page in the hacks section of the wiki.