At Bitcraze we have some history with trying to fly our Crazyflie autonomously. The most recent step is the Loco Positioning System that allows us, and you, to fly in a full room. The Loco Positioning system has boosted development of advanced algorithms for onboard position estimation and control.
Our earlier attempts where mostly based on different kinds of cameras, either a 3D camera like Kinect or regular webcams. Though, at that point, we only had the camera for position estimation and where doing the position control on the PC and not onboard the Crazyflie. This has the disadvantage to be brittle and requires a very high quality positioning from the camera: any frame where we loose the Crazyflie has a huge impact on the control behavior since the position controller relies exclusively on the camera detection.
With the Kalman filter and onboard position controller, the Crazyflie can now handle lost position information for at least a couple of seconds without big problems. This has the potential of making webcam-based position detector much more robust!
Using the new position-hold mode recently introduced in the client we can test and fly the Crazyflie under the webcam. We have taken a short video to show the performance. The result is promising and we will continue to play with ways to fly the Crazyflie autonomously.
Early on when we started to work on the Loco Positioning system, we came up with an idea of a Crazyflie autonomously flying into a light box, positioning it self for a few product pictures and then flying out again. The positioning system is now pretty mature and close to leave Early Access and this Friday we finally got around to do it. In this blog post we will share what we did and it also doubles as a brief howto on how to set up the system and fly a simple autonomous sequence.
We used a Crazyflie with a Loco Positioning Deck and eight Loco Positioning Anchors in our setup. Six anchors would have been fine too, but we happened to have eight in our flight lab.
When working with the Loco Positioning system the first step is always to make sure the anchors are set up correctly. We had an experimental version of the anchor firmware so we started out by pulling down the latest stock version of the source code and compiled it into a .dfu file. After that we fired up the brand new lps-tool that is used to flash firmware and configure the anchors. The anchors must be connected with a USB cable to the computer but the lps-tool reduces the flashing and configuration into a few clicks. When all anchors were updated we were ready for the next step.
The positions of the anchors are stored in the anchors them selves and the position is transmitted to the Crazyflie as a part of the ultra wide band messages used for measuring the ranges to the anchors. This way, the Crazyflie gets both anchor positions and ranges in the same process and has all the information needed to calculate its position. The second step is thus to store the positions in the anchors. In our “flight lab” we have fixed mounts for the anchors with known positions, so we could skip measuring the physical positions of the anchors.
We are working on making it possible to remotely configure the anchors to reduce the need to physically connect to them, and the position can now be set from the Crazyflie Client. We simply opened the “Loco Positioning” tab in the client, connected to a Crazyflie (with a Loco Positioning deck mounted), entered the anchor positions and hit the “Write to anchors” button. A few seconds later the anchor positions in the graphs were updated to indicate that the positions have been written to the anchors and then subsequently sent back through the ultra wide band messages to the Crazyflie.
Step three is to verify that the system is working as expected. First thing is to check that we did not mix the anchors up when configuring or placing them. In the “Loco Position” tab in the client, click the “Anchor Identification” button. In this mode anchors are lit up in the graphs when the Crazyflie gets close to them in the physical world. We went from anchor to anchor with the Crazyflie and checked that the correct anchor lit up on the screen. When confident that all was good we changed to “Position estimate” mode and verified that the estimated position matches the physical position of the Crazyflie. We have found that it can be very hard to understand, for instance that two anchors have been mixed up, by looking at the estimated position and that the “Anchor Identification” step simplifies the setup.
At this point we had a fully functioning Loco Positioning system ready for autonomous flight!
Now it was time to script a sequence. The easiest way to script a sequence is to start from the autonomousSequence.py example. Our intern Alfred took over at this point, he updated the uri to the correct settings and crafted a sequence to take off, fly into the light box, wait a while and then fly back out for a stylish landing in his hand!
Now we were ready for the actual photo shoot and Björn came down with the camera to shoot the product pictures. We hope you enjoy the results!
Crazyflie has been used by hackers and researchers as an autonomous flying platform for a while, though it has required expensive or very special software and equipment. Universities and the industry have, in most cases used motion capture systems that are very precise but also comes with a hefty price tag. We have developed the Loco Positioning System as a more accessible way to achieve autonomous flight with the Crazyflie. Bitcraze has worked intensively with autonomous flight and together with the community, we have added a number of advanced algorithms to the Crazyflie firmware that not only serve the Loco Positioning System, but are useful for anyone interested in autonomous flight regardless of positioning system. The functionality is useful for anyone from researchers with motion capture systems to hackers with a couple of webcams!
Lately there has been a lot of activities around control and stabilization algorithms. The main piece of software that has been merged into the Crazyflie firmware is the Extended Kalman Filter. With it, the Crazyflie uses its inertial sensor together with positioning information in order to calculate an estimated position. One great advantage of the Kalman filter is that it can handle much lower quality or intermittent position measurements and still fly autonomously. Of course the better the position information, the better the flight performance, but things like camera tracking with occlusion becomes possible.
There has also been a lot of work on a new controller. The controller is the piece of software that looks at the position estimate and calculate control commands to reach a given set point (for example go to {x;y;z}). We are currently working on adapting the firmware architecture to merge a more advanced control algorithm that supports complex trajectory following capabilities.
Finally there is a new commander architecture that adds the possibility to send complex set points to the Crazyflie via the radio. For manual flight all you need is 3 angles, roll/pitch/yaw, and the thrust. For a more advance controller you might need position, velocity and acceleration in X/Y/Z. For even more complex trajectories, we are working at merging a sequence generator to which we can send high level setpoints like spline coordinates. The sequence generator will calculate the required setpoints onboard in real time to follow the trajectory.
All these algorithms are currently work in progress, they are in various stages of completeness, but for example we have had the Crazyflie following circles in our lab last week using the on board sequence generator, so things work. However getting the code to work is only the beginning, until it is in the master branch it does not exists. Once the code land in master, like the Kalman filter did a while ago, it will be useful out of the box for anyone that wants to work with autonomous flight using any kind of localization system: Loco Positioning, Motion Capture, GPS, webcam, etc …
Bitcraze is not organized as most other companies, we are self organizing, strong believers in continuous improvement and are minimizing planing to be as agile and flexible as possible. We have written a few blog posts about this earlier. One result of this philosophy is that we don’t have a long term plan or road map to share, for instance of when a particular product will be released, but never the less we will tell you a bit about what we think lies ahead of us for the Loco Positioning system.
Our goal for the coming weeks is to finalize the first version of the positioning system, that is to leave the Early Access phase. The capabilities of the first release will be to autonomously fly one Crazyflie using two way ranging. The more advanced features such as TDoA will be considered experimental and requires compile time flags to be enabled.
We feel that the performance of the system is reaching levels that we think are good enough for many use cases, what is still lacking is ease of use. To fix that we are focusing on simplifying installation and configuration of the system by adding a few new tools.
We have found that one problem area is to install the system and get the anchor positions right. If the positions are not correct the estimated position of the Crazyflie will of course be wrong and it can be hard to understand what the cause of the problem is. To solve this we have added a new tab in the PC client (the LPS tab) that allows the user to see and configure the anchor positions as well as see the estimated position of the Crazyflie. There is also a mode in the client that is used to identify anchors by moving the Crazyflie around in the room, when close to an anchor that anchor lights up in the client to verify the setup.
Loco Positioning Tab
The anchor positions have up till now been stored in the Crazyflie or the client (ROS or python script), which is not optimal as data in the firmware or client becomes tightly coupled to the physical layout of the positioning system. If we move an anchor we either have to rebuild the Crazyflie firmware or have to transfer position data from the client to the Crazyflie before we can estimate the position. The solution is to move the anchor position into the anchor it self and send it as a part of the ultra wide band communication to the Crazyflie when ranging.
In the current Loco Positioning Node firmware, configuration changes and firmware updates are a bit cumbersome as it requires a few different external software packages. Further more different tools are needed depending on the OS of the host. To simplify this process we are working on a LPS-tool that will enable the user to configure and update the nodes using a GUI with clear feedback on the progress. The tool is written in python and the intention is that it will work on all our supported platforms.
We hope these improvements will lead to a positioning system that is easy to use and will enable all you people out there to do awesome stuff! As always, feedback is welcome.
We have spent most of our time working on the two way ranging in the Loco Positioning system lately, mainly on features that are not directly related to the actual ranging but such as making it easier to configure and upgrade the anchors. As a result we have not exercised the TDoA mode in a while, so on our Fun Friday we wanted to play a bit with that and try to fly a small swarm with some sort of coordinated autonomous flight.
Setup
We have a Loco Positioning system set up in the basement, we call it the flight lab to make it sound more fancy! The setup has been 6 anchors with three anchors in the ceiling and three on the floor, configured as triangles pointing in opposite directions. When using two way ranging that is fine as the positioning works pretty well outside the volume defined by the anchors. For TDoA on the other hand, the accuracy of the estimated position degrades rapidly when you go outside the convex hull. We decided to add two more anchors (to a total of eight) and arrange them as the corners of a box. A few hours and mounting/cabling later we were ready to try it out.
We modified the swarmSequence script to suite my (limited) space and set it up to fly four Crazyflies in a square, moving them to the next corner of the square every 5 seconds. Next problem was to find 4 working Crazyflies and Loco Positioning decks. We have a few Crazyflies lying around but a fair number of them have been modified in one way or another but finally we had the hardware we needed and could run the script. After a couple of failed tries (out of battery and such) we shot this video
Lessons learned
So what did we learn from this exercise? Adding two more anchors and changing anchor positions improved the positioning significantly. We have seen earlier that TDoA is less accurate than two way ranging, but better anchor positions reduces the problem. We could also fly the swarm using our example python script (not using ROS) without too much work and trouble, even though the TDoA mode still is very experimental.
In this flight we used the stock controller and just moved the set point to the next desired position for each copter. We are really looking forward to try out the improved controller and trajectory planning that we showed at Fosdem in combination with the TDoA mode, we think it will improve the performance a lot!
For the third year some of us from Bitcraze visited Fosdem, the biggest open-source European conference. Like the other years we enjoyed being there a lot and we had a great time hanging-out with community members like Fred.
Fred presented a great lightning talk about the news in the Crazyflie galaxy, the video and slide are already available.
Arnaud talked about the Loco positioning system. The talk and the demo, went well. Unfortunately the video from the talk is not available yet, we will tweet it and add it to this post as soon as it is online.
The Loco Positioning talk was a great opportunity for us to test the most recent bleeding edge additions to the Crazyflie autonomous algorithms. We flew the new non-linear controller from Mike Hammer using trajectory generation from Marcus Greiff. The non linear controller uses setpoints not only of position but also of velocity and acceleration to control the Crazyflie. This is where trajectory generation is useful: if you can generate a trajectory and calculate position, velocity and acceleration over time, you can feed all this information to the controller and the controller will be able to do a much better job following your trajectory. This enabled us to fly the Crazyflie fairly aggressively the week before the FOSDEM talk:
In this video the Crazyflie is accelerating to about 2g continuously to keep the trajectory. We were a bit concerned to fly such aggressive maneuvers in public without more testing so we designed a slightly safer demo the night before the talk in our hotel room:
This trajectory was successfully flown in the demo and shows the performance of this new controller. There has been a lot happening with the Crazyflie control algorithms lately: Marcus, Mike and Wolfgang have all made new controllers and Marcus has developed an on-board trajectory generator. There is still some work required in the firmware architecture to merge these into Master, but we hope this can be done in the coming weeks. Follow the Crazyflie firmware commits and github tickets if you are interested in the progress.
During the fall of 2016 fashion designer Maartje Dijkstra have in collaboration with music producer Beorn Lebenstedt (Newk) and engineer Erik Overmeire been working with the creation “TranSwarm Entities”, a dress made out of 3D prints accompanied by autonomously flying Crazyflies. The project was made during the Fashion Fusion Lab, a three-month workshop in which selected teams got to work on their fashion concepts. Maartje and her team used our Loco positioning system to enable 4 Crazyflies to do a “dance” around the dress during the show.
Copyright Fashion Fusion
Here is how Maartje describes the creation:
“The sculptural high fashion dress is totally build up out of small fragments (bird skulls), like cells building an organism.The parts are manual 3D printed and after printing all connected together by hand with polyester wires and green leather. The technology part is integrated in a special way. 4 small drones, that have given the same black 3D printed appearance as the dress, fly up from places inside so it looks like parts of the dress are flying away.The drones fly on the beats and melodies of music producer Newk around the model creating a little swarm. The shoes are digital 3D printed but finished manually.”
The finalists from the Fashion Fusion Lab got to compete during the Berlin Fashion Week at the “Fashion Fusion Challenge” and we are happy to announce that Maartje together with her team got the third place!
We at Bitcraze are very happy for Maartje and her team and think it’s very exciting to see the Crazyflie 2.0 and the Loco positioning system being used in such a different context. It shows again the potential for future applications and how versatile the Crazyflie and the Loco positioning system is.
In this beginning of 2017 we are proud to announce that there are two new decks for the Crazyflie 2.0.
The first one has been in the works for quite some time, it is the Micro SD card deck. It enables read and write access to a SD-Card from the Crazyflie firmware (where we have also implemented FAT filesystem support). Our first use case for this deck has been to implement high speed logging of the IMU sensors: the SD-Card has much higher bandwidth than the radio so it allowed us to log all the sensor values for later analysis. Another use-case could be to read an autonomous sequence from a file on the SD-Card and implement fully autonomous sequencing in the Crazyflie when used in the Loco Positioning System for example. The SD-Card deck is already available on Bitcraze web-shop.
The Second deck is the Z-Ranger deck, it is a laser time-of-flight ranging deck that measures the distance to the ground. We talked about this deck in a previous post. The manufacturing of the deck should be finished soon and so it will be available in our shop shortly. When using this new deck, the altitude hold stability between 0 and 1.5 to 2m height is greatly improved.
On a final note, FOSDEM 2017 is this coming up this weekend and we are looking forward to meet you there. There will be two presentations related to the Crazyflie, if you want to meet us come at these presentations or get in touch in the comment or by mail. The two presentations are:
Quite a lot has happened in the community in the latest month so we though we would dedicate this Monday post to you :-).
On the firmware side, the loco positioning system has sparked a lot of contribution. Most prominently is the Extended Kalman Filter by Mike Hammer and later improvement by multiple contributors. The Crazyflie is getting more stable and capable week after week which is awesome. Wolfgang from USC has also pushed enhancement coming from its CrazySwarm which will one day gives to everyone the capability to fly big Crazyflie swarm more easily.
On the clients side, we just pushed a new version of the iOS app to the app-store. The main improvement is the new tilt control mode implemented by EMart002 and beta-tested by a community member.
There has also been a new release of the Android client by Fred. This new version adds support for log and param using the Crazyradio. This way it is possible to get telemetry from the Crazyflie like the battery voltage and there is an experimental implementation of altitude-hold when using gamepad.
Running a beta (test-flight) version for the iOS client has been a good experience as it allowed to get direct feedback on functionalities. If there is interest we could release and announce beta versions for both Android and iOS in the future.
Finally last but not the least, there will be a new Crazyflie client in town: The Crazyserver created by Mike Hamer and written in go. It is a cross-platform, install-less, dependency-less server for a fleet of Crazyflies. It exposes a language-independent API, an HTTP rest API, to be able to connect and control any number of Crazyflie from any programming language. It will also include sockets (UDP, TCP and Websockets) to carry real-time data like setpoint and telemetry. It is still very much work in progress and not ready for real-world usage but if you are curious and/or would like to help check the code is on github.
For the third year we will be at FOSDEM on Saturday 4 and Sunday 5 February 2017 in Brussels. The last two years we have been present and Fred, the Android client maintainer, has had a lightning talk about the Crazyflie.
If you are planing at being at FOSDEM and want to meet with us just tell us in the comment, we would be really happy to meet Crazyflie users. There will be almost all of the Bitcraze team and we will have Crazyflie and a Loco Positioning System so it might even be possible to do some flight or demo on the side.
