The Bitcraze organization page on github contains some 60+ repositories and if you are looking for a specific piece of code it might be hard to know where to find it. In this blog post we will try to describe how the repositories are organized and hopefully make it easier to understand where to start your search.
The Bitcraze repositories contain code and information related to a wide range of applications and products, some might be of interest to many users, while others have a smaller audience. Examples are software for the Crazyflie and decks, but also things like simulation, tests, hardware and our web site. As we try to be as open as possible most of the content we produce will end up in a repository, which obviously increases the number of available repos over time. For most users there is usually only a handful of repositories that are of interest though so let’s create some order.
The main repositories
There are three repositories that are the hot spots of most functionality, these are usually the first place to look and the only repositories most users will ever use. You can find quick links to these repos in the “Pinned” section on the Bitcraze github start page.
The repositories are:
crazyflie-firmware – the source code for the STM processor firmware on the Crazyflie. This is where most of the important Crazyflie functionality is implemented, such as controllers, estimators, motor control and communication with decks. If you want to change a behavior in the Crazyflie itself, look here.
crazyflie-lib-python – the python library used to communicate with the Crazyflie. This is used to control a Crazyflie remotely. Use it if you want to use a script to control a Crazyflie, also contains lots of examples of how to do common tasks.
crazyflie-clients-python – a the python client that is used to connect to the Crazyflie. The client is simply a GUI that uses the python lib to communicate with the Crazyflie. Everything that is done in the Client can also be done by a script using the python library.
Another source of quick links to common and important repositories can be found on our web, on the Repository overview page. This page also contains short descriptions of the repositories.
Related repositories
Many of our products are implemented as multiple sub-systems, perhaps using different languages or technologies and running in separate hardware, in most cases we separate these sub-systems into their own repositories. The rule of thumb is that these repositories are named in a similar way to indicate that they are related, for instance lighthouse-bootloader and lighthouse-fpga that contain code for the lighthouse-deck. However note that in most cases there will also be related functionality implemented in the main repos as well, in the Lighthouse deck case for instance, most of the actual positioning functionality is in the crazyflie-firmware repo while Lighthouse system management is implemented in the lib and client.
Bootloaders
When powering up a CPU there must be a small piece of code available that sets up the basic configuration of the device to enable it to communicate with other parts of the system, like memories and such. This code is usually called a bootloader. Bootloaders are rarely changed and in most cases written to the device as a part of the factory production process. They have their own repositories and you can find a bunch of them, named XXXXXX-bootloader. Bootloaders are not that interesting and can be ignored by most users.
History
Some repositories might have names that do not make sense in the current context, this is most likely due to historical reasons. We might have given a repository a name that seemed to be descriptive at that time, but as time goes by it might not be as good any more. An example is the range of “LPS-XXXXX” repositories that contains code for the Loco Positioning System. Originally LPS meant “Local Positioning System” as this was the only positioning system we had, but when we also created the Lighthouse system we changed the meaning of LPS to “Loco Positioning System”. A better naming of the repositories would perhaps be “Loco-XXXX”?
A similar transition also exists for the Crazyflie that has evolved from Crazyflie 1 to Crazyflie 2.0 and 2.1. Some repositories (named “crazyflie-XXX”) were created when Crazyflie 1 was released and have evolved to be compatible with Crazyflie 2.X, while some other repositories (named “crazyflie2-XXX”) were created when Crazyflie 2.0 was released. Even though crazyflie-firmware is not compatible with Crazyflie 1 anymore, it still has the old name.
Retired code
Some repositories are obsolete, maybe they contain experimental code that is no longer of interest or perhaps the functionality has been implemented elsewhere. We try to archive these repos and if you are looking for current functionality you can safely ignore any repository that is marked with the “Public archive” flag in github.
Still unclear
There is a short description in each repository that is intended to describe the contents. It is possible (likely) that the description was written a long time ago, with a different context and that it might not be as helpful as intended. If this is the case, just ask us. Drop a question in our discussions forum or send us an email, we try to help as much as we can!
This early summer my research group (Center for Project-Based Learning at ETH Zürich) was in charge of a special week – high school students from all over Switzerland (actually even the world, they had to speak German though) could apply for a study week at different departments from our university. The departments which joined this initiative were mathematics, physics, biology, environmental sciences, material sciences, and our department, electrical engineering and information technologies (ITET). But how do you show teenagers between 15-19 in one week as much as possible from electrical engineering while also having fun? And best inspire them to study at ITET? Our solution was: drones. More specifically, Crazyflies. With those we had many possibilities to learn about electrical engineering – from sensors, microcontrollers, timers, and motors to LEDs, batteries, embedded systems, FreeRTOS tasks, state estimation, and controller – and all this with a high fun potential and a low risk of accidents, as with their weight of only 30g they hardly ever do any damage. In this blog post, I will guide you through our week, in hopes to help others who also want to use the Crazyflie to teach students about electrical engineering in a fun way.
Monday
We started in the afternoon (in the morning they had a welcoming tour) with a short introduction and splitting the 20 students into groups of two (everyone got a paper slip and had to find the matching one, accelerometer gyroscope, pitch roll, UART SPI, and so on – this gives the lecturer a great opportunity for interaction with the students later on, once their word gets relevant during the week). After a short introduction to programming and microcontrollers we moved on to the most classic beginner task: blink an LED! We chose to use the front left one, as this one is only used when communicating – so as long as we don’t connect to the drone we can observe exactly what we programmed. Most students got the LED turned on rather quickly – however, pulsing the LED to change the intensity took them some more time and forced them to learn how to write loops. They also already learned how PWM works without knowing it yet – setting the intensity of an LED or the strength of a motor is about the same thing in the end after all and this gave us a great start for Tuesday.
Tuesday
On Tuesday we looked at hardware from different perspectives. As you might have guessed, we looked at motors and how to control them with PWM and timers. The students were a bit disappointed that we still didn’t fly, but as soon as they realized that they could play their favorite song on the motors the motivation was high again! We didn’t even have any stray drones, even though we let them mount the propellers (the songs sound much better with propellers). We also looked at another aspect of electrical engineering: PCB design (this was already done though) and soldering. For this, we prepared a custom deck, with four colored LEDs, which could be populated like in industry with solder paste and then soldered with a hot plate. To make it even more fun (and partly to show off our laser cutter) they also designed a small plastic diffuser that could be mounted on top. So in the end our setup resembles the LED ring – however, it can be mounted on top which is essential if you don’t want to fly with a positioning system (and therefore need to mount a flow deck).
Wednesday
Trying out the state estimation
Now that we knew how to blink LEDs and, even more important, how to control motors we had to learn a bit about how the drone actually figures out which motor should be turned on and how much. For this, we first looked at sensors and wired communication protocols, such as I2C (for the IMU and time-of-flight sensor), SPI (for the optical flow), and UART (to the nRF) – due to limited time we didn’t go into details here though. We briefly touched wireless communication, to explain how the commands they will later send to the drone (and the firmware) are actually sent to the right drone.
Tuning PIDs
We moved on to what state estimation is in general – again jumping over all details of an extended Kalman Filter, but had a closer look at the logging and parameter system. We then spent a bit more time on the PID controller again – which was also a bit hard to explain, as half the teenagers hadn’t learned how to integrate and differentiate yet. However, they learned fast and we could move to the part they waited for all week long: Flying (and tuning the PID).
Thursday
Transport drone
This was the day that was meant for creativity – the students could choose themselves which project they want to achieve. We proposed them some ideas, such as blinking LEDs depending on the height, flying through a gate (the challenge here is to filter the height measurements when the gate border is below the drone), steering the drone with the keyboard, soldering an own sensor on a break-out board, …
In the end, we saw many cool projects, from a song played as a canon on multiple motors to a transporter drone, flying successfully through the gate, doing a successful looping (unfortunately no successful landing yet…), racing against each other (possibly with disco-lights on the deck) and trials for reaching max speed in the hallway.
The hallway was very popular to fly, as the distance to accelerate was longer…
Practice on the race parkour
The most popular base project was to steer the drone with the keyboard – unfortunately (or fortunately? they sure had fun with it once it was running and they might have learned enough in the remaining week…), this was very easy after we showed them where Marcus’ script lives (here) and which 8 lines (170-177) they have to remove for it to work without an AI-deck (and don’t forget to adapt the URI)…
Friday
Quite an audience!
Flying was forbidden – but playing music with motors was not!
On Friday it was presentation day – in the morning they could still work on their projects, but in the afternoon all the 120 students (and most of their parents) came together in a huge lecture hall to present what they did during this week. And, as at a real conference, they had posters and their drones (which we, unfortunately, were not allowed to fly without a fireproof net… Will organize this next time) to show their projects to family, friends, and even random tourists (the entrance hall of the ETH main building is on many sightseeing tours).
At the end of the week I doubted the robustness of Crazyflies for a moment – however, Monday morning once I had peace and quiet once again I figured out what was wrong with all hardware which ended up on the “not working for unknown reasons” stack in less than an hour (and fixed almost all of it). Notes to all others and my future self for the next time I give 10 drones to 20 teenagers:
If you show them how to tune a PID, also explain that “persistent” means exactly what it says – if you mess up the PID values and persist them they will stay this way until you reset them, no matter how often you reflash the drone.
Explain how fragile connectors are and that you NEVER should pull at cables. Also, mention 10x more to be careful when plugging in decks. And radios.
Keep one “private” drone no one is allowed to mess with – it will help greatly to figure out if they only broke the flow deck connectors or something more serious (which actually never happened)
Doing only warm boots with setting individual addresses with the CLOAD_CMDS while flashing saved us a lot of trouble, randomly connecting to drones only happened once they discovered the app for the phone…
The coding tasks (and at least some minimal solutions) can be found on my fork: Tasks and solutions. They are kept short on purpose – we at the Center for Project-Based Learning believe in our name – we believe the most learning (and fun) happens when you rather freely explore what you can do with the basic tools you just learned.
P.S. For completeness – I cut out all the parts which really had nothing to do with Crazyflies, we also did lab tours in the high-voltage laboratory and the laboratory for optical communication – and of course had some social events with actual university students. As much as we like the Crazyflie, even we have to admit that the field of electrical engineering is even bigger than what we can show with those tiny drones ;)
When the Crazyflie was created the intended use case was manual flight with one drone. Over the years we have added support for positioning, swarms, autonomous flight and all sorts of nice features, and it has all been built on top of the original code base. Some of the original code is actually untouched after 7-8 years and needless to say, there is a slight worry that we might have taken design decisions back then that will come back and bite us in new use cases. This blog post will outline some of the work we have been doing to handle this problem by setting up an autonomous system where a Crazyflie is continuously flying – the infinite flight project.
The original design essentially assumed one Crazyflie that is controlled by one computer with one Crazyradio. On the computer the user was running the python client and controlling the flight manually with a game pad. The user might have restarted the Crazyflie before each flight, or at least when changing battery.
Now fast forward to the current situation where a swarm of Crazyflies might be controlled by multiple radios, each connecting to multiple Crazyflies. The Crayflies are flying autonomously, perhaps getting their current position from the Lighthouse system, or via radio based on information from a mocap system. Maybe telemetry data is sent back to the ground while commands for trajectories to the Crazyflies go in the other direction. In some systems the Crazyflies use wireless charging to be able to run continuously.
Obviously the current situation is very different from the original design with new or changed requirements. One is the extended use of the radio, and this is something that we have been talking about in some previous blog posts and we will not discuss that here. This blog post will instead be about one other important topic: long term stability.
In the original design, the Crazyflie was restarted often, maybe before each flight. This means that the code did not run for a very long time, so what happens if we use wireless charging and keep the firmware running for days? Will there be a problem? We decided to find out by starting an internal project we called “Infinite flight”. The idea was to set up a system with a Crazyflie with a Qi-charger for wireless charging and a Lighthouse deck for positioning. An app in the Crazyflie takes off, flies a trajectory and lands for recharging when the battery is out, the cycle is then repeated for as many times as possible. By doing this, we hoped to find any software problems in the firmware that might show up after some time, finding hardware that is worn out over time or other finding other issues. Spoiler alert: we have not reached infinity yet, but we have got a bit closer :-)
The setup is fairly straight forward, the firmware is based on the app used in the demo we used at IROS and ICRA, with some modifications. We have a ground station computer that collects data, it tries to continuously maintain a connection and re-establish it if it is lost. We log as much as possible to be able to analyse problems and understand what happened. We also added some tools to make it easier to visualize and dig in all the log data. The usual work flow has been to
Start the Crazyflie and run the app
Wait for something to go wrong (sometimes days)
Analyze what happened and figure out if something needs to be changed
Update and start over again
A surprising number of runs failed fairly quickly, only after a few flights. The reason has usually been some sort of handling error or problems with the test software, but some have been of more general interest or bugs.
Stopped logging
We had a problem where the logs from the Crazyflie in the ground computer stopped without any apparent reason. It turned out to be related to the session-less nature of the CRTP protocol, there is no good way to determine if a session is alive or not, other than using a timeout. It turned out that there is a timeout in the Crazyflie firmware (that was not fresh in our memories) that stops logging if no packets are received for a while. The rationale is to avoid having old logs running if a client is disconnected. In our case we lost communication for a short period of time and the firmware simply stopped the logging. The python lib on the other hand had a longer timeout and did not have the view that the connection was lost. The solution we used is to set up the logs again if we don’t receive logging for a while (we fixed an issue in the python lib related to this)
In the future com stack we plan to have proper session handling which should remove this problem.
Controller tuning
We have been using various flavors of Crazyflies in the tests, including some prototypes. We had some issues with one prototype that we did not understand, it had a hard time hitting the landing pad when landing. It turned out that the STM that was used in the prototype was reused from an old Crazyflie and it had some weird PID controller settings stored as persistent parameters. With the persistent parameters cleared, it worked as expected.
We have played a bit with tuning the controller, but the default settings are fairly OK and we used them most of the time.
Some of the prototypes we used had some glitches or irregularities, it is very hard to hand solder PCBs. These problems are only related to a specific hardware individual and can cause some unexpected behavior which takes time to figure out.
Landing pad edge
The landing/charging pad we use (also used in our demos) generally works fine, it has a “slope” towards the center which helps the Crazyflie slide to the correct position. If the Crazyflie miss the landing too much though, it will end up on the edge with one leg on the ground and an angle away from the center. In this case it sometimes fails to take off properly and crash. We solved this by adding a foam pad around the landing pad to “raise” the surrounding floor the the same level as the landing pad and thus reduce angle.
Lighthouse bug
There is a known bug in the lighthouse deck that prevents it from receiving data at certain angles. We had some cases where our landing pad was located in a spot where we lost tracking of both the active base stations for some yaw angles. If the Crazyflie happened to land in that exact yaw angle, it lost the position while charging and it did not know its current position when it should take off again. This was solved by moving the landing pad to a different position.
Not yet investigated possible problems
One possible known problem is the system tick counter in FreeRTOS. The counter is a 32-bit unsigned word and it is increased every millisecond. The counter is used in the firmware as the internal “clock” to determine for instance how long ago the estimator was updated or to determine when to execute some piece of code the next time. This counter will wrap after 2^32 ms, that is around 7 weeks, and we don’t really know what will happen.
Results and conclusions
The longest we managed to keep the system running was 5 days. We had a slow charging cycle and only flew 57 times. In this session, the flight time was very stable between 5:30 and 5:45 for all flights. This test was done with a standard Crazyflie 2.1 with a motor upgrade kit.
The second longest session was 3 days, in this case we used a brushless prototype and pushed the charging very hard. We ended up doing 276 flights but the battery was pushed beyond specs and being too warm and charged too fast, it degraded over time and the flight time was reduced to only 2-3 minutes at the end.
We believe we have fixed most of the long term stability issues, but it is hard to know. There might be bugs lurking in the firmware that only show up under very special conditions. What we do know is that it is possible to fly for 5 days!
Today, Lennart Bult from Emergent Swarns presents us with this project of a 24/7 swarming demo. Enjoy!
Over the last few months our team has been working on creating a 24/7 swarming demo. Initially tasked by Guido de Croon and Chris Verhoeven from TU Delft MAVLab and the TU Delft Robotics Institute, we set out to find our way within the Crazyflie ecosystem to gradually increase the size and capabilities of the swarm. In this article we will first talk about some of the work and methods that we used. After that, we will introduce the TU Delft Science Centre Swarming Lab and talk about some applications of swarming drones.
Developing the 24/7 swarm
The project started in February with the goal of creating a physical swarm capable of real-time collision avoidance with drones and static obstacles. We started out with three drones equipped with the Flow Deck, and by setting them up in a clever way we could perform the first collision avoidance and landing tests. We were impressed with the performance we got out of the Flow Deck, however, eventually, it is mostly a battle against the drift of the position estimate, that is, we could increase some of the margins on the collision avoidance only so far before we would either fly out of the test zone or collide with another drone. Luckily with short test flights, we were able to see some of the flaws in our algorithms and correct them before testing with the new setup.
Setup after the first expansion to eight drones.
After a few weeks of testing we got approved for the first swarm expansion, five more drones and a Lighthouse positioning setup. This is when we could do our first real tests with the collision avoidance algorithm, which, much to our own surprise, worked on the first try. This is also when we first posted a project update on LinkedIn. There were however a lot of bugs that still needed to be worked out, and a lot of system experience still to be gained. After flying for a bit longer we noticed that some of the drones would flip quite often, which is when we discovered that we needed the thrust upgrade to control the additional weight of the larger battery and charging deck.
For the charging setup we took inspiration from the Bitcraze IROS 2022 demo; we 3D printed sloped landing pads that we tape onto a wireless charger. After a few iterations we landed on a design that uses minimal printer resources and allows the Crazyflie to land a bit off-center. This last feature turned out to be quite useful considering the large amount of destabilizing airflow that is generated by 40 drones. After receiving the last order of drones we also expanded the charging setup, which at this point takes up quite a bit of floor space. There are some ideas to create a vertical landing pad stack, which would bring the additional challenge of missing the landing pad not being an option.
All 40 drones recharging before their next flight.
After prototyping the charging setup and building confidence with the initial setup, we were confident enough in our system capabilities to expand it to the point where a continuous demo of 5-8 drones is possible. Although the system integration of the previous expansion went without much trouble, we did encounter a few issues when expanding to 40 drones. The first issue of which was radio communication, we noticed that a delay in the radio communication would be present if we increased the update rate above a certain level for a specific number of drones per radio. The second issue we encountered were performance drops related to the violation of certain bounds in the collision avoidance algorithm. These two issues were very difficult to debug since it was not immediately obvious where the source of the issue was.
The third and last major issue was the increase in destabilizing airflow of 40 drones compared to 8. With 40 drones there is a noticeable breeze when you stand next to the drone cage, which is nice for summertime, but not so nice when drones need to land in a tight-packed configuration. To combat this issue there is a limit to the amount of drones that can land at the same time. There is also a minimum separation distance between two active landing pads, which reduced the severity of the induced turbulence. There are still ongoing efforts to increase the landing success rate, which is currently affected by drones running out of power during the landing procedure.
To control and monitor the swarm we designed a custom GUI, an impression of which you can see below. Although some of the buttons are still a work in progress, there are a lot of features that have already proven very useful, especially when testing a new feature.
V1 of the graphical user interface developed for the 24/7 swarm.
The code base that we created for the swarm will be largely open-sourced (only the collision avoidance will not be open-source) to provide researchers all around the world with the possibility to setup their own Crazyflie swarm for research. You can find the repository through this link. Note that the documentation and code base are still under development and might contain bugs/errors.
Human interaction
After creating all functionality to provide a continuously operating swarm demo, it was time to work on some of our stretch goals: 1. walking through the swarm whilst it is operating and 2. controlling the swarm using our arms. In the image below you can see an impression of precisely this functionality. The drones are following the operator’s gesture commands whilst performing live collision avoidance with an operator.
Team member Seppe directing the 40 drone swarm, see the full video here.
This demo requires multiple techniques and hardware elements working together to create a relatively low-latency, human-controlled swarm. We used a Kinect-like 3D sensor to perform human pose estimation, we subsequently used this data to create a dynamic obstacle in our collision avoidance software. An important element to consider here is the synchronization of the Lighthouse- and 3D sensor coordinate frames, i.e. without proper calibration the human will not be correctly positioned with respect to the drones and the drones will crash into the human. The interaction between the swarm control software and the human gesture commands also requires careful consideration, proper tuning is required to ensure a responsive system that is reliable and not too aggressive.
TUD Science Centre Swarming Lab
The next step in this project will be to set up the swarm at its new location, the TU Delft Science Centre. Here, the swarm will first and foremost be visible as a public demo, showcasing the capabilities of TU Delft state-of-the-art swarming research. There will also be a focus on developing the swarm as a research platform. This will allow TU Delft students and researchers to extend swarm functionalities and test their theory on a physical swarming system. Besides demos and academic research, there will also be worked on developing educational applications across the full educational board (primary school, high school and applied education). If you are interested in working on, or collaborating with the swarming lab on any of the above-mentioned tasks, feel free to email the lab management at operations.swarminglab@tudelft.nl.
The TU Delft Swarming Lab setup with 40 drones and charging pads for continuous operations and research.
Applications of Swarming
There are a lot of potential use-cases for fully autonomous drone swarms, ranging from indoor applications such as warehouse monitoring and factory inspection to outdoor applications such as search and rescue and surveillance. In our opinion, the true potential of drone swarms lies in applications where there is a significant need for a scalable system with a lot of built-in redundancy. A lot of additional use cases open up when we consider fully onboard autonomous systems, where the full benefits of decentralized swarming can be utilized. Currently, the size of drones needed to achieve such feats is quite large, though maybe in a few years, we could see more and more being done on drone platforms such as the Crazyflie.
A swarm inspection of an F-16 Fighting Falcon at Deltion College in Zwolle, the Netherlands.
An interesting area of application for drone swarms could be in the inspection of aircraft. Drone swarms provide a scalable and flexible means to perform a fast inspection of aircraft across an entire airfield or military base. To showcase that this can be done with any size of drone, we went to Deltion College in Zwolle to perform a mock inspection of an F-16 fighter jet. Above you can see an impression of the inspection. Another area of application is search and rescue, where there is a need for systems that can find people or objects of interest in unknown and cluttered environments. Furthermore, the area that needs to be searched is usually very large and sometimes difficult to travel on foot. A drone swarm could provide fast and reliable coverage of the area of interest, whilst providing full data traceability. Seppe and Lennart will work on creating drone swarms for these use cases with the start-up Emergent Swarms.
We are happy to announce that we are working on a new upgrade battery for the Crazyflies! It will soon hit production and hopefully, keeping our fingers crossed, it will arrive in our stock in early 2023-Q4.
The upgrade battery is based on the “Tattu 350mAh 3.7V 30C 1S1P” cell and with some additional great features:
Protection Circuit Module (PCM) to protect against short circuits, overcharge, over discharge etc.
Gold-plated connectors for lower contact resistance.
Shrink wrap around connector for better rigidity.
Cool Bitcraze matched graphics.
And if we list the benefits compared to the stock Crazyflie battery:
Higher current capabilities, 30C burst current, that is >10 Amp.
350mAh instead of 250mAh
Higher energy density, ~130 Wh/kg instead of ~105 Wh/kg
There are some drawbacks too:
It is ~1 mm thicker and does not fit well with all deck boards and the short or medium size pin headers. We will release longer pin headers at the same time though.
Price will be higher
~1.5 grams extra weight
With this upgrade battery, you will experience longer flight times, more “punch” during acceleration and it is great combined with the thrust upgrade kit!
As you may have noticed from the recent blog posts, we were very excited about ICRA London 2023! And it seems that we had every right to be, as this conference had the highest number of Crazyflie related papers compared to all the previous robotics conferences! In the past, the conferences typically had between 13-16 papers, but this time… BOOM! 28 papers! In this blog post, we will provide a list of these papers and give a general evaluation of the topics and themes covered so far.
So here some stats:
ICRA had 1655 papers accepted (43 % acceptance rate)
28 Crazyflie papers (25 proceedings, 1 RA-L, 1RO-L, 1 late breaking result postor)
Haven’t included the workshop papers this time (no time)
The major topics we discovered were swarm coordination, safe trajectory planning, efficient autonomy, and onboard processing
Additionally, we came across a few notable posters, including one about a grappling hook for the Crazyflie [26], a human suit that allows for drone control [5], the Bolt made into a monocopter with a Jetson companion [16], and a flexible fixed-wing platform driven by a barebone Crazyflie [1]. We also observed a growing interest in aerial robotics with approximately 10% of all sessions dedicated to UAVs. Interestingly, 18 out of the 28 Crazyflie papers were presented in non-UAV specialized sessions, such as multi-robot systems and vision-based navigation.
Swarm coordination
Swarms were a hot topic at ICRA 2023 as already noticed by this tweet of Ramon Roche. We had over 10 papers dedicated to this topic, including one that involved 16 Crazyflies [9]. Surprisingly, more than half of the papers utilized multiple Crazyflies. This already sets a different landscape compared to IROS 2022, where autonomous navigation took center stage.
In IROS 2022, we witnessed single-drone gas mapping using a Crazyflie, but now it has been replicated in the Webots simulation using 2 Crazyflies [23]. Does this imply that we might witness a 3D gas localizing swarm at IROS 2023? We can’t wait.
Furthermore, we came across a paper [11] featuring the Bolt-based platform, which demonstrated flying formations while being attached to another platform using a string. It presented an intriguing control problem. Additionally, there was a work that combined safe trajectory planning with swarm coordination, enabling the avoidance of obstacles and people [12]. Moreover, there were some notable collaborations, such as robot pickup and delivery involving the Turtlebot 3 Burger [22].
Given the abundance of swarm papers, it’s impossible for us to delve into each of them, but it’s all very impressive work.
Safe trajectory planning and AI-deck
Another significant buzzword at ICRA was “safety-critical control.” This is important to ensuring safe control from a human interface [15] and employing it to facilitate reinforcement learning [27]. The latter approach is considered less “safe” in terms of designing controllers, as evidenced by the previous IROS competition, the Safe Robot Learning Competition. Although the Crazyflie itself is quite safe, it makes sense to first experiment with safe trajectories on it before applying them to larger drones.
Furthermore, we encountered approximately three papers related to the AIdeck. These papers covered various topics such as optical flow detection [17], visual pose estimation [21], and the detection of other Crazyflies [5]. During the conference, we heard that the AIdeck presents certain challenges for researchers, but we remain hopeful that we will see more papers exploring its potential in the future!
List of papers
This list not only physical Crazyflie papers, but also papers that uses simulation or parameters of the Crazyflie. This time the workshop papers are not included but we’ll add them later once we have the time
Enjoy!
‘A Micro Aircraft with Passive Variable-Sweep Wings’ Songnan Bai, Runze Ding, Pakpong Chirarattananon from City University of Hong Kong
‘Onboard Controller Design for Nano UAV Swarm in Operator-Guided Collective Behaviors’ Tugay Alperen Karagüzel, Victor Retamal Guiberteau, Eliseo Ferrante from Vrije Universiteit Amsterdam
‘Multi-Target Pursuit by a Decentralized Heterogeneous UAV Swarm Using Deep Multi-Agent Reinforcement Learning’ Maryam Kouzehgar, Youngbin Song, Malika Meghjani, Roland Bouffanais from Singapore University of Technology and Design [Video]
‘Inverted Landing in a Small Aerial Robot Via Deep Reinforcement Learning for Triggering and Control of Rotational Maneuvers’ Bryan Habas, Jack W. Langelaan, Bo Cheng from Pennsylvania State University [Video]
‘Ultra-Low Power Deep Learning-Based Monocular Relative Localization Onboard Nano-Quadrotors’ Stefano Bonato, Stefano Carlo Lambertenghi, Elia Cereda, Alessandro Giusti, Daniele Palossi from USI-SUPSI-IDSIA Lugano, ISL Zurich [Video]
‘A Hybrid Quadratic Programming Framework for Real-Time Embedded Safety-Critical Control’ Ryan Bena, Sushmit Hossain, Buyun Chen, Wei Wu, Quan Nguyen from University of Southern California [Video]
‘Distributed Potential iLQR: Scalable Game-Theoretic Trajectory Planning for Multi-Agent Interactions’ Zach Williams, Jushan Chen, Negar Mehr from University of Illinois Urbana-Champaign
‘Scalable Task-Driven Robotic Swarm Control Via Collision Avoidance and Learning Mean-Field Control’ Kai Cui, MLI, Christian Fabian, Heinz Koeppl from Technische Universität Darmstadt
‘Multi-Agent Spatial Predictive Control with Application to Drone Flocking’ Andreas Brandstätter, Scott Smolka, Scott Stoller, Ashish Tiwari, Radu Grosu from Technische Universität Wien, Stony Brook University, Microsoft Corp, TU Wien [Video]
‘Trajectory Planning for the Bidirectional Quadrotor As a Differentially Flat Hybrid System’ Katherine Mao, Jake Welde, M. Ani Hsieh, Vijay Kumar from University of Pennsylvania
‘Forming and Controlling Hitches in Midair Using Aerial Robots’ Diego Salazar-Dantonio, Subhrajit Bhattacharya, David Saldana from Lehigh University [Video]
‘AMSwarm: An Alternating Minimization Approach for Safe Motion Planning of Quadrotor Swarms in Cluttered Environments’ Vivek Kantilal Adajania, Siqi Zhou, Arun Singh, Angela P. Schoellig from University of Toronto, Technical University of Munich, University of Tartu [Video]
‘Decentralized Deadlock-Free Trajectory Planning for Quadrotor Swarm in Obstacle-Rich Environments’ Jungwon Park, Inkyu Jang, H. Jin Kim from Seoul National University
‘A Negative Imaginary Theory-Based Time-Varying Group Formation Tracking Scheme for Multi-Robot Systems: Applications to Quadcopters’ Yu-Hsiang Su, Parijat Bhowmick, Alexander Lanzon from The University of Manchester, Indian Institute of Technology Guwahati
‘Safe Operations of an Aerial Swarm Via a Cobot Human Swarm Interface’ Sydrak Abdi, Derek Paley from University of Maryland [Video]
‘Direct Angular Rate Estimation without Event Motion-Compensation at High Angular Rates’ Matthew Ng, Xinyu Cai, Shaohui Foong from Singapore University of Technology and Design
‘NanoFlowNet: Real-Time Dense Optical Flow on a Nano Quadcopter’ Rik Jan Bouwmeester, Federico Paredes-valles, Guido De Croon from Delft University of Technology [Video]
‘Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning’ Cheng Liu, Erik-jan Van Kampen, Guido De Croon from Delft University of Technology
‘Relay Pursuit for Multirobot Target Tracking on Tile Graphs’ Shashwata Mandal, Sourabh Bhattacharya from Iowa State University
‘A Distributed Online Optimization Strategy for Cooperative Robotic Surveillance’ Lorenzo Pichierri, Guido Carnevale, Lorenzo Sforni, Andrea Testa, Giuseppe Notarstefano from University of Bologna [Video]
‘Deep Neural Network Architecture Search for Accurate Visual Pose Estimation Aboard Nano-UAVs’ Elia Cereda, Luca Crupi, Matteo Risso, Alessio Burrello, Luca Benini, Alessandro Giusti, Daniele Jahier Pagliari, Daniele Palossi from IDSIA USI-SUPSI, Politecnico di Torino, Università di Bologna, University of Bologna, SUPSIETH Zurich [Video]
‘Multi-Robot Pickup and Delivery Via Distributed Resource Allocation’ Andrea Camisa, Andrea Testa, Giuseppe Notarstefano from Università di Bologna [Video]
‘Multi-Robot 3D Gas Distribution Mapping: Coordination, Information Sharing and Environmental Knowledge’ Chiara Ercolani, Shashank Mahendra Deshmukh, Thomas Laurent Peeters, Alcherio Martinoli from EPFL
‘Finding Optimal Modular Robots for Aerial Tasks’ Jiawei Xu, David Saldana from Lehigh University
‘Statistical Safety and Robustness Guarantees for Feedback Motion Planning of Unknown Underactuated Stochastic Systems’ Craig Knuth, Glen Chou, Jamie Reese, Joseph Moore from Johns Hopkins University, MIT
‘Spring-Powered Tether Launching Mechanism for Improving Micro-UAV Air Mobility’ Felipe Borja from Carnegie Mellon university
‘Reinforcement Learning for Safe Robot Control Using Control Lyapunov Barrier Functions’ Desong Du, Shaohang Han, Naiming Qi, Haitham Bou Ammar, Jun Wang, Wei Pan from Harbin Institute of Technology, Delft University of Technology, Princeton University, University College London [Video]
‘Safety-Critical Ergodic Exploration in Cluttered Environments Via Control Barrier Functions’ Cameron Lerch, Dayi Dong, Ian Abraham from Yale University
We’re happy to announce that there is a new release of the software for the Crazyflie ecosystem! The new release is called 2023.06 and is available for download on github or through the python client.
Major changes
The main addition is an extended supervisor framework and updated arming functionality.
Extended supervisor framework
The purpose of the supervisor is (will be) to keep an eye on the Crazyflie and make sure that everything is fine. If it detects a problem it can take action to hopefully handle the situation in a way that is better for the Crazyflie as well as people close by. The supervisor taps into the stabilizer loop and has the power to take control of the motors when needed.
The current version actually behaves very much like the previous version, but the underlying framework has been re-written to enable better handling in the future. There are now well defined states that the Crazyflie goes through for preflight checks, when flying and after landing.
Arming
Basic arming functionality has been added, mainly intended for larger platforms with brushless motors. A manual action is required after preflight checks have passed, to let the Crayflie know that a human is in control. If the system is not armed, it is not possible to fly.
Arming is required by default for the Bolt platform. For the Crazyflie 2.X, there is an auto-arming feature that immediately arms the platform when the preflight checks have passed, that is it works like it used to do.
If you use a BigQuad deck, auto-arming will also be enabled by default (as it uses the Crazyflie 2.X platform) and the firmware should be rebuilt with the MOTORS_REQUIRE_ARMING kbuild config flag set to enable manual arming.
The arming functionality is built on top of the supervisor.
Updates to the python client
An arming button has been added to the flight tab in the client to support the new arming functionality.
An emergency stop button has also been added to the top of the client window that shuts down the motors immediately.
Updates to the python library
A new CRTP message has been added to arm/disarm the system. The CRTP version has been updated to version 6.
Note, if you are controlling a Bolt from a script (or any other platform with arming enabled) you have to send an arming message to the platform before you can fly.
Release details
The following versions were released. See each release for details.
As mentioned in a previous blog post, we have a both at ICRA in London this week. If you are there too, come and visit us in booth H10 and tell us what you are working on!
Barbara and Arnaud is getting the booth ready
We are showing our live autonomous demo and our products in the booth, including the flapping drone Flapper Nimble, don’t miss it!
The autonomous demo
The decentralized autonomous demo that we are showing is based on technologies in the Crazyflie ecosystem. The general outline is that Crazyflies are autonomously flying in randomized patterns without colliding. The main features are:
Positioning using the Lighthouse positioning system, all positioning estimation is done in the drone. The Lighthouse positioning system provides high accuracy and ease of use.
Communication is all peer-to-peer, no centralized functionality. Each Crazyflie is transmitting information about its state and position to the other peers, to enable them to act properly.
Collision avoidance using the on-board system without central planing. Based on the position of the other peers, each Crazyflie avoids collisions by modifying its current trajectory.
Wireless charging using the Qi-deck. When running out of battery, the Crazyflies go back to their charging pads for an automatic re-fill.
The App framework is used to implement the demo. The app framework provides an easy way of writing and maintaining user code that runs in the Crazyflie.
We are happy to answer any questions on how the technology works and implementation details. You can also read more about the demo in the original blog post by Marios.
Developer meeting
The next developer meeting is next week, Wed June 7 15:00 CEST and the topic will be the demo and how it is implemented. If you want to know about any specific technologies we used, how it is implemented or if you are just curious about the demo in general, please join the developer meeting. We will start with a presentation of the different parts of the demo, and after that a Q&A. As always we will end up with a section where you can ask any question you like related to our ecosystem. Checkout this announcement on our discussion platform for information on how to join.
When designing flying robots like drones it is important to be able to benchmark and test the propulsion system which in this case is a speed controller, motor and propeller. As we at Bitcraze are mainly working with tiny drones we need a thrust stand designed for small motors and propellers. We have actually already designed our own system identification deck, which can measure overall efficiency, thrust, etc., but is lacking the ability to measure torque. Torque is needed to be able to measure propeller efficiency which is now something we would like to measure. Before we developed the system-id deck we searched for of the shelf solutions that could satisfy our needs and could not find any. This still seems true, please let us know if that isn’t the case.
Expanding the system-id deck to measure torque doesn’t work and building something from scratch was a too big of a project for us. Next natural option would then be to modify an existing thrust stand and our choice fell for the tyro robotics 158X series.
Looking at specifications, images and code we could figure out that replacing the load cells for more sensitive ones should be possible. The stock setup of 5kgf thrust and 2Nm of torque is just too much as we are looking for around 100 grams of thrust and around 10 mNm of torque. So we decided to give the replacement of load cells a shot! Assembly was quite smooth but we managed to break one of the surface mount load cell connectors off, luckily this was easily fixable with a soldering iron. With the stock setup we did some measurements with a 0802 11000KV brushless motor and a 55mm propeller in a pushing setup. It works but the measurements are noisy and repeatability is not great. Next thing would be to replace the load cells. The 158X uses TAL221 sized load cells which are available down to 1kg. We got those and with a calibration-allways-pass code we got from Tyto robotics we could make the calibration pass (note that modifying the thrust stand breaks the warranty). Now the thrust stability was much better but still the torque was a bit to noisy. We decided to go for even smaller thrust cells, the TAL220, and build 3D printable adapters to make them fit.
Now the torque noise level looked much better and so did the repeatability. By empirically measuring the thrust and torque using calibrated weights and by checking the measurements in RCBenchmark we got these values:
Thrust, calibrated weight [g]
Measured [g]
Noise [g]
200
200
1
100
100
1
50
50
0.5
20
20
0.5
10
10
0.5
0
0
0.5
Trust (calibrated using 200g weight)
Torque, calibrated weight [g]
Measured [mNm]
Noise [mNm]
200
257
2
100
128
1
50
64
0.3
20
25.7
0.3
10
12.7
0.3
0
0
0.2
Torque (calibrated using 200g weight)
Simple repeatability test
The thrust stand modification is still very fresh and we have to figure out some things but it all looks promising. For example we get 13% less overall efficiency when measuring it using our system-id thrust stand. Our guess is that it is due to that the Crazyflie arms in the system-id case blocks the airflow.
If you would like to do this modification yourself there are some simple instructions and STL files over at out mechanical github repository. Have fun!
It is easy to forget that the reason why it is nice to develop for the Crazyflie is because it weighs only about 30 grams. In case something goes wrong with your script or there is a fly-away, you can simply pick it up from the air without worrying about the propellers hitting you. Moreover, when the Crazyflie crashes, it usually only requires a brush off and a potential replacement of a motor-mount or propeller. The risk of damage to yourself, other people, indoor furniture, or the vehicle itself is extremely low. However, things become very different if you’ve built a larger platform with the Bolt or BQ deck with large brushless motors (like with this blogpost), where the risk of injury to people or to the vehicle itself increases significantly. That is one of the major reasons why the BQ deck and the Bolt are still in early access and have been for a while. In our efforts to get it out of early access, it’s time to start thinking about safety features.
In this blog post, we’ll be discussing how other open-source autopilot programs are implementing safety features, followed by a discussion on current efforts for Crazyflie, along with an announcement of the developer meeting scheduled for May 3rd (see below for more info).
Catching the Crazyflie with a net
Safety in other Autopilots
We are a bit late to the game in terms of safety compared to other autopilot programs such as PX4, ArduPilot, Betaflight and Paparazzi UAV, which have been thinking about safety for quite some time. It makes a lot of sense when you consider the types of platforms that run these autopilots, such as large fixed VTOL or fixed-wing vehicles or 10-kilo quadcopters with cinematic cameras, or the degree of outdoor flight regulation. Flying a UAV autonomously or by yourself has become much more challenging as the US, EU, and many other countries have made it more restrictive. In most cases, you are not even allowed to fly if fail-safes are not implemented, such as what to do if your vehicle loses GPS signal. These types of measures can be separated into pre-flight checks and during-flight checks.
Pre-flight checks
Before a vehicle is allowed to fly, or even before the motors are allowed to spin, which is called ‘arming’, several conditions must be met. First, it needs to be checked if all internal sensors, such as the IMU, barometer, and magnetometer, are calibrated and functional, so they don’t give values outside of their normal operating range. Then, the vehicle must receive a GPS signal, and the internal state estimator (usually an extended Kalman filter) should converge to a position based on that information. It should also be determined if an external remote control is connecting to the vehicle and if there is any datalink to a ground station for telemetry. Feasibility checks can also be implemented, such as ensuring that the mission loaded to the UAV is not outside its mission parameters or that the start location is not too far away from its take-off position (assuming the EKF is functional). Additionally, the battery should not be low, and the vehicle should not still be in an error state from a previous flight or crash.
All of these features have the potential to be turned off or made less restrictive, depending on your situation. However, keep in mind that changing any of these may require recertification of the drone or make it fall outside what is required for outdoor flight regulation. Therefore, these should only be changed if you know what you are doing.
Now that the pre-flight checks have passed, the UAV is armed and you have given it the takeoff command. However, there is so much more that can go wrong during a UAV flight, and takeoff is one of the most dangerous moments where everything could go wrong. Therefore, there are many more safety features, aka failsafes, during the flight than for the pre-flight checks. These can also be separated into ‘triggers’ and ‘behaviors,’ so that the developer can choose what the UAV should do in case of a failure, such as ‘GPS loss’ to ‘land safely’ and so on.
Thus, there are triggers that can enable the autopilot’s failsafe mechanics:
No connection with the remote control
No connection with the Ground station or Datalink
Low Battery
Position estimate diverges or full GPS loss
Waypoint going beyond geofence or Mission is not feasible
Other vehicles are nearby.
Also, sometimes the support of an external Automatic Trigger system is required, which is a box that monitors the conditions where the UAV should take action in case there is no GPS, other aerial vehicles are nearby, or the UAV is crossing a geofence determined by outdoor flight restrictions. Note that all of these triggers usually have a couple of conditions attached, such as the level of the ‘low battery’ or the number of seconds of ‘GPS loss’ deemed acceptable.
Fail-safe behavior
If any of the conditions mentioned above are triggered, most autopilot suites have some failsafe behaviors linked to those set by default. These behaviors can include the following:
No action at all
Warning on the console or remote control display
Continue the mission autonomously
Stay still at the same position or go to a home position
Fly to a lower altitude
Land based on position or safely land by reducing thrust
No input to motors or completely disarming the motors
Usually, these actions are set in regulation, but per trigger, it is possible to give a different behavior than the default. One can decide to completely disarm the vehicle, but then the chances of the UAV crashing are pretty high, which can result in damage to the vehicle or cause harm to people or objects. By the way: disarming is the opposite act of arming, which is not allowing the motors to spin, no matter if it is receiving an input. If you decide to never do anything and force the drone to finish the mission autonomously, then in a case of GPS or position loss, you risk losing your vehicle or that it will end up in areas where it is absolutely not allowed, such as airports. Again, changing these default behaviors should be done by someone who knows what they are doing, and it should be done with careful consideration.
Fail-safes are measures that ensure safe flight. However, there will always be a chance that an emergency will occur, which will require an immediate action as well. If the vehicle has crashed during any of its phases or has flipped, or if the hardware breaks, such as the motors, arms, or perhaps even the autopilot board itself, what should be done then?
The standard default behavior for this is to completely disarm the vehicle so that it won’t react to any input to the motors itself. Of course, it’s difficult to do if the autopilot program is on, but at least it won’t try to take off and finish its mission while laying on its side. It might be that a backup system is connected to the ESCs that will take over in case the autopilot is not responding anymore, perhaps using a different channel of communication.
Also, the most important safety feature of all is the pilot itself. Each remote control should have a special button or switch that can put the drone in a different mode, make it land, or disarm it so that the pilot can act upon what they see. In case the motors are still spinning, have a net or towel available to throw over them, disconnect the battery as soon as possible, and make sure to have sand or a special fire retardant in case the LiPo batteries are pierced.
All of the autopilots have some tips to deal with such situations, but make sure to do some good research yourself on how to handle spinning parts or potential LiPo battery fires. I’m just giving a compilation of tips given in the documentation above here, but please make sure to read up in detail!
Safety in the Crazyflie Firmware
So how about the Crazyflie-firmware ? We have some safety features build in here and there but it is all over the code base. Since the Crazyflie is so safe, there was no immediate need for this and we felt it is more up to the developer to integrate it themselves. But with the Bolt and BQ deck coming out of early access, we want to at least do something. As we started already started looking into how other autopilot softwares are doing it, we can get some ideas, however we did notice that many of these are mostly meant for outdoor flight. The Crazyflie and the Crazyflie Bolt have been designed for indoor use and perhaps deal with different issues as well.
Current safety features
This is a collection of safety features currently in the firmware at the time of writing this blogpost. Most safety features in the Crazyflie are up for the developer to double check before and during flight, but these are some automatic once that are scattered around the firmware:
Watchdog, hard faults and asserts scattered throughout the firmware.
We might find more on the way…
However, if for instance your Crazyflie or Bolt platform loses its positioning in air, or doesn’t have a flowdeck attached before takeoff, there are no default safety systems in check. You either need to catch it, make it land or use an self-made emergency stop button using one of the emergency stop services above.
Safety features in works
As mentioned earlier, we have safety features spread throughout the code base of the Crazyflie firmware. Our current effort is to collect all of these emergency stops and triggers in the supervisor module to have them all in one place.
In addition, since indoor positioning is critical, we want to be notified when it fails. For instance, if the lighthouse geometry is incorrect, we need to see if the position diverges. This check was done outside of the Crazyflie firmware in a cflib script, but it has not been implemented inside the firmware. We also want to provide some options in terms of behavior for these triggers. Currently, we are working on two options: ‘turn the motors off’ or ‘safe land,’ with ‘safe land’ decreasing the thrust while keeping the drone level in attitude.
Furthermore, we want to integrate these features into the cfclient as well. For example, we want to add more emergency safety features to our remote control through the cfclient, and show users how to arm and disarm the vehicle.
These are the elements we are currently working on, but there might be more to come!
Developer meeting May 3rd
You probably already guessed it… the topic about the next developer meeting will be about the safety features in the Crazyflie and the Bolt! We will present the current safety features in the Crazyflie and what we are currently working on to make it better. In this sense, we really want to have your feedback on what you think is important for brushless versions of the Crazyflie for indoor flight!
The Dev meeting will be on Wednesday May the 3rd at 3 PM CEST. Please keep an eye on the discussion forum in the developer meeting thread.
