The semiconductor/electrical components shortage has not gone unnoticed by now, especially with the news coverage it had so far. It’s effecting most industries dealing with electrical components such as car manufacturers, which seem to have an especially hard time. We have been doing our best to handle the crisis, often solving it by throwing money at the problem. When you are a small player like us, you just don’t get prioritized by the big electronics manufacturers, and paying more at the open market is most often the only solution. In the beginning of May this year we had to increase our prices to counteract our increased cost as communicated in this blog post. Hopefully we will not have to increase them more but the shortage is now estimated to last for yet another year or two, and who knows how it eventually will effect us.
Another outcome of the semiconductor shortage is that the manufacturing lead-times has become very long and uncertain, so stock management has become much harder. We have solved it partially by increasing our stock but that also requires capital, which is a limited resource. We think we have managed quite OK so far but as of now Crazyflie 2.1, Multiranger-deck and SD-card decks are out of stock. Crazyflie 2.1 will hopefully arrive at the end of this week and the Multiranger and SD-card decks at the end of December. Even though times are a bit tough right now we are positive and believe we will be able to keep good stock levels throughout the crisis. We might not be able to guarantee that no products will be out of stock, but we can promise to do our very, very best to stay on top of this situation!
Sometimes we get the question of where to modify or add code to change some behavior of the Crazyflie. There is no quick answer to this question but we thought that we should write a post to clear up some question marks and give a better idea of how to approach the problem.
There are quite a few repositories on the Bitcraze github page but there are two that are the main focal point for almost any Crazyflie work, those are the crazyflie-firmware and the crazyflie-lib-python. The crazyflie-firmware contains the source code (written in C) for the firmware that runs in the Crazyflie, that is the code responsible for flying, blinking LEDs, communicating with the radio, scanning sensors and so on. The crazyflie-lib-python (often called the python lib) on the other hand is running on the PC side and is the API to use to communicate with the Crazyflie from a script. The crazyflie-lib-python is also used by the python client which means that anything you see in the client can be done by a script using the python lib.
Let’s assume we have a system of one Crazyflie connected to a computer using a Crazyradio. Now we want to control the Crazyflie and make it take off for instance, how should this be done?
The easiest way would be to use the python lib. The python lib is used to communicate with the Crazyflie and we can use it to send instructions to the Crazyflie, for instance to take off or fly a trajectory. It is also possible to use the parameter framework to change values in the Crazyflie. The main way of monitoring what is going on in the Crazyflie is to use the log framework to read variables from the Crazyflie. The python lib is perfect for controlling the Crazyflie or prototype ideas as it is very fast to make changes and try things out. The best to get started with the python lib is to start from an example that already uses functionalities you want to use.
Another option is to add code in the firmware. Originally this has been quite hard since the firmware has not been initially designed to accept user code. This means that unless you want to modify an already existing code, it is quite hard to find where to add your code so that it runs in the Crazyflie firmware and you would have to make a fork of the firmware which can he hard to maintain and keep up to date in the long run. This is one of the things the out-of-tree build and the app layer is solving, it is now quite easy to add and run your own C files to the firmware in your own project without having to fork the Crazyflie firmware. There is a bunch of examples in the firmware that shows how to implement autonomous behavior as an app. The easiest is to start with the hello world example. When it comes to modifying exiting functionality in the firmware code, most of the time forking and modifying the official firmware unfortunately is the only solution. We are however working our way to make more and more of the firmware modular so that it can be expanded out of tree. For example there has been work to make an out of tree estimator possible to implement.
A prototype written as a python script is often pretty easy to move to the Crazyflie firmware. This is a good pattern when writing an application, rapid prototyping in python and then finalizing in firmware if needed. The best example of that is the push demo. It is a demo where the Crazyflie can be pushed-around with the help of the flow deck for autonomous flight and the multiranger deck for detecting obstacle/hands. We have a python cflib push demo as well as a Crazyflie firmware push demo app.
There is some support in the python lib for interacting with multiple Crazyflies and it is probably a good start point for simple swarms. For more advanced swarm work Crazyswarm may be a better option.
If you would like to see some of the process in action, we have made a workshop during our BAM days about implementing functionality both using the python lib and in the firmware as an out of tree app:
During the preparations for BAM, for example the awesome demo that Kristoffer did (link to demo), we had a lot of discussions about landing on charging pads. The Qi deck is a good solution, but the design is a bit complex and makes it hard to have other electronics pointing dowards (although we’ve seen some solutions to this). So after spending a few days thinking about this, I set out to spend a few fun Fridays looking at solutions for this. The results (so far) are detailed in this post.
Contact charger and decks
The idea I decided to try out was something we’ve been discussion on-and-off for a few years, using contact charging instead of Qi. So I revived a few of our old ideas and starting looking for parts. This is always a fun phase of an idea, the sky is the limit! I’m kind of used to electronic parts, but I find looking for mechanical parts is a lot harder. I don’t really have the vocabulary for this and all the parts look the same size in the product photos…
Finally the solution I wanted to test was using small pogo-pins. The idea is that the pogo-pins could be included into any design of new boards and easily enable charging. With the parts found and the ideas fresh again, I jumped into KiCad (5.99) and started the design. Two different designs were made, pictured below.
4 segments (2xGND and 2x5V)
Two circles, one GND and 5V
Why two solutions? I wasn’t sure which one would work well. I suspected the 4 segments would be better, but I was hoping for the two circles, since you would not need additional electronics to handle 5V/GND switched depending on how you landed.
Since there was two solutions for the pad, there’s also two solutions for the deck. I wasn’t sure about what pogo pins to use, so there’s two different sets of pins (which makes it a bit confusing).
Deck for segmented charger (with diode bridge on back)
Deck for circular charger (without diode bridge)
With the components and PCBs in hand I started testing, and…the results were not promising. The design was made to work with our current Qi charging pad (link). Although the alignment for Qi is important, the tolerance was not as tight as what I needed. Back to the drawing board, this time the mechanical one. I used FreeCAD to create a new version where the Crazyflie would align better. After a few iterations with the 3D printer I ended up with this:
Updated charge pad design
Crazyflie on 3D printed charging pad
Putting it all together, the results were promising. With the new mechanical design the area the Crazyflie can land in is larger and the alignment much better. I think it’s good enough to show that the concept can work. The next step is to look a bit more at the pins, they might be a bit too easy to damage.
Crazyflie on circular charge pad
Charge pads and decks
The Glow deck
I mentioned above that the idea was to be able to put charging on decks with other designs on them. To test out this concept I revived another old idea, the Glow deck! The Glow deck is a design where the LEDs are visible from the side and also a bit stronger. To achieve this a diffuse lens would be mounted on the PCB.
The first version of the Glow I made was years ago. It contained a high-power LED, which was really bright…and really warm. We stopped measuring the PCB temperature at 140 C, so that wasn’t something that was going to work out. It also only had one color and I wanted more.
This time around I decided to try two different versions. The reason for the two versions was that I wanted to see if more power actually added something (except complexity and price) when it came to visual effects while flying. One version uses 4xWS2812B LEDs (the same as on the LED-deck) while the other one uses a RGBW high-power LED. I quickly ran into the same problem as with the charging deck, mechanics. I wasn’t able to find a good size of a diffuse lens. Not wanting to be blocked by this I decided on designing one myself and printing it. Not sure how the size would affect the effect I designed two versions.
2 versions of diffuse lens (3D printed)
Left: WS2812B version, right: high-power RGBW version
I haven’t had the time to try out the high-power LED version yet, but the results for the WS2812B version with the diffuse lens looks promising. It’s hard to see the effect in the images, but at least it shows that it’s possible to see the LED from the sides.
Conclusions
I think both the contact charging and the Glow looks promising and I’m happy with the results so far. But making one prototype is easy, making something that can be manufactured and properly sourced is different. If any of these ever turn into a product, the challenge will be finding the right pins, lens, charge base etc.
For the next step, I’m excited to see the high-power version of the Glow in action. This time around I’ve added proper dimming and temperature measurement :-) If you have any ideas, comments or questions drop a line below!
From Star Wars to Black Mirror, sci-fi movies predict a future where thousands of drones will fill our sky. Curving sharply around trees or soaring over buildings, they fly just like a flock of starlings. To turn this vision into a reality, real drone swarms need to increase their autonomy and operate in a decentralized fashion. In a decentralized swarm, each robot makes its own decision based only on local information. Decentralization not only allows the swarm to be more robust to the failure of single individuals, but also removes the dependency from a single computing unit, thus making the swarm more scalable in terms of size.
We at LIS (EPFL) have shown that predictive controllers can improve the safety of aerial swarms by predicting and optimizing the agents’ future behavior in an iterative process. However, the centralized nature of this method allowed us to only control five drones and prevented us from scaling up to a large number of drones. For this reason, we have worked on a novel decentralized and scalable swarm controller that allows the safe and cohesive flight of aerial swarms in cluttered environments. In our latest article, published in IEEE Robotics and Automation Letters (RA-L), we describe how we designed the controller, show its scalability in size, and demonstrate its robustness to noise. We studied the swarms’ performance and compared how it changes in two different environments: a forest and funnel-like environment.
The Crazyflie 2.1 was the perfect platform for our experiments. They are lightweight, modular, and tough. This quadcopter can survive big hits when things don’t go as planned… and, if you work on swarms, things can go wrong!
The fleet of Crazyflies equipped with a single marker.
With our algorithm, sixteen robots were able to fly through an artificial forest that we set up in our indoor motion capture arena. In our previous work, we installed four markers on each quadcopter and used the rigid body tracking from Motive (the Optitrack software). The large volume of our experimental room required the usage of big markers for long-distance detection, which added considerable weight to the drone. Hence, in our new work, we use a single marker per drone. Tracking is supported by the ‘crazyswarm’ package and communication with the entire swarm only requires two radio links. However, despite our model being decentralized, in our implementation robots relay the information to an external brain, which does the computations for them. In the future, all the necessary code will be embedded onboard, removing the dependency on external infrastructure.
Our predictive swarm of Crazyflies flying among obstacles in our indoor experimental room.
Video about the article
This work is a step forward towards the fully autonomous deployment of drone swarms in our cities. By enabling safe navigation in cluttered environments, drone fleets will be able to integrate with conventional air traffic, search for missing people, inspect dangerous areas, transport injured people to hospitals quicker, and deliver important packages right to our doors.
We have recently worked on functionality in our web site to generate documentation from source in a few ways that we hope will improve the quality as well as simplify maintenance. We have already written a bit about the log and param documentation in the crazyflie-firmware repository, but we now also added and API reference in the python library as well as generating a list of publications related to the Crazyflie.
The Log and Param documentation
Earlier this year we worked on generating documentation for Log and Params from doxygen comments. We will not dig deeper into this here, but you can read more about it in this blog post. The latest version is available on our web in the repository documentation for the crazyflie-firmware: logs and params
However along the way we have also added comments and examples to our code. And this, combined with the way we have structured the library actually enables us to automagically generate reference API documentation. That is, something that shows you everything that is possible to do with the library, all modules and classes, all methods and constants that the library offers.
After some recent work this is now happening and the documentation will now get generated each time we deploy our website!
The API reference documentation can be found in the repository documentation for the library. Please check it out! And be picky, complain where the documentation is lacking! Or if the formatting seems weird! We are trying to get the hang of this and we need you to push us!
Managing publications
Some years ago we started to add publication that are related to the Crazyflie to our Research page, we hope that it might be inspiring to read about all the awesome things that the Crazyflie is used for.
Until now it has been a simple list in markdown but with an increasing number of publications it has become harder and harder to maintain it, finally we have put the work into generating the list from a BibTeX (.bib) file instead. One advantage of the new solution is that BibTeX is a well known format to the research community with lots of tools around to manage BibTeX files while another improvement is that the list will be formatted in a consistent manner (which was not always the case earlier).
If you want to add a publication to the page, simply update the .bib file with your data and create a pull request with the changes. We will merge when appropriate and the publication will become visible on the web after the next deploy, usually within a few days.
Behind the scenes
The code for generating documentation from source tends to spread out over multiple repositories and creates some complexity with a multitude of tools for different languages. It should not be necessary to understand the details and we hope the system will be easy to use for contributors to the code base.
Only 8 days left before the first Bitcraze-organized conference ever. We are really excited about it, and this coming week is pretty much dedicated to organize the conference. We’re working hard on getting the best experience possible for there 3 days, both for us and for you. That means collaborating with great speakers, creating awesome workshops, but also think about the best way to come together – one of the main purpose behind the BAM days.
It’s been a couple of months since we decided that we wanted to organize our own conference. The main idea behind it was that we missed going to conferences, meeting people and talking with a lot of interesting persons.
Expression of the BAMdays
Bitcraze turning 10 felt like a big occasion that we wanted to celebrate. And indeed we did, internally, but we wanted to share this accomplishment with you too. An online conference seemed like the best idea. So when time came to choose a platform for it, we looked for something which put emphasis on social experience. We finally went with the Welkom platform, from the Netherlands. It’s easy and natural to use, with a simple interface, and lots of options for us to organize different activities.
The event platform
The Welkom platform we’ll use hosts two different Mibo rooms. It’s a spatial chat, where your avatar can run around a 3D world, discovering different environments. Your camera is your head, so you’ll need a webcam to join. There are some fun activities around, plenty interactive stuff, from chairs to basketballs. We have used it to for our coffee breaks when we were working from home, and had fun using this world: we’ll be happy to meet you there and hope you’ll find it fun too.
Bitcraze goofing around in Mibo
The plan is to have between each talk or workshop at least 30 minutes in Mibo to talk to the speakers, take a coffee break, discuss the exciting things that’s been happening, and generally catch up in ways we haven’t be able to the past year.
At the end of each day, we’ve programmed social events to gather together. On Tuesday, we’ll host a special quizz. Telling you its content may be a bit cheating, but if we’ve read our blogposts regularly, the questions should not seem hard to you. For the winner, a once-in-a-lifetime reward: a GoldenFlie ! You will be able to put it in your chimney mantle with your others awards, or actually use it – because, yes, it flies!
The GoldenFlie!
The second day, Wednesday, we’ll have a roundtable with Q&A. A perfect occasion for us to get feedback from you, and for you to ask us all the questions you have been burning to ask us.
The last day (Thursday), we’ll talk a little about Bitcraze and its future: where it is headed. We’ll then have a last party – in Mibo of course.
All the program and other useful information is in our event page. We really hope you’ll take some time to come by and say hello !
Last week it was time for the International Conference on Intelligent Robots and Systems (IROS), which is one of the biggest yearly robotics conferences! In previous editions (see 2018 blog post, 2019 event page), we joined IROS’ exhibition and showed an autonomous demos for everybody to enjoy during the coffee breaks. Then… as everybody knows… the Covid-19 pandamic hit and we had to cancel our plans to go to the 2020’s Las Vegas edition. This year’s IROS was supposed to be held in Prague, but was very early announced that this was going to be an online edition.
Eventhough we considered joining IROS’s online exhibition, after our decision to organize the Bitcraze Awesome Meetup (BAM) days as a celebration to our 10 year anniversary (see this blo gpost), we wouldn’t have enough time to prepare for both. However, we still signed up for the conference itself, so we could see how and where the Crazyflie is used in the robotics community! So this blog post will give a list and small overview of the Crazyflie-fueled research with some highlights.
Crazyflie as a Demonstration Tool in Research
A common use-case that we saw is to use the Crazyflie as a research demonstration tool. In the case of designing algorithms for quadcopters, many of the researchers we heard from or talked to told us that there is significant added value to demonstrate their ideas on a real platform. Then it can be truly evaluated with real environmental issues and platform dynamics. For this, the majority of the papers have used the ROS package developed for the CrazySwarm project but some researchers has gone as far as implementing it fully on the Crazyflie’s MCU [6, 8]!
We also noticed the large spread of the Crazyflie-featured papers throughout the sessions of the conference. A bunch of the papers could obviously be found in Aerial System sub-sessions as in ‘Perception and Autonomy’ [1], ‘Applications’ [3], ‘Mechanics and Control’ [4 , 5], but many were spread out in areas like Reinforcement Learning [2], Localization [6] and Collision avoidance [7]. There were also some papers to be found in the Swarm Robotics session [8, 9], as well as in Distributed Robot Systems [10].
Note that there is usually quite some overlap between the different sessions, but it is still very exciting to see that the Crazyflie being used in so many fields!
Highlights and the Community
We saw lots of awesome applications but there are a few that we really like to highlight! There was a presentation of an actual solar-powered Crazyflie [2] which can literally fly forever. This research was done by the same group at the University of Washington that also was featured in a blog post last year, and who is responsible for the Smellicopter. Also, our jaw literally dropped at the sight of the 4 Crazyflie-fueled mega-copter by the University of California, Los Angeles [4, 5]. Last but not least, the fully onboard autonomous gas-seeking swarm, Sniffybug [8], from the MAVLab (TU Delft), never stops to amaze us. They wrote a blog post about their work on our website this summer!
We are hoping for more interesting blog posts related to the papers in the list below in the future. We also would like to invite all the researchers, who have been working with the Crazyflie, to join us for discussions at the Community Q&A at the BAMdays. Most of the amazing aspects of the Crazyflie in terms of autonomy, localization and swarming have all been contributions of the research community in the past. So join the discussion to continue that path in order to bring the Crazyflie to a new level!
List of IROS 2021 Papers featuring the Crazyflie
Here is a list of all the Crazyflie-related papers we could find, but please let us know if we are missing any!
[1] Target-Visible Polynomial Trajectory Generation within an MAV Team Yunwoo Lee, Jungwon Park, Boseong Jeon and H. Jin Kim
Lab for Autonomous Robotic Research (LARR), Seoul National University
[3] Toward battery-free flight: Duty cycled recharging of small drones Nishant Elkunchwar, Suvesha Chandrasekaran, Vikram Iyer and Sawyer B. Fuller
Department of Mechanical Engineering, University of Washington
[4] An Over-Actuated Multi-Rotor Aerial Vehicle with Unconstrained Attitude Angles and High Thrust Efficiencies Pengkang Yu, Yao Su , Matthew J. Gerber, Lecheng Ruan and Tsu-Chin Tsao
[6] A Computationally Efficient Moving Horizon Estimator for Ultra-Wideband Localization on Small Quadrotors Sven Pfeiffer, Christophe de Wagter and Guido C.H.E. de Croon
[9] micROS.BT: An Event-Driven Behavior Tree Framework for Swarm Robots Yunlong Wu, Jinghua Li, Huadong Dai, Xiaodong Yi, Yanzhen Wang and Xuejun Yang
Artificial Intelligence Research Center, National Innovation Institute of Defense Technology, Beijing
[10] Neural Tree Expansion for Multi-Robot Planning in Non-Cooperative Environments Benjamin Riviere, Wolfgang Honig, Matthew Anderson and Soon-Jo Chung
[11] Trust your supervisor: quadrotor obstacle avoidance using controlled invariant sets Luigi Pannocchi, Tzanis Anevlavis, Paulo Tabuada
University of California, Los Angeles
[12] Continuous-time Gaussian Process Trajectory Generation for Multi-robot Formation via Probabilistic Inference Shuang Guo , Bo Liu , Shen Zhan , Jifeng Guo and Changhong Wang
Harbin institute of Technology
[13] Non-Prehensile Manipulation of Cuboid Objects Using a Catenary Robot Gustavo A. Cardona , Diego S. D’Antonio , Cristian-Ioan Vasile and David Saldana
Previously we have been using off the shelf scales and other methods to measure characteristics, such as thrust or efficiency, of the Crazyflie products. We thought it was time to build something that is easier to use, more repeatable and tailored to our needs. Well, this has been on our wanted list for a long time, already back from when we did the RPM-deck. It was however first when Wolfgang visited us this winter that he nudged us over the edge so we finally allocated some time for it. We started off by buying some load cells and breakout boards to do something simple as a start, so we could at least measure thrust more easily. We actually started looking for off the shelf thrust stands but could not find anything suitable for the Crazyflie’s size. As is often the case here at Bitcraze, the project grew. Already before we had any load cells up and running I was designing a deck with RPM sensors, load cell amplifier and power meter. Now with the objective to easily do system identification. Therefor we named the deck the system-id deck.
For the RPM sensors we used the same as on the RPM-deck, the QRD1114. They are not great as they need a reflective surface, this means adding white stickers or paint to black propellers, but they work well enough. The load cell amplifiers ended up to be the NAU7802 as it has a high accuracy and sample rate. For power metering we chose the new ACS37800 power monitoring IC that can handle up to 30A, this looked exiting.
The QRD1114 we wired the same way as previously done on the RPM-deck:
The NAU7802 was configured as per the datasheet suggestion and similarly to other open designs out there:
The ACS37800 was very new so the datasheet had to be used as the main information source. A bit tricky as this chip is mainly intended to measure mains supply, and we wanted to measure low voltage DC, which it said it could do…and in the end we managed to get it working.
We also added a buck/boost DC/DC that could provide a stable 3.3V from 2-5V input, just in case, as the ACS37800 is specified for this voltage and not the 3.0V the Crazyflie can supply.
The outcome
The PCB was designed as small as possible so it could be mounted on a Crazyflie 2.X and used while flying. A bonus would be if it could be used on a Bolt as well.
Here it is mounted on a Crazyflie 2.1 together with a 3D printed stand and load cell.
The load cell would then be mounted to a desk or similar so the the Crazyflie is mounted up-side down, pushing down on the load cell.
Software
The software, as often, took most of the time to make. Three major deck driver files was created, rpm.c, acs37800.c and loadcell_nau7802.c. Aside from these there where only small changes to make, like making it work when being up-side-down. The modifications have all been pushed to the dev-systemid branch for those that are interested. As for now we are mainly using the logging framework to transfer the data to the PC, which is quick and easy to setup and use, but writing to SD-card is also possible. The scripts for this can be found in the tools/sytem_id folder.
Remaining work is to test, gather and analyse more data. When we have done so, we will post more. Until then below is a sample of what it can measure. The data is taken with a ramping PWM from 0% – 100% – 0%. The added resistance of the extra wires and connectors are not taken into account, but the estimated efficiency of 4g/W is probably not that far off.
How to handle our documentation has been always a bit of struggle. For almost 2 years (see this blogpost and this one) we have working on improving the documentation structure, with by transferring information from the wiki, putting information closer to the code and setting up automating documentation. A few months ago, we managed to have automated logging and parameter documentation (see this blogpost).
Even though we think there is some improvement already, it can always be better! We have noticed that some of our users are a bit confused of how to go through our documentation. So in this blogpost we are discussing some navigational strategies of how you can maneuver yourself through the documentation as it is presented on bitcraze.io, which can also be found here.
Ecosystem-based navigation
So more than a year ago, we also started with a Ecosystem overview page, which are meant to take first-timers by the hand through the Crazyflie ecosystem.. This type of overview pages are starting from the three main pillars: the Crazyflie Platform, the Clients and Positioning Technology. This is a type of navigation that we mostly advise to take if you are a beginner Crazyflie user who do not know the structure of the eco system fully.
For those that already have experience with the Crazyflie and its Ecosystem, the previous way of navigating through the docs might be a bit convoluted. With the Ecosystem-based navigation, it takes about 3 scrolls and clicks to reach the STM development documentation, which is a bit to much of a round way if you already know what you are looking for. We have made the repository overview page not for this purpose but we actually started using ourselves a lot within the company, as a direct pathway to the development repository per element. So this is a page that would be useful to other advanced developers as well!
So the repository overview page is split up in 4 main categories: Python-based software, C-based firmware, Other languages and bootloaders. See the navigation tree which of those repositories approximately point too. By the way, have you noticed that repository documentation has a gray header (like this one) and all the overview pages on the web have a green header (like this one)? This are meant to make you aware if you are still on a fluffy overview website page or going in the nitty gritty details of the development documentation.
Repository-based navigation tree
Feature-based navigation ?
Still a remaining problem is that the repository documentation might not be enough to get a good overview. Where do you need to look if you are interested in ‘controllers’ or ‘state estimators’, or how to make an app-layer application? Currently all of this is within the stm32 firmware documentation, as that is the exact location of where all of this is implemented. But how to document spanning features like the CRTP, where not only the STM chip but also the NRF, Crazyradio PA and the Crazyflie python library are also involved? Or how about the loco positioning system, where the Crazyflie communicates through the LPS deck with a separate LPS node?
So perhaps a good way how to present all this information, is to do it feature-based, like ‘controllers’, ‘positioning’, ‘high level commander’, where we present a structure that points to parts of the detailed documentation within the repo-docs. With ecosystem-based, or even repository-based, navigation documentation strategy, it will take for instance 4-7 clicks to come to the specific controller page, as you can verify by looking at the bread-crumb of the header. Perhaps splitting it up based on feature instead of Ecosystem elements or programming language might be a more logical structure of the current state of the Crazyflie documentation.
Feedback
One reason why it is so difficult to do this properly, is that we have a lot of repositories based on each microprocessor of all of our products, which makes our opensource projects quite unique. It is therefore difficult to find another opensource project of which we can take inspiration from. So, let us know what you would prefer for navigating through our documentation in this poll, but we are always open to other suggestions! If you know of any example of a similar opensource software project that is doing it the right way, or have any other tips, send us an email (contact_at_bitcraze.io), contact us on social media platforms or post a comment on this blogpost!
Update 2021-12-21:
The poll is closed and this is the result! Thanks all for responding!
We like to describe the Crazyflie as a versatile open source flying development platform. It is something that enables you to do cool stuff. It is not a finished, polished end product in its own.
This approach makes determining the expectations and requirements for the platform and the surrounding ecosystem a bit tricky. It is dependent on what you as a user plan to create using our products. And since the ecosystem is growing we need a, scalable, way to handle these fuzzy expectations during development and maintenance.
We think testing is big part of solving this, testing in a systematic, scalable and reproducible way. This is the reason for setting up our first physical test lab, aimed at providing stability while moving forward with new products and features.
Testing today
Continuous integration
On each change proposal to our software we run tests. For the firmware in the device we build multiple different configuration and run unit-tests. For the Crazyflie client and the Python library we make sure we can build for Linux and Windows and that the code passes our style guidelines. If any test fail we go back and update the proposed changed and re-run the tests. This is our first level of defense against defects.
Release testing
For each release we follow a checklist of procedures and tests to make sure that quality does not degrade. We make sure all the examples in the Python library are working and that the Crazyflie can fly around as expected in our flight arena, using various positioning systems.
Limitations
The way we test today makes it very difficult to determine if we regress in, for example, flight capability or in radio communication quality. We either test different software packages in isolation, without hardware in the loop, or we test by having a human trying to estimate if any degradation has happened since the last release.
We run into scalability issues as our ecosystem grows, it is near impossible to test all the different combinations of products. And it is very hard for us to detect stability or quality regressions without having a more systematic approach to testing the software on relevant hardware.
Introducing the test lab
What we have done so far are two things:
Created an infrastructure for setting up a site for testing our software on real devices
Prepared a test lab in our printer room at our office in Malmö
This new Git repository contains the building blocks we need to setup our new test lab. You can check out the repository README.md file for information on how to run it.
The repository contains a way to specify a test site, which is a collection of devices to run the test suite against. You specify your site in a TOML file which contain information about each device, such as which decks are connected and the radio:// URI. This site specification is then used by the test framework when running the tests, making sure each test is run against all devices compatible with that test.
The infrastructure also has management scripts to perform tasks like flashing all devices in a site, or recovering them if they go into boot loader mode by accident. All aimed at being able to handle testing with the least amount of human intervention possible.
This acts as a safety net for us. We will quickly know if the communication performance degrades, or if we have messed up with our parameter or logging frameworks. And we can catch silly firmware bugs as early as possible.
Future
We want to keep adding test cases and other infrastructure to our testing framework. Going forward it would be really nice to be able to test positioning systems in some way. And of course, some type of test of flight, be it free flying tests or using some kind of harness.
It might be interesting to look into adding simulations (hardware in loop or not) to our testing setup. It is all a question of bandwidth, there are a lot of cool things to work on, and a limit on time and bodies.
You can help! You might even be able to help yourself while helping us! If you contribute tests that correspond to your use-case, then you can relax knowing that those tests will run each night, and that Bitcraze engineers will be notified the minute they fail.
Or you can define your own site and run the test-suite against all your devices to make sure nothing strange is going on.
Hopefully this infrastructure and lab will help all of us to do more cool stuff using the Bitcraze ecosystem!
