Author: Kimberly McGuire

Home / Blog

State estimation: To be or not to be!

  2020-01-20  |     |    2 Comments

How does a Crazyflie manage to fly and stay in the air in the first place? Many of us tend to take this for granted as much research tend to happen on the application level. Although we try to make the low level elements of flight as stable as possible, it might happen that whatever you are trying to implement on the application level actually effects the Crazyflie on the low level controls and estimation. We therefore would like to focus a little bit on the inner-workings of the autopilot of the Crazyflie, starting with state estimation. The state estimation is part of the stabilizer loop in the Crazyflie, an overview of is was made in a previous blog post.

State estimation is really important in quadrotors (and robotics in general). The Crazyflie needs to first of all know in which angles it is at (roll, pitch, yaw). If it would be flying at a few degrees slanted in roll, the crazyflie would accelerate into that direction. Therefore the controller need to know an good estimate of current angles’ state and compensate for it. For a step higher in autonomy, a good position estimate becomes important too, since you would like it to move reliably from A to B.

There are two types of state estimators in the crazyflie firmware, namely a Complementary Filter and an Extended Kalman Filter.

Complementary Filter

The complementary filter is consider a very lightweight and efficient filter which in general only uses the IMU input of the gyroscope (angle rate) and the accelerator. The estimator has been extended to also include input of the ToF distance measurement of the Zranger deck. The estimated output is the Crazyflie’s attitude (roll, pitch, yaw) and its altitude (in the z direction). These values can be used by the controller and are meant to be used for manual control. If you are curious how this code is implemented exactly, we encourage you to checkout the firmware in estimator_complementary.c and sensfusion6.c. The complementary filter is set as the default state estimator on the Crazyflie firmware.

Schematic overview of inputs and outputs of the Complementary filter.

Extended Kalman Filter

The (extended) Kalman filter is an step up in complexity compared to the complementary filter, as it accepts more sensor inputs of both internal and external sensors. It is an recursive filter that estimates the current state of the Crazyflie based on incoming measurements (in combination with a predicted standard deviation of the noise), the measurement model and the model of the system itself. We will not go into detail on this but we encourage people to learn more about (extended) Kalman filters by reading up some material like this.

Schematic overview of inputs and outputs of the Extended Kalman Filter

Shortly said, because of the more state estimation possibilities, we preferred the Kalman filter in combination with several decks: Flowdeck, Loco positioning deck and the lighthouse deck. If you look in the deck driver firmware (like for instance this one), you see that we set the required estimator to be the Kalman and that is of course because we want position/velocity estimates :). Important though is that each input of the measurement effects the quality of the position, as positioning of the Lighthouse deck (mm precision) is much more accurate that the loco positioning deck (cm precision), which has all to do with the standard deviation of the measurement of those values. Please check out the content of estimator_kalman.c and kalman_core.c to know more about the implementation. Also good to know that the Kalman filter has an supervisor, which resets if the position or velocity estimate is gets out of hand.

Of course this blogpost does not show the full detailed explanation of state estimation, but we do hope that it gives some kind of overview so you know where to look if you would like to improve anything. The Kalman filter can easily be extended to accept more inputs, or the models on which the estimates are based can be improved. If you would like implement your own filter, that would be perfectly possible to do so too.

It would be great if you guys could share your thoughts and questions about the state estimation on the crazyflie on the forum!

Happy Holidays!

  2019-12-23  |     |    1 Comment

Happy holidays to all our users, community members and friends! We are happy to announce our 2019 Christmas video which we have made in collaboration with Ben Kuper! It is starring 7 Crazyflies, the lighthouse positioning system, our office Christmas tree and a whole lot of holiday spirit, so go ahead and take a look!

Here are some words from Ben how it was to work on this year’s Christmas video at our office:

Coming to Bitcraze’s HQ and working with them has been once more a wonderful experience, technically and humanly ! The main goal of this session was to test and implement the new lighthouse tracking system in the tool suite I’m creating, and it was an amazing surprise to witness for real the uncanny stability of the drones when they’re on lighthouse tracking !

Of course, my first reaction was to push the limit and see what can be done with this new power, this is why I created this choreography : to see what can be done in a limited amount of time (1 and a half day to create the full choreography, the official video shows the first part only), and trying to go at the limit of the current possibilities. As the team was working on occlusion recovery, we decided to have the drone fly around the tree as a fun test, and it works !

In the new year we will have a followup blog-post going into detail on how exactly we made this video. Until then, happy holidays and have an awesome new year!

Enabling Swarm Exploration

  2019-10-28  |     |    Leave a comment

For the last four years of doing my PhD at the TU Delft and the MAVlab, we were determined to figure out how to make a swarm/group of tiny quadcopters fly through and explore an unknown indoor environment. This was not easy, as many of the sub-challenges that needed to be solved first. However, we are happy to say that we were able to show a proof-of-concept in the latest Science Robotics issue! Here you can see the press release from the TU Delft for general information about the project.

Since we used the Crazyflie 2.0 to achieve this result, this blog-post we wanted to mostly highlight the technical side of the research, of the achievements and the challenges we had to face. Moreover, we will also explain the updated code which uses the new features of the Crazyflie Firmware as explained in the previous blogpost.

A swarm of drones exploring the environment, avoiding obstacles and each other. (Guus Schoonewille, TU Delft)

Hardware

In the paper, we presented a technique called Swarm Gradient Bug Algorithm (SGBA), which borrows (as the name suggests) navigational elements from the path planning technique called ‘Bug Algorithms’ (see this paper for an overview). The basic principle is that SGBA is a state-machine with several simple behavior presets such as ‘going to the goal’, ‘wall-following’ and ‘avoiding other Crazyflies’. Here in the bottom you can see all the modules were used. For the main experiments (on the left), the Crazyflie 2.0’s were equipped with the Multiranger and the Flowdeck (here we used the Flow deck v1). On the right you see the Crazyflies used for the application experiment, were we made an custom Multiranger deck (with four VL53L0x‘s) and a Hubsan Camera module. For both we used the Turnigy nanotech 300 mAh (1S 45-90C) LiPo battery, to increase the flight time to 7.5 min.

Hardware used in the experiments. Adapted from the science robotics paper.

Experiments

With this, we were able to have 6 Crazyflies explore an empty office floor in the faculty building of Aerospace engineering. They started out in the middle of the test environment and flew all in different preferred directions which they upheld by their internal estimated yaw angle. With the multi-rangers, they managed to detect walls in their, and followed its border until the way was clear again to follow their preferred direction. Based on their local odometry measurements with the flowdeck, the Crazyflies detected if they were flying in a loop, in order to get out of rooms or other situations.

A little before half way of their battery life, they would try to get back to their initial position, which they did by measuring the Received Signal Strength Intensity of the Crazyradio PA home beacon, which was located at their initial starting position. During wall-following, they measured the gradient of the RSSI, to determine in which directions it increases or decreases, to estimate the angle back the goal.

While they were navigating, they were also communicating with each-other by means of broadcasting messages. Based on those measurements of RSSI, they could sense other Crazyflies approaching, which they first of all used for collision avoidance (by letting the low priority CFs move out of the way of the high priority CFs). Second of all, during the initial exploration phase, they communicated their preferred direction as well, so that one of them can change its exploration behavior to not conflict with the other. This way, we tried to maximize the explored area by the Crazyflies.

One of those experiments with 6 Crazyflies can be seen in this video for better understanding:

We also showed an application experiment where 4 crazyflies with the camera modules searched for 2 dummies in the same environment.

Challenges

In order to get the results presented above, there were many challenges to overcome during the development phase. Here is a list that explains a couple of the elements that needed to work flawlessly:

  • Single CF robustness: We used the Flowdeck v1, for the ‘deadlock’ detection and the basic velocity control, which was challenging in the testing environment because of low lighting conditions and texture. Therefore the Crazyflies were flying at 0.5 meters in order to ensure robustness. The wall-following was performed solely using the Multiranger. This was tested out in many situations and was able to handle a lot of type of obstacles without any problem. However the limited FOV of the laser range finder can not detect all types of obstacles, for instance thin ones or irregular ones such as plants. Luckily these were not encountered in the environment the Crazyflies flew in, but to increase robustness, we will need to consider adding a camera to the navigational drive as well.
  • Communication base-station. SGBA by essence only needs one base-station Crazyradio PA, since all the behavior is completely on board. However, in order to show results in the paper, it was necessary for the CF to communicate information back, like odometry, state and such. As this was a two way communication (CFs needed RSSI to get back) each Crazyflie needed 1 base-station. Also, they all needed to be on different channels to avoid package collisions and RSSI accumulation.
  • Communication Peer to Peer. At development time, P2P didn’t exist yet, so we had to implement broadcast communication between the Crazyflies. Since the previous pointer required them to listen on different channels, the NRF had to be configured to send separate broadcast messages on all those channels as well. In order to time this properly, the home beacon had to sync the Crazyflies accordingly by sending out a timer. Even so, the avoidance maneuvers were done very conservatively to try to prevent inter-drone collisions.

Many of the issues, especially the communication challenges, will be solved with the updated code implementation as explained in the next section.

Updated code

The firmware that the Crazyflies used to fly in the experiments showed in the paper, can all be found in this public repository. However, the code is based quite an old version the current Crazyflie firmware, as it was forked almost a year ago. The implementation of the SGBA state machine and the P2P broadcasting were not generic enough to integrate this back to the development cycle, therefore the current code is only suitable for the old Crazyflie 2.0.

Therefore, we developed two major changes in the latest firmware which will make it much easier for me (and other ideas as well we hope!) to implement SGBA and the P2P communication in a way that should be compatible with any version of the firmware (and hardware) from here and on. We implemented SGBA as an app-layer and also handled all the broadcast messaging directly from this layer as well. Please check out this Github repository with this new app layer implementation of SGBA.

Excited about IMAV 2019

  2019-09-23  |     |    Leave a comment

The Crazyflie Bolt and the Crazyflie 2.1 with the lighthouse deck are coming to Madrid!

Only one week away until the start of the big Bitcraze Conference frenzy, with the first stop… Madrid! We will visit the Micro Air Vehicle Competition and Conference, which is a robotics event that is more specialized in (as the name implies) MAVs! So it should be right up our alley. This is the first time that we attend as Bitcraze, although the writer of this blog post has experienced fun times at the conference and the competition as a participant with her previous lab, the MAVlab.

The IMAV has been around for almost 12 years, starting in Toulouse, France in 2007. Although it initially mostly was held in various places in Europe, in 2016 into a more worldwide phenomenon by making it’s tribute in Beijing, China and Melbourne, Australia in 2018. It hosts a conference to which researchers can send their work in anything related to MAVs, from autonomous navigation, state estimation and design.

IMAV is mostly know for hosting big indoor and outdoor competitions for MAVs. The outdoor competitions can range from survey tasks to finding a hidden person or object. This year the focus will be on the delivery of packages from one place to another. The judges will look at how many packages that can be safely delivered and if the drone is able to detect certain objects in the outdoor environment. The indoor competition is oriented around the application of MAVs in a warehouse. They should be able to take off autonomously, monitor boxes in shelves and make an innovatory, and pickup packages to release them in their designated location. 40 teams of 28 universities will show their awesome implementation in these difficult tasks.

We will have a booth at the main company fair at the conference and indoor competition, and will also be present at the outdoor competition day as well. We will bring the lighthouse positioning system and show the awesome swarming demo we developed. Also we want to bring the new Crazyflie Bolt with us, which we are sure of that the regular IMAV crew will love. If you are at the IMAV between the 30th of September to the 4th of October, come by and say hello!

Autumn conference schedule

  2019-08-26  |     |    Leave a comment

We have a busy schedule in the fall, since we are planning to travel all around the world for all kinds of awesome conferences and symposia. Here is a quick overview of where we are planning to go in October and November.

IMAV Madrid 1/10-4/10

We will attend the IMAV 2019, and this time it is in Madrid (Spain) from the 1th to the 4th of October. This is a conference for anything related to micro-aerial vehicles and also includes an awesome indoor and outdoor competition. Arnaud and Kimberly will be present at both the indoor and outdoor fair, where we will show some demos with the lighthouse system and show off our new Crazyflie Bolt!

https://imav2019.org/

Symposium Seoul 14/10 – 15/10

In collaboration with our partner CLRobur, we will have a symposium in Seoul (Korea) on the 14th and 15th of October. Kristoffer will explain the attendees all about positioning systems, and show off the LPS and our new lighthouse deck.

Sign up here or send an email to clrobur@naver.com

IROS Macau 4/11-8/11

Last but not least, we will be there at IROS 2019 as well! This is one of the flagship robotics conferences of IEEE. In Macau (China) from the 4th till the 8th of November, Marcus and Tobias will show some awesome demos with the lighthouse system and hopefully we will present the new AI deck!

https://www.iros2019.org/

Forum problems

  2019-07-29  |     |    Leave a comment

A lot of you are using our forum on a regular basis. From our point of view, we prefer giving support and guidance on our forum instead of by email, since solutions we provide to our users can be found by others as well. However, we are experiencing some spam-related issues with the forum!

Somewhere last week, our forum started getting some weird posts, including some links to websites that are…. not exactly Bitcraze or Crazyflie related… We didn’t think about it too much, so we cleaned the forum, deleted the users responsible and moved on. But the very next day more spam post appeared, and it was getting so bad that we changed our sign-up from the regular security question to Captcha. Unfortunately, spambots are way too smart these days, so that didn’t work either… By the end of the weekend we had to remove over an hundred spam accounts and 500 spam posts!

The forum is experiencing a major spam attack at the moment and the usual security measures are not helping. Since yesterday, we enabled manual activation of new users (meaning that we physically need to approve each account), since currently this is the only way to catch the dodgy new accounts (we already caught over 100 in one day alone !!). This week we will look into other measures and plugins to better secure our forum, so we don’t have to manually go through all the newly registered users to see if they are not spam bots. We heard about Cleantalk by one of our users, so we probably will give that a go!

However, please be contact us if the following occurred or is occurring to you in our spam removing frenzy:

  • If we accidentally removed a post of you
  • If we removed or banned your account
  • If we are not activating your new account

Also make our lives a bit easier this week, and send us an email straight away if you have just have made a new account on the forum. Send an email to: contact@bitcraze.io with your username and a sentence about why you are using the Crazyflie (so we know that you are real).

This is only a temporary solution, so hopefully at the end of this week or the next we will not need to worry about this anymore :).

Improving Documentation

  2019-07-22  |     |    Leave a comment

Improving the flow of information

Our usual blog posts usually consists of the awesome new products and demos that we make here at Bitcraze, but now we will talk about… documentation! Alright alright, it is maybe not the most thrilling topic, however you should be excited about it! Good documentation about the Crazyflie and its tools will not only enable you to recreate the demos and the work of others, but also to implement your own ideas and to contribute to our open-source firmware.

In the years that Bitcraze has been around, there has been quite the build-up of information, which can be either found on the main website, the wiki, the github repositories, and in bits & pieces on the forum. Although we try to provide all the information necessary for getting started with development, it is currently quite a clutter. If we at Bitcraze already have difficulty of finding and maintaining all the documentation, we can only imagine how difficult it would be for a starting developer. We therefore would like to improve the flow of information dramatically!

Here are some ideas of what we would like to do with the documentation:

Moving product information to the shop.

LED ring expansion deck in the main website, shop, and wiki.

Currently there are three different locations where you can find information about physical Crazyflie, localization systems or expansion decks, which is the main website, the online store and the wiki. We see that a lot of electronic and hardware shops usually put all the details of the product directly on the product page of their shops. We aim to do that as well, since there will only be one page for users to go to for schematics, specifications, instructions and more, and for us it will be also easier to maintain and update any product information

Moving Software Info to GitHub

There is a lot of bits and pieces of information to handle the firmware on the Crazyflie and all the tools in the tutorials on our main website, wiki and Github repository. This again makes a lot of duplicate information, which is difficult for us to maintain and therefore gets easily outdated. We could put all the information on the wiki, but what if somebody changes something in the code which requires a change in the procedure as well?

It would be the best to keep all the information about the firmware as close to the source as possible, therefore we think is best to move everything to the github repositories. For instance, the wiki cfclient instructions can be moved to the documentation of the cfclient repostitory, or Onchip debugging instructions can go to crazyflie firmware repostitory To keep it all manageable we will:

  • Create a /doc folder on the repositories to better structure all the information
  • Add more Doxygen comments to all the function in the the codes and automatically generate documentation for this.

Restructuring the Wiki Content

After moving all the hardware- related content to the shop, and moving all the firmwar- related info to the Github repositories, we will need to think about what we want to do with the Wiki! You would think that there is nothing left to put on the wiki anymore after the replacement of the earlier documentation, but we beg to differ! For instance, there is so many Github repositories that there is a really a need for an overview. The wiki can educate developers on which tools we have an how to properly use them. Of course, we already have the getting started tutorials, but we want to also provide a more in-depth explanation of the overall structure and how the different repositories would need tho work together, like this .

This does mean that we would need to restructure the wiki entirely and only focusing on topics like:

  • System Architecture Crazyflie
  • Communication protocol between STM and the NRF
  • Communication protocol between the Python library and the Client
  • Overview Github-repositories
  • Projects and hacks
  • etc etc

What do you think?!

Of course we can change all we want in the documentation, but you guys are the ones who actually use it! We are very curious of what what you think of the plans and give us more tips or suggestions on how to improve the overall documentation experience. Please leave a message below or express your opinion on this forum thread.

Summer 2019 Wrap Up

  2019-07-01  |     |    Leave a comment

Summer is here and temperatures are rising. Since many of us will be on holidays, we will focus this quarter on a special summer clean up! See here what we are working on:

  • Fixing issues: This time we are aiming to close many of the issue tickets in our Github repositories, so that after the summer everything will run much more smoothly (we hope!). Definitely our test rig will come in very hand to sniff out more issues in terms of radio communication as well. You can help as well! Everybody who is using and developing on with the STM-firmware, NRF-firmware or python library, or anything else and is noticing issues, please make a ticket in that same Github repository (if you are familiar with the code) or post about it on our forum (if you do not know exactly what is going on). Together we can make the code better.
  • Lighthouse calibration: In March we released our lighthouse deck for positioning with the HTC Vive base stations. We did feel that the setup process could be improved further, since currently, the Crazyflies’ firmware must contain hardcoded information of the Steam VR’s base station position. We will try to apply the factory calibration direct from the Base stations itself. This will enable us to do 2 additional things: (1) The Crazyflie with the LH deck itself could be used to setup the Lighthouse system, so that SteamVR would not be necessary anymore. (2) Only 1 base station is needed for positioning instead of 2, which will improve the robustness in case of loss of visual-line-of-sight of one base station.
  • Documentation: We try to provide all the possible information for everybody to be able do anything they want with their Crazyflie. But with high flexibility comes great responsibility…. for proper documentation! We are planning to restructure all of our media outlets and try to improve the flow and level of detail for our users. We hope to make it easier for beginning developers to get started and more advanced developers to gain better understanding of the system in order to implement their own awesome ideas. So our very first step is to restructure and clean up the Bitcraze wiki and see where we can add more content.
  • Products: We have a lot of products coming out in the 2nd half of the year
    • AI Deck: We are working hard to get the AI deck all ready for production and we are estimating that they will be available for early access in late autumn. Keep a look out on our forum for regular updates on the progress!
    • Lighthouse breakout board: We made our first working prototype of the lighthouse breakout board, which should make it easier for the lighthouse positioning system to also work on other platforms than the Crazyflie.
    • Active Marker Deck: We are very much on on track with the Qualisys active marker deck! It should be available in the Autumn.
    • Crazyflie Bolt: This has been send off to production for the early access version, which should be available in the Autumn!

Introducing our new team member: Kimberly

  2019-05-20  |     |    Leave a comment

At ICRA 2019

Hi, very nice to meet you all!  My name is Kimberly McGuire and I would like to introduce myself as the newest member of the Bitcraze team. Although I probably have never written anything bigger than a paragraph about myself before, but I will try my best to get you to know me a little better!

The first thing you should know about me is that I’m Dutch. I grew up in the Netherlands, which is a small little country that has fought for eternity against the sea and still managed to exist. If you do visit and would appreciate some impressive pieces of Civil Engineering,  don’t linger around in Amsterdam but  visit the “Delta werken” in the province of Zeeland or the ‘Afsluitdijk” in the north. But if you don’t like to take your chances near our oceans, ‘Utrecht’, ‘Groningen’ and ‘Maastricht are really nice to visit too! Probably my last name already gave it away, but I’m also half-American and I visit California almost once a year.

So, let’s talk about hobbies! I have played tennis for almost all my life (started when I was just 6 years old), and I’ve played competitively in tournaments and team-based competitions.  You can also find me skiing on some mountains at least once per year. In general, I really love visiting other countries and to experience the culture and, most importantly, the food. During my studies I’ve managed to go on extended stays abroad: I’ve spent a half a year in Santa Barbara, 3 months in Liverpool, a half a year in Tokyo. Talking about the latter: I’m a huge fan of pretty much everything related to Japan. All their food, Ghibli studios and other Japanese productions, the culture and people, and I do admit that I’m quite the karaoke addict. Moreover, I love going to the movies, playing videogames, and enjoying Dutch coziness, a.k.a. “gezelligheid”, with friends. 

To show in a slice of my educational background: for my Bachelor studies at the Delft University of Technology, I started out at Industrial Design Engineering. Here I learned everything there is to know about product development, from analyzing market-trends, creating concepts to finally to the full production process. For my Masters I was leaning more towards the technical side. After a bridging course to Mechanical engineering I went to do an M.Sc. in BioMecanical design with a specialization in Bio-Robotics (currently known  cognitive robotics). I ended developing a cerebellum-based stabilization system for a binocular robotic head (google “jaguar chicken commercial”:) ). After this, I came across a project at the MAVlab where I was allowed to work with tiny MAVs called pocket-drones, and the first thing I thought was ‘awesome!!’, however, that was before I realized that working with tiny MAVs was not a piece of cake.  I’m glad that the inner-geek in me kept me excited for 4 years straight, which is a crucial element or any PhD I believe, or any job!

So now a new chapter will start for me in Sweden. I hope that I will append to Bitcraze’s vision and I’m quite sure that they will teach me how to become a better hardware and software developer. I’m excited to start working with Arnaud, Marcus, Kristoffer and Tobias and develop more things that will make any’s inner-geek jump a little from excitement. At the moment of writing, we will be on ICRA in Montreal, Canada, so jump by booth 101 to say hi!