App layer and P2P communication

2019-10-21 | Arnaud | 4 Comments

Over time the scope of Crazyflie has changed a lot. At first, Crazyflie was “just flying” with the only possible control was attitude (roll, pitch, yaw) and thrust setpoint sent from the Radio. Soon after, autonomous flight was investigated, first by implementing position controller outside Crazyflie and then, over time, moving position control on-board and sending position or trajectory setpoint to the Crazyflie. Now that the Crazyflie has good position control, the next step is to implement autonomous behavior and until now the most practical way is to do this from code running in an external computer. Similarly to what happened in the history of position control (first off-board and now onboard), it needs to be possible implement autonomous behavior in the Crazyflie itself. This blog post is about two newly implemented capabilities that will allow to implement automation in the Crazyflie firmware in an easy and maintainable way, namely the ‘App layer’ and the P2P communication.

App layer

The “App layer” is a term we have been using internally in Bitcraze to describe a set of functionalities that would allow to implement code in the Crazyflie. This includes the infrastructure to compile and maintain external code running in the Crazyflie as well as a set of API to control flight and behavior from C code rather than from radio communication.

Last week we implemented the first step of the App layer: the infrastructure part. It is now possible to build the Crazyflie firmware out-of-tree. This means that it is now possible, from a project, to point to the Crazyflie firmware folder and to compile a firmware from the project folder without touching the Crazyflie firmware folder. Practically it allows to create a git-repos implementing custom firmware code that has Crazyflie firmware as a sub-repos. This makes the maintenance of custom firmware code much easier than maintaining a branch of the Crazflie firmware as previously required.

A second piece that has been implemented is the app entry-point. It allows to start running code by just creating an “appMain()” function. The function will be called from a dedicated FreeRTOS task after the Crazyflie has initialized and started. This should make it much easier to get started.

For an example, we have extracted the multiranger push demo into a standalone git repos. This demonstrate the implementation of autonomous behavior using these new infrastructures.

Peer to Peer communication

The Crazyflie has been used for many research related to swarming, some examples are the crazyswarm project or the work done by Carnegie Mellon University. However, it is now time to turn it up a notch. On the forum and on the Github repository, there has been several request of enabling direct peer to peer communication to the Crazyflies. Now we finally found time to work on it and implement some basic functionality on the NRF and STM side of the firmware.

Currently, it is possible to send and receive a P2P packet in broadcast mode from the STM directly (see how to do this in the documentation). This enables data to be send from one Crazyflie to another with a maximum data size of 60 bytes. We were able to stress-test this with our test rig, by sending broadcast messages in a round-robin-kind of fashion, where the broadcast message was transferred through 10 Crazyflies in 10-20 ms. Even-though the current implementation is for now very minimal, we were able to fix some existing issues in the radiolink framework.

We will not stop there, as we are hoping to implement a communication system similar to how the CTRP protocol has been implemented. We are getting a lot of help by our active community members, so check out this github issue to be up-to-date with the current discussion.

Switching Bitcraze servers to Kubernetes

2019-09-09 | Arnaud | Leave a comment

Working at Bitcraze means designing electronics as well as writing software for it, but there is also a lot of other things going-on and managing servers is one of them.

Originally, in 2011 we had a virtual server in the cloud (a VPS like it was called back then). We carefully setup and configured this server and we maintained it over the years. This was easy and served us well but it was also very ‘manual’: any updates where done on the production machine directly and we where doing all changes directly in the production machine.

In 2015, when Kristoffer joined Bitcraze, he revamped completly the server setup. Suddenly we had not only one server but two, one production and one staging. And all servers running on these machines where running in docker containers. This means that the servers where nothing more than a well configured machine to run docker, and all important software and configuration are in docker images and are launched using docker-compose. This is the architecture we are still using today, a great description was written by Kristoffer in a previous blog post.

The docker architecture has worked well but we still have one major problem, we still have servers to take care of. They are still configured manually, needs to be kept up to date and happy at all time. We have been wanting to get rid of these servers and “just” run our containers.

Now enter Kubernetes. Kubernetes is a container Orchestrator originally made by Google that abstracts the servers and allows to run containers independently of handling servers. We had heard of Kubernetes back when we started working with Docker but deemed it overkill: we did not want to setup and handle a cluster of server by ourselves and back then it was not clear that how long Kubernetes would be around. Over time though Kubernetes has had support by all major cloud providers and there is managed offers that would allow us to not handle any servers ourselves.

We are not there yet, but we have been working over the summer at converting our infrastructure to Kubernetes and we are very close to deployment. One of the main parts is our internal Jenkins build server that currently build and deploy services by ssh-in into servers. With Kubernetes this deployment phase will be much simpler and will allow us to update the website without downtime, this is a welcome functionality now that the documentation is moved from the wiki to the main website.

With lightweight Kubernetes distributions like K3S there is also an opportunity to run containers in more domains. For instance we have been talking about making automated hardware system test-bench for a while to test every commit against the real hardware. With K3S and a couple of raspberry-pi that could be achieved quite easily. This is a subject for future fun Fridays though … :-).

Synchronous swarm example

2019-07-08 | Arnaud | Leave a comment

Two weeks ago we posted about the demo we did for our new office move-in party. There has been multiple requests to share the script but unfortunately this is a hacked old script that is not going to be useful at all as an example. So, last week, we made an example that could run a synchronized swarm sequence.

The example has been pushed in the example folder of the Crazyflie-lib-python project. It is called synchronizedSequence.py. Running this example unmodified with 3 Crazyflies in a positioning system will give you this result. (Like the previous demo, this was done in a lighthouse system.)

One of the key design of the example is that it is based on a single control loop that can be synchronized with an outside system: in this example, there is a simple sleep of one seconds between each step of the sequence but it could for example be changed into a midi clock receiver to synchronize the sequence with music.

The example was developed with the help of Victor, a student we have hired to help-out during the summer. He has then played around a little bit to make a 9 Crazyflies sequence that is more impressive:

I uploaded Victor’s sequence in a github gist as it can be good for inspiration. One bit of warning though: as is, the sequence contains some vertical movements that are quite aggressive and the part where Crazyflies fly directly on top of each-other is more to be considered as a stress test.

Playing with Crazyflie 2.1 choreography with Lighthouse positioning

2019-06-24 | Arnaud | 11 Comments

We have recently moved to a new bigger office. With the summer arriving in Sweden, it was time to organize a small move-in after-work party with friends and family. For the occasion we wanted to play around with a small swarm of Crazyflies and the new Lighthouse positioning. Time being a sparse resource, we setup the ICRA2019 demo in the flight lab so that we would be able to fly during the party. We also started looking at our old swarm show that we ran with the LPS a year ago to see if we could run it with Lighthouse:

The show was a essentially a sequence of setpoints sent from a python script and controlling 9 Crazyflies 2.1 equipped with Lighthouse deck on the top and led-ring deck on the bottom synchronized on music. We setup the Crazyflie in the Lighthouse positioning system and converted the script to use the high-level-commander GOTO setpoints. We look forward at trying more advance control problems like trajectories to make more impressive synchronized flight choreography in the future but for now it already look quite good even with only GOTOs:

Making a radio test-bench for Crazyflie2

2019-06-03 | Arnaud | 2 Comments

While running our ICRA demo, we came across a bug in the Crazyflie python-lib radio handling, limiting the number of Crazyflie that could be controlled using one Crazyradio PA. Communication with many Crazyflies is crucial as flying swarms is becoming more of an interesting topic for research and education. So we decided to take the problem at hand and create a radio test-bench:

To make the test-bench we have attached 10 roadrunner boards to a plank of wood together with USB switches that can provide enough power to the roadrunners. We used the roadrunner because it is mechanically easier to use in this context and it has an identical architecture to the Crazyflie 2.1 when it comes to the radio implementation.

Initially we will use the test-bench to run test scripts that pushes the communication to its limits and that consistently test the communication stack functionalities. This should allow us to find bug and verify that we solve them as well as discovering and documenting limitations.

Eventually we want to connect a raspberry-pi to the test-bench and run tests for each commit and pull-request to the crazyflie-firmware, crazyflie2-nrf-firmware and crazyflie-lib-python projects. This will guarantee that we do not introduce new limitations in the communication stack. The test-bench will also be very useful in implementing new functionalities like direct crazyflie-to-crazyflie P2P communication.

As a final note, the Crazyswarm project is not affected by the Crazyflie-lib bug since it is using the C++ implemented crazyflie-ros driver. Hence Crazyswarm can control more Crazyflies per Crazyradio PA, so it is still the preferred way to fly a swarm mostly when using a motion capture system. Though, with the progress made on LPS and Lighthouse positioning, running swarms, using the python API directly is a probably a more lightweight alternative.

Back from ICRA 2019

2019-05-27 | Arnaud | Leave a comment

3 of us where at ICRA 2019 in Montreal last week, where we met a lot of interesting people and a lot of Crazyflie users. Thanks a lot to everyone that drop by our booth, and for the ones that missed it we are planning on being at iROS2019 later this year so we might see you there :-).

We have already described our demo in a previous post, now that we run it we can update on how it went. We are also updating the ICRA2019 page with the latest source code and information so that anyone interested can reproduce the demo.

In its final state at the conference, the demo contained 8 Crazyflies 2.1 equiped with Lighthouse deck and Qi charger deck. There were 8 3D-printed charging pads on the floor with Ikea Qi wireless chargers and two HTC Vive base stations (V1) on tripods. The full system was contained in a cage, built from 50 cm-long tubes or aluminium and nets.

The full setup of the booth took us about 4 hours, this included about 3 hours for the cage, 15 min for the demo including calibration of the lighthouse base-station geometry and the rest to fine-tune things. This is by far our best setup time, we still need to prettify the cage a bit and to make is easier to install, but we will most likely re-use this system for upcoming conferences.

In this demo we aimed at keeping a Crazyflie in the air at every moment, to do so we had a computer connected to all 8 Crazyflies sending to one of them the signal to start flying if no other where actually in the air flying a trajectory. The flight was completly autonomous as we explained in our previous blog post. We setup the Crazyflie to fly 2 cycles and then land, which increase the rate of swap and so increased the ‘action’, though it also meant that during the swap two Crazyflies where flying. This drained the batteries a bit more than expected and meant that after about an hour all the Crazyflies where bellow the take-off threshold and we had to wait ~30 seconds between flights. Here is a video of it in action:

The demo was very care-free, we had very few Crashes and we mostly restarted the Crazyflies to swap batteries manually to add a bit of power in the swarm. The last day we decided to spice it up a little bit by adding a chair in the cage and by calibrating the chair position and flight trajectory, we managed to have the Crazyflie partly fly under it. This worked quite well most of the time and showed that the lighthouse positioning is repeatable and works fairly well with short occlusion in the path. Though we also found out that even though a single Crazyflie would always fly the same trajectory, two different Crazyflies will not. We think differences in propeller stiffness and the fact that the our Mellinger position controller has not been calibrated for changing YAW are the main reasons.

If you want to know more about the demo or if you want to reproduce it do not hesitate to visit the ICRA 2019 page that explains it in more details and links to the source code of everything including 3D printed parts for the cage and the landing pads.

Drawing images with Crazyflie and Bitcraze at ICRA 2019

2019-03-25 | Arnaud | Leave a comment

Last week we posted about painting with the Lighthouse deck. This week we continue on the same track but add a new dimension, all in our “let’s try this crazy idea” spirit. So last Friday, after having a lot of fun painting with the Crazyflie led-ring using long exposure photo and the Lightouse deck for positioning, we had one extra crazy idea. Can we use the Crazyflie to show a raster image, very much like the way a CRT monitor works by sweeping line by line and displaying the pixel color one by one, using the led-ring? Unfortunate we did not have enough time that day…

However the idea was so intriguing that Kristoffer couldn’t stop himself from writing a prototype script during the week-end. So last Monday, just after publishing the blog post, we went to the flight arena and tried it. After a couple of trial and error we found a display algorithm that showed a pretty good result:

The source for this image is this very low resolution Mona Lisa:

It was a very fun experiment, it is magic to see the Crazyflie going back and forth blinking for ~3 minutes, click on the camera and see the resulting picture. It is also a really nice way to observe the current state of the lighthouse positioning. The lines are spaced by about 3 cm and the Crazyflie is controlled using the PID controller. The controller do a decent job of keeping the Crazyflie in lines and the space seems a little bit ’tilted’.

If you are curious or if you want to try by yourself, we pushed the script in the Crazyflie-lib-python example folder.

As a side note, we will be exhibiting at the ICRA 2019 conference May 20-24, 2019 in Montreal, Canada. We will running demo of the LPS and Lighthouse (though I am not sure we can print long exposure picture, this is not so exciting to look in real-time :). We hope you would like to come and meet us there!

Release of the Lighthouse deck early access

2019-03-11 | Arnaud | 3 Comments

We have just released the Crazyflie Lighthouse deck as Early Access! It is now available in our web store.

The lighthouse deck allows the Crazyflie to estimate its position using the HTC Vive tracking base-station normally used for Virtual Reality. The positioning is done by tracking the timing of rotating infra-red laser beams emitted from the base-stations. This system has the advantages of having a very good precision and of allowing the Crazyflie to acquire its position autonomously: once the Crazyflie knows the position and orientation of the base-station, it can calculate its own position without the help of any external systems.

The release as Early Access means that we have finished the hardware and we are confident that the hardware is working properly. Though we have not yet finished all the software and firmware, by releasing the hardware early we can get the hardware into the hands of users quickly to try it out. In return we hope we can get some help making the software better.

Current state

The Crazyflie can calculate its position from the received Vive Base-Station V1 signals.
Direct line of sight should be kept to both base-stations. The Lighthouse deck has 4 receivers so in the future it will be possible to get a position from seeing only one base station.
Base-Station V2 support is still being worked-on, it will only require a software update.
The Base-station position is hard-coded in the Crazyflie and found using SteamVR. Ideally this should be sent from the ground and the Crazyflie should calculate the positions of the Base-Stations automatically.
The previous point means that a full VR system or at least two base stations and a controller or tracker is required to setup the system. In the future we hope to setup the system with only a Crazyflie and two base stations.
Since this version of the deck only has horizontal sensors, it is important that the base-stations are placed above the flight space and the Crazyflies should fly ~40cm bellow the base-stations

As long as the deck is in early access, the main documentation will be the lighthouse positioning page in the wiki. This page is going to be updated a lot in the near future and will track the progress in development.

Demo

We have written a small demo script that allows to set the position of the Crazyflie using a Vive controller. It is a good demo to experiment with the precision of the system and the ability to mix VR and Crazyflie since they are in the same tracking space:

In this demo, a python script connects to two Crazyflies and acquire the controller position using OpenVR and makes the Crazyflies take-off above the controller. Then, when the controller trigger is pushed, the setpoint to the closest Crazyflie is changed to follow the controller movement, the Crazyflies are flying autonomously only getting position setpoints from the python script. The position estimation and control is handled onboard.

We are pretty excited by this release since we think this positioning technology will be very useful for a lot of use-case. Let us know what you think and do not hesitate to contribute if you want to improve the system :).

Lighthouse deck manufactured, now it just needs software!

2019-02-11 | Arnaud | 4 Comments

We are glad to announce that we have manufactured the fist batch of Lightouse positioning decks and hopefully it will be ready to ship by the end of the month!

The Lighthouse positioning deck is a Crazyflie 2 deck capable of receiving IR signals from HTC Vive tracking base station (ie. Lighthouses). The basestations works by spinning IR laser beams that are received by the deck to measure the angle at which the base station sees the receiver. This allows the Crazyflie to estimate its position with great accuracy and so to fly autonomously.

The board we produced is very similar architecture-wise to the prototype we showed in previous blog posts. The main physical difference is that we now only have horizontal receivers. This change was made because we do not yet have a satisfactory mechanical solution to mount vertical IR receivers and we arbitrated that horizontal-only sensor already provides great performance for autonomous flight. Functionally it means that the Crazyflie should fly bellow the base stations to be able to position itself, we found that flying ~40cm bellow the base station gave good flying performance. We will continue looking at solution to make a deck with more receiver to increase the flight space in the future.

The lighthouse deck acquires the IR pulses transmitted by the lighthouses, the Crazyflie can then interpret these pulses to estimate its position. We also added soldering pads for a 2.54mm pin header which would allow to interface other microcontroller boards to the deck:

HTC has released 2 versions of the base stations that are incompatible with each other. Version 1 supports 2 base stations per system, and version 2 can support more than 2. We have good initial support for version 1 both in the deck and in the Crazyflie. Version 2 is currently being worked-on but early work shows that the deck should be compatible with version 2 with only a firmware update.

This leads to the current state of the product. The boards have been manufactured and we have received them but they are currently programmed with a test firmware. As previously stated the basic functionality is there but we still don’t have any finished bootloader. As soon as this is finished and tested we will start flashing all the boards. After that is is just a matter of adding them to the web-store stock and they will be ready to ship!

For now we consider this deck as early access, which means that we will document it in the wiki and that the software will still be heavily developed. For example an early limitation that will be worked-on is that it is currently required to run SteamVR on a computer to setup the system, this means that you need to have a full Vive VR setup or at least a vive gamepad or tracker to setup your flight space. Eventually we want to make it possible to setup the system with only base stations and a Crazyflie, without using steamVR.

We have added the deck to our web store so that you can subscribe to get notified as soon as it is in stock, we will of course post on the blog with more informations when this happens. In the mean time we can share again the video we did for the holidays that was made with 3 Crazyflie 2.1 equipped with the lighthouse deck using 2 V1 base stations:

Crazyflie mobile clients in 2019

2019-01-28 | Arnaud | 1 Comment

While Crazyflie is nowaday mostly used connected to a computer, we have mobile clients that can be used to fly a Crazyflie using either Bluetooth Low Energy or a Crazyradio with an Android device or an iphone.

The Android client is currently the most advanced one with support for some decks. The goal of these mobile clients is to at least allow to fly a Crazyflie manually, though a lot more could be done by supporting the various decks of the Crazyflie (for example using the flow deck, one might imagine drawing a trajectory on the phone and having the Crazyflie following it :-).

As for development, we have not been very active in the development of the mobile clients and are relying mostly on contributions. So if you are interested into adding functionalities do not hesitate to drop by the Github page of the Android or iOS clients and to propose functionalities and pull requests.

Android client

In 2018, Fred the maintainer of the Android client, has worked hard to stabilize the current app and solve the last few bugs and problem in the current app. A new version was released last week that incorporate all the fixes.

Last years the Android client has seen big internal changes including separating all Crazyflie protocol handling in a separate java library. All these changes will make it easier to implement new functionality in the future and to make the functionality available to android as well as, to any Java program using the Crazyflie java lib.

Iphone client

The iPhone client has seen much less activity in 2018. It has been kept updated with the new versionsd of the Swift language and have seen some bugfixes, all thanks to Github contributors.

There have been reports of a couple of pretty bad bugs that have appeared in the latest release, as soon as these bugs are fixed we plan to release a new version of the iPhone client. The new version will also include the possibility to control the Crazyflie by tilting the phone, and with the bug fixes in place we should be off for a good start of 2019.

Windows client

The Windows UAP Crazyflie client is the least advance of all the mobile Clients. It has the particularity to work on Windows 10 for computers as well as for Phones. This makes it the only implementation of a Bluetooth low energy Crazyflie client for computers. However, Windows 10 for phones being pretty much dead now, the future of this client might be more on the Computer side if any.

Anyway, if anyone is interested in improving the Windows client, we will gladly test and merge pull requests when they come.