Documenting the new Lighthouse Functionality

2021-04-19 | Kimberly McGuire | 2 Comments

A few weeks ago we released version 2021-03 including the python library, Cfclient and the firmware. The biggest feature of that release was that we (finally) got the lighthouse positioning system out of early access and added it as an official product to the Crazyflie eco system! Of course we are very excited about that milestone, but the work does not end there… We also need to communicate how to use it, features and where to find all this new information to you – our favourite users!

New Landing Page

First of all, we made a new landing page for only the lighthouse system (similar to bitcraze.io/start) we now also have bitcraze.io/lighthouse. This landing page is what will be printed on the Lighthouse base station box that will be available soon in our store, but is also directly accessible from the front page under ‘Product News’.

This landing page has all kinds of handy links which directs the user to the getting started tutorial, the shop page or to its place within the different positioning systems we offer/support. It is meant to give a very generic first overview of the system without being overloaded right off the bat and we hope that the information funnel will be more smooth with this landing page.

New tutorial and product pages

For getting started with the lighthouse positioning system, we heavily advise everybody to follow the new getting started tutorial page, even if you have used the lighthouse system since it’s early access days. The thing is is that the procedure of setting the system up has changed drastically. The calibration data and geometry are now stored in persistent memory on-board the Crazyflie and the lighthouse deck itself is now properly flashed. So if you are still using custom config.mk, hardcode geometry in the app layer or use get_bs_geometry.py to get the geometry… stop what you are doing and update the crazyflie firmware, install the newest Cfclient, and follow the tutorial!

We also already made some product page for the Lighthouse Swarm bundle. Currently it is still noted as coming soon but you can already sign up to get a notification when it is out, which we hope to have ready in about 1-2 month(s). The lighthouse deck was of course already available for those that can not wait and want to buy a SteamVR base station somewhere else. Just keep in mind that, even though the v1 is supported, in the future we will mostly focus on the version 2 of the base stations.

Video tutorial

Once again we have ventured into the land of videos and recorded a “Getting started with the Lighthouse positioning system” tutorial for those who prefer video over text.

Feedback

We love feedback and want to improve! Please don’t hesitate to contact us on contact@bitcraze.io if you have comments or suggestions!

Successful emulation

2021-04-12 | Max | Leave a comment

Approximately two month ago we wrote a blog post presenting our planned master thesis. Time flies and we have now reached a sufficient state where the results are presentable and possible to use. We have used the Renode framework and created a platform for the Crazyflie 2.1. In Bitcraze’s Github repository there now is a Renode fork with a custom Renode-infrastructure submodule. To get Renode up and running on your computer check the README found there.

On the Renode branch crazyflie there are two new REPL (Renode platform) files describing the platform. An example of the syntax is given below.

// I am a comment peripheralname: Namespace.ClassName @ parent 0x08000000 numericConstructorField: 0x100000 stringConstructorField: "template" Interrupt -> interrupthandler@3

In the cf2.repl all the external peripherals are connected while the stm32f405.repl contains the STM32F405 peripherals. Note that only the peripherals used by the current Crazyflie 2.1 have been added since they are the only ones that can be tested using the Crazyflie firmware.

When running Renode a RESC (Renode script) file is loaded. There are currently two RESC files for the Crazyflie, one that only loads the Crazyflie plattform and one that can be used for testing. The one for testing automatically starts the simulation and it also has a hook to exit Renode once the self test has passed.

As mentioned, Renode is usable both for automatically and interactively testing firmware. The current version of automatic testing is based on the firmware passing the self test. The plan is to incorporate this in the CI pipeline.

When used interactively it is possible to pause the emulation whenever the user wants to, either manually via a Renode command or by connecting to GDB. This allows reading (as well as writing to) memory addresses. Want to read the DMA status at a specific line of code or mess with the system by randomly flipping bits? Doable in the emulator without risking your Crazyflie crashing.

In the platform there also are our customized sensors to which data can be loaded. The data can be loaded either manually or via a file and then sent to the STM32. The scope of this master thesis however has been on firmware testing, not getting a simulated Crazyflie to fly in a virtual environment.

Emulation of hardware is not a trivial task, there are still improvements to be made and the everlasting question whether the emulation actually represents the real system.

One of the Crazyflie features that had to be simplified was the syslink and connection to the nRF microcontroller, which in the emulation simply sends messages back to the STM32. The most exciting part about it currently is how the STM32 receives a signal that no expansion decks are connected via the 1-wire when a scan is executed. Further improvements would be to emulate the radio, power management and support expansion decks, either via an external program or through Renode.

Of course there are other things to improve as well, there will always be someone who thinks of better ways to implement features and only time will tell how this emulator is going to evolve.

Josefine & Max

Transitioning back to Windows Development

2021-04-05 | Kimberly McGuire | Leave a comment

When I was started my Robotics Master back in 2012, I remembered how frustrated I was at the time to setup my development environment in Windows for the C++ beginners course. My memory is a bit fuzzy of course but I remembered it took me days to get all the right drivers, g++ libraries right, and to setup all in the path environmental in Eclipse at the time. Once I started working on Ubuntu for my Master thesis, forced to due to ROS, I was hooked and swore I will never go back to Windows for robotics development again… until now!

I always used Windows on my personal machine on the side for gaming and have a dual-boot on the work computer for some occasional video editing, but especially I had begun to learn game development for Fun Fridays, I started to be drawn to the windows side of the dual boot more and more. But if I needed to try something out on the Crazyflie or needed to debug a problem on the forum, I needed to restart the computer to switch operating systems and that was starting to become a pain! Slowly but steadily I tried out several aspects of the crazyflie ecosystem for development on Windows 10 and actually…. it is not so traumatic as it was almost 10 years ago.

Python Library and Client

It went quite smooth when I first try to install python on windows again. Adding it to the PATH environment variable is still very important but luckily the new install manager provides that as an option. Moreover, Visual Studio Code also provides the possibility to switch between python environments so that you try different versions of python, but for now version 3.8 was plenty for me.

With the newest versions of the Windows install of Python, pip is by definition already installed, but I experienced that it would still be necessary to upgrade pip by typing the following in either a Command Prompt or (my favorite) Powershell:

python -m pip install --upgrade pip

After this, install the cflib from release was quite an ease (‘pip install cflib’) but even installing it from source with Git configured on Windows was no problem at all and very similar to a native Ubuntu install.

Until recently the cfclient was a bit more challenging to install through pip from due the SDL2 windows library had to be downloaded separately, so the only options would have been installing from source or the .exe application release. The later has not been updated since the 2020.09 release due to building errors. Luckily, with the latest release, this has now been fixed as a SLD2 python library was found. Now the cfclient can be installed with a simple ‘pip install cfclient’.

Firmware Building with WSL

The firmware development was the next thing that I tried to get up and running, which managed to be slightly trickier. About a year ago I tried to get Cygwin to work on Windows, but my bad memories of the past came back due to the clunkiness of it all and I abandoned it again. Also there are some reported issues with the out-of-tree build (aka the App layer). Some colleagues at Bitcraze already mentioned the Windows Subsystem for Linux (WSL) but I never really looked at it until the need came to move back to Windows for development. And I must say, I wish I had tried it out a while ago.

With some repositories downloaded already on my Windows system with Git, I installed Ubuntu 20.04 WSL, got the appropriate gcc libraries and accessed the crazyflie-firmware by ‘cd /mnt/c/my/repos‘. Building the firmware with ‘make all’ went pretty okay… although it took about a minute which is a little long compared to the 20 secs on the native Ubuntu install. The big problem was that I could not use Docker and the handy bitcraze toolbelt due to the WSL version still being 1. These functionalities were only available for version 2 so I went ahead and upgraded the WSL and linked it to docker desktop. But after upgrading, building the firmware from that same repository on the C:/ drive took insanely long (almost 10 minutes). So I switched back the WSL ubuntu 20.04 to version 1, installed a second WSL (this time Ubuntu 18.04), updated that one to WSL2 and used solely for docker and toolbelt purposes. Not ideal quite yet… but luckily with visual studio code it is very easy to switch the WSL .

But there is more though! Recently I timed how long it took to build the crazyflie-firmware with ‘make all -j8’ from both WSL version in a repository that is on the C:/ drive on Windows (accessible by /mnt/c from the WLS), or from a repository on the local file system:

WSL 2 (ubuntu 18.04)
- C:/ = 11m06s
- WSL local = 00m19s
WSL 1 (ubuntu 20.04)
- C:/ = 01m04s
- WSL local = 00m59s

This is done on an Windows laptop with an i7-6700HQ with 32 Gb RAM. The differences with WLS2 between build firmware on the windows file system or the local WSL file system is huge! So that means that the right way is to have WSL2 with the repo on the WSL file system, which is similar to build time as a native install of Ubuntu.

Flashing the Crazyflie

The main issue still with WSL is that it does not allow USB access… So even if the crazyradio driver is installed on windows with Zadig, you will not be able to see if you type ‘lsusb‘ in WSL for both version 1 and 2. So when I still had the repository on the C:/ drive and build the crazyflie-firmware from there I could flash the Crazyflie through the Cfclient or Cflib (with cfloader) through Powershell, but building it from the local subsystem, which is way faster for WSL2, would require to first copy the cf2.bin file to my C drive before doing that.

Another option, although still in Alpha phase, is to use the experimental Crazyradio server for WSL made by Arnaud, for which the user instructions can be found in an issue thread only for now. The important thing is that the Zadig installed driver has to be switched to WinUSB and switched back again to LibUSB if you want to use the Cfclient on windows. It would still needs some work to improve the user experience but gives promise of better integration of WSL development for the Crazyflie.

To Conclude

Soon I’m planning to soon reinstall the Windows part on the dual boot laptop but there are already some things that I will integrate on my freshly installed Windows based on what I experienced so far:

Keep using Python on windows and install the Cfclient and Cflib by pip only.
Only use Ubunu 20.04 as WSL2 and install the Crazyflie-firmware on the WSL local file system.
Use Visual Studio Code as the editor for both C:/ based and WSL based repos.
Use the Crazyradio server or copy the bin file to C:/ whenever I need to flash the crazyflie with development firmware.

For any AIdeck development, I would still need to use the native Ubuntu part or the bitcraze-VM since there is not a USB access or server yet for the programmer. However, if Windows would support USB devices and a graphical interface for WSL, that will make all our Windows-based Crazyflie development dreams come true!

Release 2021.03: Lighthouse is stable!

2021-03-29 | Arnaud | 4 Comments

We are happy to anounce the availability of the 2021.03 release of the Crazyflie firmware and client! This release includes new binaries for the Crazyflie (2021.03), the Crazyflie client 2021.03 as well as the Crazyflie python library 0.1.13.2. The firmware package can be downloaded from the Crazyflie release repository (2021.03) or can be flashed directly using the client bootloader window. The firmware package contains the STM firmware (2021.03), the NRF firmware (2021.03) and the Lighthouse deck FPGA binary (V6).

The main feature in this release is the stabilization of the Lighthouse positioning system. The main work done has been on the system setup and management, it has taken a lot of work spawning all the projects and a brunch of documentation, but we think we have reached a stage where the lighthouse positioning system is working very well and is very easy to setup and get working. We have now published the new Lighthouse getting started guide and will be working this week at updating all required materials to mark Lighthouse as released!

When the Lighthouse positioning system was released in early access, it required to install SteamVR, run some custom scripts and flash a modified firmware to get up and running. This has been improved slightly over time with scripts that allows to setup the system without using SteamVR and some way to store the required system data in the Crazyflie configuration memory rather than hard-coded in the firmware. With this release, everything is coming together and it is now possible to go from zero to an autonomous Crazyflie flying in a lighthouse system in minutes by just using the Crazyflie client.

Lighthouse positioning setup and management tab in the Crazyflie client

Another major improvement made to support the lighthouse is the modification of the bootloader Crazyflie update sequence in the client as well as in command line. The new sequence will restart the Crazyflie a couple of times while upgrading the Crazyflie, this allows for an upgrade of the firmware in the installed decks if required. The lighthouse deck firmware has been added to the Crazyflie .zip release file and will be flashed into the deck while flashing the release to a Crazyflie that has the deck installed.

An alternative, robust TDoA implementation has been added for the Loco Positioning System. This change has been contributed by williamwenda on Github and can optionally be enabled at runtime.

An event subsystem has also been added to the firmware. It allows to log events onto the SD-Card which can be very useful when acquiring positioning data from the various positioning system supported by the Crazyflie. We have described this subsystem in an earlier blog post.

There has also been a lot of smaller improvement and bugfixes in this release. See the individual project releases not for more information.

We hope you are going to enjoy this new Crazyflie and lighthouse release. Do not hesitate to drop a comment here, questions on the forum if you have any or bug reports of github in the (very unlikely ;-) event that there are bugs left.

AI-deck workshop by PULP lab

2021-03-22 | Kimberly McGuire | Leave a comment

About a month ago we released the AIdeck 1.1, which has some slight upgrades and changes compared to the 1.0. Even though the AIdeck 1.1 is still in early access, we do see the number of support questions increase on our forum and in the issue list of the AIdeck example repo. Therefore we are planning to host an AI-deck getting-started workshop by the PULP lab on the 16th of April at 14:00 (Central European Summer time)!

PULP

The PULP lab has done many amazing research on the field of Edge ML and were one of the collaborators in the development of the AI-deck of which their work on the Pulp Shield was the main inspiration. For more information check out their guest blogpost and be sure to read their latest work on the AI-deck!

More over, they have been working on an opensource tools that also work on the GAP8chip which are must try-outs for any AI-deck users

DORY: Deployment ORiented to memorY (Github, Paper)
NEMO: NEural Minimizer for pytOrch (Github, Paper)
PULP dronet (Github, Paper)

All in all, since they clearly know what they are talking about, they are more than qualified to teach the rest of us how to work with all this! Also check out Luca Benini’s keynote at RISC-V or this week at the TinyML summit if you would like to learn more about PULP!

Date and content Workshop

The workshop will be tailored to those that have just started to to work with the AI-deck however, we think it will be interesting for regular users as well. Note that the tools mentioned above will not be handled this time.

These are the topics that will be discussed:

Hardware explanation (Gap8 specifics and AIdeck)
Software Preliminaries (GapSDK,, VM)
Hands-on examples

The workshop will take approximately 2 hours and will be on 16th of April in the afternoon, but the exact specifics will be given at a later date. So make sure to already block it in your calendar and to sign up for more information!

Sign up for more information

You can sign up to receive more information by giving your email address in this google form. We will also keep you up-to-date on our discord channel and the event page.

Event-Based Data Logging

2021-03-15 | Wolfgang Hoenig | Leave a comment

One crucial aspect of any research and development is to record and analyze data, which then can be used for quantifying performance, debugging strange behavior, or guide us in our decision making. We have been trying to improve the way that this is done to help all of you researcher out there with their work, so this blog post will explain an alternative, better, method to record whatever is happening on the Crazyflie in real-time.

Example data collection of received Lighthouse angles over time. The y-axis contains a unique measurement ID (16 in total for 4 sensors * 2 basestations * 2 sweeps/basestation). Thus, each dot represents the time when a certain kind of measurement was received. We do not receive all angles in fixed intervals, because of the interference between basestations.

Existing Logging Approaches

So far there have been two principle ways of recording data with the Crazyflie:

Logging: In the firmware, one can define global variables that can be streamed out at a fixed frequency, using logging configurations. Variables have a name and data type, and the list of all available variables can be queried from the firmware.
Debug Prints: It is possible to add DEBUG_PRINT(…) in any place that contains a string and possibly some variables. These are asynchronous, but not timestamped, so they are mostly useful to notify the user of some status change.

Both approaches have multiple backends. The logging variables can be streamed over USB, via the Crazyradio, or on a Micro-SD-card using a specialized deck. When using the radio or USB, the frequency is limited to 100 Hz and only a few variables can be streamed at this speed due to bandwidth limitations. The Micro-SD-card deck, on the other hand, allows to log a lot more logging variables at speeds up to 1 kHz, making it possible to access high-speed sensor information, e.g., the IMU data. The Debug Prints can be used over the radio or USB, or with a SEGGER J-Link and a Debug Adapter, over J-Link Real-Time Transfer (RTT). The former is very bandwidth limited, while the latter requires a physical wire connection, making it impossible to use while flying.

New Approach: Event-Based Logging

Event-based logging combines the advantages of the existing approaches and offers a third alternative to record data. Similar to debug prints, a user can trigger an event anywhere in the code and include some mandatory variables that describe this event (so-called payload). Similar to logging, these events are timestamped and have fewer bandwidth limitations. Currently, there is just one backend for the Micro-SD-card deck, but it would be possible to add support for radio and USB as well.

One of our first test cases of the event-based logging is to analyze the data we get from the Lighthouse deck. Here, we trigger an event whenever we received a raw sweep. This allows us to visually see the interference that happens between two LH2 basestations. We also use the event-based logging for time synchronization of the Crazyflie and a motion capture system: when we enable the IR LEDs on the active marker deck, we record an event that contains the Crazyflie timestamp. On the PC side, we record a PC timestamp when the motion capture system first detects the IR LEDs. Clock drifts can be computed by using the same logging mechanism when the IR LEDs are turned off.

Adding event-based logging had some other good side effects as well: the logging is now generally much faster, there is more user feedback about the correct buffer size usage, and the binary files are smaller. More details are in the documentation.

Future Work

We are working on adding event triggers to the state estimator. This will enable us to record all the sensor information during real flights with different positioning systems, so that we can tune and improve the Kalman state estimator. It will also be interesting to add support for events in CRTP, so that it can be used over USB and radio.

A new addition to the team: Jonas Danielsson

2021-03-08 | Jonas Danielsson | Leave a comment

Hi all! I am Jonas Danielsson and I started at Bitcraze on the first of March. I am a software developer with experience of Linux-, open source- and embedded development. Sometimes all at once.

I have worked with embedded systems of different sizes since 2007 and look forward to getting to know the Crazyflie and all of its ecosystem.

I am interested in and care a lot about software- and product maintenance. At Bitcraze I hope to be involved both in developing new features, fixing bugs and finding ways of keeping the product and code in nice shape. To find ways of working that allow us to add a bunch of cool stuff as fast as we want to, without endangering the cool stuff we already have.

Also, I am interested in getting to know the Crazyflie community and to work together with you to create the best development experience we can achieve. Do not be afraid to reach out with ideas or suggestions on how we can improve!

AI-deck 1.1 now in stock

2021-03-01 | Tobias | 4 Comments

It did take much longer than anticipated, but we finally managed to get the AI-deck 1.1 back in stock! We had some difficulties with the post-production testing and Chinese new year was also a main contributor to the delay, but we are now very happy that we have a batch of AI-deck’s which are ready to ship!

There are only minor changes between the AI-deck 1.0 and 1.1 which can be read about in the “AI-deck product update” blog post. Since the AI-deck 1.1 is now using the gray-scale version of the camera module and the AI-deck 1.0 was using the Bayer RGB version, we now also offer the camera modules as stand-alone products for those that rather have the other variant.

For those of you that already have the previous AI-deck 1.0 and rather wished for the gray-scale camera module, please send us an email at contact@bitcraze.io, and we will work something out!

The AI-deck 1.1 is still a early access product, so remember to post any question or problems you have on our forum in the AIdeck group or to check out examples / documentation on the Github Repo. We also are planning to organize an online workshop / tutorial for the AI-deck in the coming months. We will hopefully have more information about that soon, so keep an eye out on the blogposts!

Communication Reliability

2021-02-22 | Wolfgang Hoenig | Leave a comment

Communicating with your Crazyflie is an important pillar for its operation. As more robots are controlled, the reliability of this communication link becomes more and more important, as the probability that there is no failure at any of the Crazyflies decreases exponentially with the number of robots. We have written about the low-level radio link before. Today, we focus on past, on-going, and future improvements to make the communication more reliable.

Reliability Challenges

As part of doing research with the Crazyswarm, I noticed several issues:

Sometimes commands do not seem to arrive at the Crazyflie, especially when using unicast (i.e., direct) communication with a specific Crazyflie.
Sending ‘too much’ data while the Crazyflie is flying can cause catastrophic crashes. One example is trying to upload a trajectory, while flying in a motion capture space. However, this occasionally even happens if just trying to update a parameter.

The radio link has a feature called Safelink, which essentially guarantees that packets are send and received in order and no packet gets dropped. This feature never worked reliably in the crazyflie_cpp implementation and is therefore not used in the Crazyswarm. However, it is the default mode for cflib. The Crazy RealTime Protocol (CRTP) also has the notion of different communication ports in order to prioritize important messages such as control commands over less important ones such as trajectory upload. However, this prioritization was never implemented in any of the clients.

Native Link Implementations

Another, non reliability related, issue always was that the performance of cflib is not stellar when connected to many Crazyflies. This is mostly because (C)Python has a Global Interpreter Lock (GIL), which prevents true multi-threaded operation. A common solution to allow true parallelism is multiprocessing or implementations in a language that compiles to machine code (native code).

Crazyflie-link-Cpp

The first native implementation is written in C++ and includes Python bindings using pybind11. The overall API is simple: A connection can be created given a URI, and data can be send and received. Internally, this library implements Safelink to guarantee packets are in order, uses priority queues to prioritize messages on important CRTP ports, and handles all multi-radio and multi-thread related synchronization issues. Unlike prior implementations, the bandwidth is shared uniformly between all connections, i.e., there is no race between different Crazyflies communicating over the same radio. We measured a 50% lower CPU utilization when connecting with the CF client, a 20% higher bandwidth, and a 20% lower latency compared to the pure Python implementation.

For now, this feature is experimental and needs to be enabled using an environment variable. If you want to give it a try, you can use

pip install cflinkcpp
USE_CFLINK=cpp cfclient

once everything is merged to master in the next couple of days.

Crazyflie-link-rs

The second native implementation is written in Rust. As it turns out, writing multi-threaded code in C++ is very difficult and error-prone, because it is up to the user to use the various synchronization primitives correctly. Rust, on the other hand, provides extensive compile-time checks to increase reliability by design. This implementation does not have quite as many features as the C++ version yet and requires a bit more work to be used with the client but it is going to be worked-on on Fridays. The goal is to reach functional parity with the other link implementations and be able to use the rust-implemented link with “USE_CFLINK=rs cfclient” in the future.

Future Work

As part of the development process, we also wrote many system tests that verify correctness and measure performance of the various link implementations. We already found and fixed several firmware bugs (681 and 688). The major open issue is that there is no flow control on the system link (the connection between the NRF51 and the STM32), which still causes packet loss in some cases.

Hardware emulation thesis

2021-02-15 | Max | Leave a comment

Hi!

Since the middle of January, Bitcraze has had two additional guests: us! We, Josefine & Max, are currently doing our master thesis at Bitcraze during our final semester at LTH.

Unfortunately, the pandemic means remote working which could have caused some difficulties with hardware and equipment accessibility. Fortunately, for us, our goal with the thesis is to emulate the Crazyflie 2.1 hardware in the open source software Renode. We can therefore do the work at home.

Since this is the first time either of us have tried to emulate hardware it is exciting to see if it will even be possible, especially as we do not know what limitations Renode might have. Thus far, four weeks in, there have been some hiccups and crashes but also progress and success. One example was when we got the USART up and running and it became possible to start printing debug messages and another was when the LED lights were connected and it was possible to see when they were turned on and off.

Example of output from partially implemented hardware.

If all goes well, the emulated hardware could be used as a part of the CI pipeline to automatically hunt for bugs. Academically, it would be used to further study testing methods of control firmware.

Getting a glimpse into the workings at Bitcraze and the lovely people working here has been most interesting and we are looking forward to the time ahead. Until next time, hopefully with a working emulation, Josefine & Max.