The IROS 2022 demo

2022-11-07 | Kristoffer Richardsson | Leave a comment

IROS in Kyoto is over and all Bitcrazers are finally back in Sweden again. We had a really good time in Japan and enjoyed all the interesting discussions we had with all of you, thanks!

In this blog post we will describe the demo we were running in the both and talk a bit about all the cool tech that was used. If you want to reproduce it at home or just take a look for inspiration, the code is available on github in the iros-2022 branch of our experimental firmware repo. There is also a page on our web for IROS 2022 with some more information.

The demo has similarities with our previous demo (see IROS 2019) but has been upgraded to be a fully autonomous and decentralized swarm with 9 Crazyflies buzzing around in a cage, going back to charging pads for wireless charging when the battery is running out. The demo supports multiple Crazyflies flying at the same time, avoiding collisions without a central authority, all decision making is done in each Crazyflie, that is fully decentralized.

The hardware is off-the-shelf products available in our store (links here). The software is obviously written specifically for the demo, but we wanted to use the building blocks already available in the system so the demo code is mainly “glue” to connect them together.

The cage/flying space

The flying space was box shaped, 3×2 meters in foot print and 2.5 meters high. We enclosed it in our lightweight travel cage made from aluminium pipes and a light net. It is a pretty small space to fly multiple Crazyflies in at the same time but it worked! The main problem with such a small space is down-wash from other Crazyflies and having enough room to avoid collisions. 3 Crazyflies worked pretty well, but had the space been larger it would have been possible to fly all nine.

Localization

Localization was handled by the Lighthouse positioning system. We used two base stations and the lighthouse deck on each Crazyflie which provides the Crazyflies with their current position with high accuracy.

Since the position is computed in the Crazyflie, using only data from on-board sensors, no external communication is needed in relation to the localization system. The only exception was that we uploaded the physical geometry of the system when setting up the cage.

Path planing

When a Crazyflie is flying in the demo, the standard mode of operation is to fly a randomized pattern of straight lines. From time to time (randomized) the Crazyflie can also chose to fly the spiral that we have used in earlier demos (see the IROS 2019 demo for instance).

When the battery is running out, the Crazyflie goes back to the charging pad for charging. The position is sampled before taking off and this coordinate is used as the landing point to find the charging pad. When landed the Crazyflie verifies that the battery is being charged. If the battery is not charging the Crazyflie assumes it missed the charging pad and it takes off again to adjust the position.

Charging

The Crazyflies were equiped with the Qi-charging deck for wireless charging. The charging pads are 3D-printed pads with a slope to make the Crazyflie slide into position also if the landing is not perfect. In the center of the pads there are standard Qi-chargers from IKEA mounted to provide power.

To fly continuously, the system charging rate must be higher than what is consumed by the flying Crazyflies. With a system of nine Crazyflies that are charging through Qi-chargers it is possible to keep one Crazyflie flying, just. To get some margin we increased the charging speed a bit, the down side being that the Crazyflies get warm and the batteries ware out faster.

Collision avoidance

We use the built in collision avoidance system contributed by James Alan Preiss at University of Southern California. Thanks James, it works like a charm!

There is no planing ahead, but each Crazyflie must know where the other Crazyflies are located. Based on this information they avoid each other and chose a new path to reach their target position. For this to work each Crazyflie is continuously broadcasting its position to the other Crazyflies using the peer-to-peer framework.

Swarm control and collaboration

As mentioned earlier there is no central authority that decides which Crazyflie that should take off or go to a specific position, instead this functionality is handled in each Crazyflie. To make it possible for each Crazyflie to have a rough idea of the system state, each Crazyflie is broadcasting its position and state (landed, flying etc) to the other Crazyflies. If a Crazyflie realizes that too few drones are flying, it will simply take off to fix the problem, if it sees that too many are flying it will go back to the charging pad. To avoid that all Crazyflies takes off or lands at the same time, a randomized hold-back time is used before the actions is executed. This does not fully prevent two individuals from taking off at the same time, but makes it less likely, and eventually the correct number of drones will fly.

The number of drones that should fly at the same time is a system wide parameter that can be set from one of the peers in the system. To make sure they all agree on the value, a simple mechanism is used based on the age of the data. The value and the age of the value is included in the broadcast data. When another Crazyflie receives the data it compares the age of the received data with the age of the data it already has and replaces it only if it is younger.

Sniffer

A tenth Crazyflie is used in the demo as a sniffer. It is essentially a non-flying member of the swarm that listens to the broadcast traffic and it is used to feed data to a GUI that displays the state of the system. It can also be used to inject a new value for the desired number of flying Crazyflies.

Implementation and how to run it

The code is mainly implemented as an app in the Crazyflie firmware, using the app layer. The main part is a state machine that keeps track of what to do next with some other modules handling communication and trajectories.

The code is available in the iros-2022 branch of the crazyflie-firmware-experimental repository, in the examples/demos/decentralized_swarm folder.

The examples/demos/decentralized_swarm/src/common_files/choose_app.h file controls if the code is compiled for a swarm member or the sniffer.

All Crazyflies should have the same radio channel and the same address, except the last byte. Swarm members must use addresses ending in 01 to 09 while the sniffer must use the address ending in 00.

The demo is based on the work that Marios did for a decentralized swarm this summer. Thanks Marios!

Bitcraze is hiring

2022-10-31 | Barbara | Leave a comment

Since last March, some members of the Bitcraze team went away. Jonas left, and Arnaud took his parental leave. We welcomed an intern, and eventually Arnaud came back but our 6 person team feels a little short.

At Bitcraze, it always seems that there are not enough hours in a day to do everything we want to do. It’s even truer now, when the day-to-day maintenance is longer and it feels like we don’t have the time to dive into our new projects – even though we still manage a lot of new things, like the motor upgrade kit or a conference in Japan.

But if there is not enough man-hour per hour, one solution is simple: hire a new person ! Since this summer, a job post has been up in our webpage. We are officially actively looking for a new teammate !

So, who are we looking for exactly ?

Well, we’re looking for someone that is open, honest, and passionate about technology. That last part is easiest to determine: you must be interested in everything technological, and more specifically embedded systems. At the moment, we’re mostly looking for someone that is interested in the manufacture part of the process, but also in embedded software. However, if you’re a developer with some Python knowledge, and a passion about hardware, you’re might be a right fit as well! Generally speaking, someone with a lots of different interests that would fill the holes we have right now.

And that’s why, frankly, writing a job post is not an easy feat here. In Bitcraze, no one is the manager or “boss”, and we all decide together on what to work on. No one has a definite part, and we talk more easily of passions than fixed role . So, we don’t know exactly now what profile would be the best for us, as it’s more about what you’re willing to do than what you can accomplish. Right now, we’re in need of someone with hardware passion; but maybe we also need someone with a totally different interest that we don’t know about – because we ourselves are not interested in it and didn’t think about it.

The way we work is something that asks for a specific mindset, and we want to make sure that every prospect has the same mentality. Being open-minded, and ready to take on a lot of different tasks and roles, is a very important part of the daily life at Bitcraze, and something we’re searching for in every candidate we meet. And that covers the open and honest part !

In Japan, we came across a shrine dedicated to matchmaking. It is said that the goddess Kuchiire Inari Okami ties together people – whether for marriage or job hires. We made a small offering to her – you never know what can help.

The Kuchiire Inari Okami in Fushimi Inari

Of course, a more effective way to help us would be to send us your CV at jobs@bitcraze.io and tell us who you are !

By the way, some of us are still in Japan (while the others are recovering from jetlag) so the company might still be running a little slowly this week.

We’re in Japan !

2022-10-24 | Barbara | Leave a comment

I already talked about it here and there, but this day finally came: the whole company is in Japan !
Kimberly travelled first, to account for jetlag, meet with some people, and attend ROScon.

It was last week, and she got the opportunity to learn a lot, meet people from the ROS community, and give an exciting talk.

Kimberly’s talk at RosCon (made by Ramón Roche)

Happy to be in Japan (Made by Ramón Roche)

The rest of the company travelled last week with all the equipment needed divided into our suitcases.

*Our suitcases at the office, to gather the materials before going*

We chose to rent a traditional machiya while there, where we can all stay together and enjoy the life in the center of Kyoto.

*Us chilling out in the Bitcraze mansion*

Our first day here was to account for jetlag, but we managed to sightsee the amazing sites of Kyoto – and enjoy the most praised Japanese food, much appreciated after a long walk among the Tori gates of the Fushimi Inari shrine.

*Us after climbing on top of Mt Inari – with the beautiful path of Tori gates*

But it was soon time to start working, and yesterday we worked really hard on setting up everything to have a nice demo at IROS.

After some head scratching, emergency taping and hacking we managed to get the autonomous demo that Marios implemented last summer flying – just before the event hall We got time to explore the Kyoto International Conference Center, a beautiful venue with a Japanese garden and a futuristic look – as imagined in the 70′.

*Some views from the Kyoto Conference Center*

We invited those of you that are attending IROS to come and see us for a tech meet-up. It’s today and it would be a real nice opportunity for us to finally chat in person with our users ! Since there are a lot of aerial systems talks, we realize it may be difficult to come during the sessions, so the tech meet-up can begin during the break, at 15.40

Next up this week is the safe nanocopter competition. Kimberly will actually deliver the prize for that, we can’t wait to see what this competition will show – and how fun it is to remote-control the Crazyflies that are in the University of Toronto Institute for Aerospace Studies!

Of course, we will share some news on social media – and we will have a blogpost in a few weeks to debrief on the whole trip.

As you’ll understand, maintaining the day-to-day of the company is a little trickier this week, but we still monitor email, github discussions, and are shipping orders. You should just expect a longer time to process those, as we’re too busy – either at the booth or… at karaoke ! (no, there will be no videos of us singing).

Crazyswarm2 development

2022-10-10 | Kimberly McGuire | Leave a comment

As you probably noticed already, this summer I experimented with ROS2 and connecting the Crazyflie with multi-ranger to several mapping and navigation nodes (see this and this blogpost). First I started with an experimental repo on my personal Github account called crazyflie_ros2_experimental, where I managed to do some mapping and navigation already. In August we started porting most of this functionality to the crazyswarm2 project, so that is what this blogpost is mostly about.

Crazyswarm goes ROS2

Most of you are already familiar with Crazyswarm for ROS1, which is a project that Wolfgang Hönig and James Preiss have maintained since its creation in 2017 at the University of Southern California. Since then, many have used and referred to this work, since the paper has been cited more than 260 times. From all the Crazyflie papers of the latest ICRA and IROS conferences, 50 % of the papers have used Crazyswarm as their communication middleware. If you haven’t heard about Crazyswarm yet, please check-out the nice BAMdays talk Wolfgang gave last year.

Unfortunately, ROS1 will not be there forever and will be phased out anno 2025 and will not be supported for Ubuntu 22.04 and up. Therefore, Wolfgang, now at the Intelligent Multi-robot Coordination Lab at TU Berlin, has already started with the ROS2 port of Crazyswarm, namely Crazyswarm2. Here the same principle of the C++ based Crazyflie server and the python wrapper were been implemented, along with the simple position based simulation and Teleop nodes. Mind that the name Crazyswarm2 is just the project name out of historic reasons, but the package itself can also be used for individual Crazyflies as well. That is why the package names will be called crazyflie_*

Porting the Summer Hack project to Crazyswarm2

The crazyflie_ros2_experimental was fun to hack around, as it was (as the name suggests) experimental and I didn’t need to worry about releases, bugfixes etc. However, the problem of developing only here, is that the further you go the more work it becomes to make it more official. That is when Wolfgang and I sat down and started talking about porting what I’ve done in the summer into Crazyswarm2. This is also a good opportunity to get more involved with the project, especially with so many Crazyfliers using the ROS as well.

The first step was to write a second crazyflie_server node that relied on the python CFlib. This means that many of the variables I used to hardcode in the experimental node, needed to be defined within the parameter structure of ROS2. The crazyflies.yaml is where anything relevant for the server (like the URIs and parameters) needs to be defined. Both the C++ backend server and the CFlib backend server are using the same parameters. Also the functionality of the both servers are pretty similar, except for that logging is only possible on the CFlib version and uploading/follow trajectories is only possible on the C++ version. An overview will be provided soon on the Crazyswarm2 documentation website.

The second step was to make the crazyflie_server (cflib) node suitable to be connected to external packages that I’ve worked with during the hack project. Therefore, there are some special logging modes, that enables the server to not only output topics based on logging, but Pose/Odometry/LaserScan messages along with Transforms. This allowed the SLAM_toolbox to use the data from the Crazyflie itself to create a map, which you can see an example of in this tutorial.

Moreover, for the navigation it was important that incoming Twist messages either from keyboard or from a navigation toolkit were handled properly. Most of these packages assume a 2D non-holonomic robot, but a quadcopter like the Crazyflie needs to first take off, stay in the air and land. Therefore in the examples, a separate node (vel_mux.py) was written to receive incoming Twist messages, first have the Crazyflie take off in high level commander, and keep sending hover commands to keep it in the air until a land service is called.

What’s next?

As you probably noticed, the project is still under development, but at least it is now at a good state that we feel comfortable to presented at the upcoming ROScon :) We also want to include an more official simulation package, especially now that the Crazyflie has recently became part of the official release of Webots 2022b, but we are currently waiting on the webots_ros2 to be released in the ubuntu packages. Moreover, the idea is to provide multiple simulation backends that based on the requirement of the topic (swarms, vision-based etc), the user can select the simulation most useful for their situation. Also, we would like to even out the missing items (trajectory handling, logging) in both the cflib and cpp backend of the crazyflie_server so that they can be used interchangeably. Also, I saw that the experimental simple mapper node has been featured on social media, so perhaps we should be converting that to Crazyswarm2 as well :)

So once we got the most of the above mentioned issues out the way, that will be the time that we can start discussing the official release of a ROS2 Crazyflie package with its source code residing in the Crazyswarm2 repository. In the meantime, it would be awesome that anybody that is interested in ROS2, or want to soon upgrade their Crazyswarm(1) packages to ROS2 to give the package a whirl. The more people that are trying it out and report bugs/proposing fixes, the more stable it becomes and closer it will come to an official release! Please join us and start any discussions on the Crazyswarm2 project github repository.

Crazyswarm2 documentation: https://imrclab.github.io/crazyswarm2/
Crazyswarm2 source code: https://github.com/IMRCLab/crazyswarm2
ROScon 2022 talk: ROS 2 and the Crazyflie: Aerial swarms and Autonomy with a tiny flying robot

Bitcraze in Japan

2022-10-03 | Barbara | Leave a comment

This fall is full of exciting events for us, and none are more excitedly expected than our visit to Japan. Yes, the whole company (6 people) are travelling to Kyoto for at least a week – but not for sightseeing (well, not only). Here is what we have planned:

ROSCon

As per tradition, ROSCon is held shortly before IROS. So, on the 19 to 21 October, Kimberly will be here to represent us along the ROS community. She will even have a presentation about the latest ROS2 integrations in collaboration with the maintainers of Crazyswarm2. It’s on October 21st, 16.50 local time so if you’re there make sure to hear her talk !

IROS

From the 21st to the 27th of October, IROS will be held at the Kyoto International Conference Center. it’s one of largest robotics conference worldwide, with almost 1750 papers presented. As the first in-person session of the conference since the beginning of the Covid pandemic, we had to be there. We will man the booth during the whole conference, with the demo our intern Marios has worked on a lot. And since it’s been a long time since we’ve been able to gather and talk together, we thought it would be great to have an official meetup at IROS for those interested.

So, please note this official invitation to Bitcraze’s tech meetup at IROS! If you’re at IROS and want to meet us together with other Crazyflie users, then let’s get together on Monday 24th of October at 16.00 at our booth 59. It’s the perfect occasion to (re)connect, to get the latest news about Bitcraze, to talk about development, share what you’ve been doing and even possibly hack together! Be sure to say hi if you’re there. We will try to make it something similar to a Swedish fika, with some sweets and coffee, but we can’t promise that there will be kanelbullar.

IROS Safe Robot Learning Competition

And this year, we’re happy to announce that there will be a Crazyflie competition during IROS. The goal is to develop safe learning-based algorithms that can cope with uncertainties not known at design time. Our friends at Dynamic Systems Lab are organizing this competition with two simulated phase, and one experimental phase at IROS… And the experiment is a remote access to the Flight Arena at the University of Toronto Institute for Aerospace Studies in Toronto, Canada via high-speed internet connections. You don’t need to be present at IROS to participate, but if you wish to do so, beware, the registration for the competition ends on October 12th. We’re really curious and excited to see what this competition is going to show!

What about Bitcraze during that week?

But, if everybody is in Japan, what about Bitcraze’s regular activities ? You may be wondering. Well, no worries. Even though we’re going to be half a world away, the business is going to follow us. Of course, some of us are going to take that opportunity to take some vacations and visit this beautiful country, so during IROS’ week and the week after, the company will run a little bit more slowly than usual. We won’t be as reactive as usual on emails and discussions, but we will still monitor our emails and ship some orders.

Are you planning to visit IROS or ROSCon ? Is there anything in particular in the schedule that you don’t want to miss ? Don’t hesitate to tell us if you want to join the meetup !

IMAV 2022: NanoQuadcopter challenge and more!

2022-09-26 | Kimberly McGuire | 1 Comment

Last week we went on a nice trip to Delft, The Netherlands to attend the 22th International Mico Aerial Vehicle Conference and Competition, this time organized by the MAVlab of the TU Delft. Me (Kim), Barbara and Kristoffer went there by train for our CO2 policy, although the Dutch train strikes did made it a bit difficult for us! Luckily we made it all in one piece and we had a great time, so we will tell you about our experiences… with a lot of videos!

First Conference day

For the conference days we were placed in the main aula building, so that everybody could drop by during the coffee breaks, right next to one of our collaborators, Matěj Karásek from Flapper Drones (also see this blog post)! In the big lecture hall paper talks were going on, along with interesting keynote speeches by Yiannis Aloimonos from University of Maryland and Antonio Franchi from TU Twente.

In between the talks and coffee breaks, we took the opportunity to hack around with tiny demos, for which the IMAV competition is a quite a good opportunity. Here you see a video of 4 Crazyflies flying around a Flapperdrone, all platforms are using the lighthouse positioning system.

The Nanoquadcopter challenge

The evening of the first day the first competition was planned, namely the nanoquadcopter challenge! In this challenge the goal was to autonomously fly a Crazyflie with an AIdeck and Flowdeck as far as possible through an obstacle field. 8 teams participated, and although most did offboard processing of the AIdeck’s camera streaming, the PULP team (first place) and Equipe Skyrats (3rd place) did all the processing onboard. The most exciting run was by brave CVAR-UPM team that managed to do pass through 4 gates while avoiding obstacles, for which they won a Special Achievement Award.

During the challenge, Barbara also gave a presentation about the Crazyflie while Kristoffer build up the lighthouse positioning system in the background in a record breaking 5 minutes to show a little demo. After the challenge, there were bites and drinks where we can talk with all the teams participating.

Here there is an overview video of the competition. Also there was an excellent stream during the event if you would like to see all the runs in detail + presentations by the teams, you’ll have have a full 3 yours of content, complemented by exciting commentary of Christophe de Wagter and Guido de Croon from the MAVlab. Thanks to all the teams for participating and giving such a nice show :)

The overview video

The full stream of the nanoquadcopter challenge

The Green House Challenge

On Wednesday, we were brought to Tomato world, which is a special green house for technology development in horticulture. Here is where the Greenhouse challenge, which was the 2nd indoor drone competition took place. The teams had to participate with their drone of choice to navigate through rows of tomato plants and find the sick variant. Unfortunately we could not be up close and personal as with the nanoquadcopter challenge, but yet again there was a great streaming service available so we were able to follow every step of the way, along with some great presentations by Flapper drones and PATS! drones among others. For the later we were actual challenged to an autonomous drone fight! Their PATS-x system is made and detect pest insects that are harmful for green house crops, so they wanted to see if they can catch a Crazyflie. You can see in the video here that they manage to do that, and although the Crazyflie lived, we are pretty sure that a real fly or moth wouldn’t. Luckily it was a friendly match so we all had fun!

PATS versus Crazyflie battle

Here is an overview of the Green house challenge. At the end you can also see a special demo by the PULP team successfully trying out their obstacle avoiding Crazyflie in between the tomato plants. Very impressive!

Last days and final notes

Due to the planned (but later cancelled) train strikes in the Netherlands, the full pack were not able to attend the full event unfortunately. In the end Barbara and I were able to experience the outdoor challenge, where much bigger drones had to carry packages into a large field outdoors. I myself was able to catch the first part of the last conference day, which included a keynote of Richard Bomprey (Royal Veterinary College), whose lab contributed to the mosquito-inspired Crazyflie flight paper published in Science 2 years ago.

We were happy to be at the IMAV this year, which marks as our first conference attendance as Bitcraze after the pandemic. It was quite amazing to see the teams trying to overcome the challenges of these competition, especially with the nanoquadcopter challenge. We would like to thank again Guido de Croon and Christophe de Wagter of the MAVlab for inviting us!

IMAV website: https://2022.imavs.org/

Crazyflie IMAV papers:

‘Handling Pitch Variations for Visual Perception in MAVs: Synthetic Augmentation and State Fusion’ Cereda et al. (2022) [pdf]
‘Seeing with sound; surface detection and avoidance by sensing self-generated noise‘, Wilshin et al. (2022)

Towards Autonomous Flight on Nano-Drones with Multi-zone Depth Sensors

2022-09-12 | Hanna Müller | Leave a comment

This weeks guest blog post is from Hanna Müller, Vlad Niculescu and Tommaso Polonelli, who are working with Luca Benini at the Integrated Systems Lab and Michele Magno at the Center for Project-Based Learning, both at ETH Zürich. Enjoy!

This blog post will give you some insight into our current work towards autonomous flight on nano-drones using a miniaturized multi-zone depth sensor. Here we will mainly talk about obstacle avoidance, as it is our first building block towards fully autonomous navigation. Who knows, maybe in the future, we will have the honor to write another blog post about localization and mapping ;)

A Crazyflie 2.1 with our custom multi-zone ToF deck, a flow deck and a vicon marker.

Obstacle avoidance on nano-drones is challenging, as the restricted payload limits on-board sensors and computational power. Most approaches, therefore, use lightweight and ultra-low-power monocular cameras (as the AI-deck) or 1d depth sensors (as the multi-ranger deck). However, both those approaches have drawbacks – the camera images need extensive processing, usually even neural networks to detect obstacles. Neural networks additionally need training data and are prone to fail in completely new scenarios. The 1d depth sensors can reliably detect obstacles in their field of view (FoV); however, no information about the size or exact position of the obstacle is obtained.

On bigger drones, usually lidars or radars are used, but unfortunately, due to the limited weight and power consumption, those cannot be carried and used on nano-drones. However, in 2021 STMicroelectronics introduced a new multi-zone Time-of-Flight (ToF) sensor – with maximal 8×8 pixel resolution, a range up to 4m (according to the datasheet), a small form-factor and low power consumption of only 286mW (typical) it is ideal to use on nano-drones.

In the picture on top, you can see the Crazyflie 2.1 with our custom ToF deck (open-sourced at https://github.com/ETH-PBL/Matrix_ToF_Drones). We described this deck for the first time in [1], together with a sensor characterization. From this, we saw that we could use the sensor in different light conditions and on different colored obstacles, but from 2m on, the measurements started to get incomplete in all scenarios. However, as the sensor can detect invalid measurements (due to interference or obstacles being out of range), we can still rely on our information. In [2], we presented the system and some steps towards obstacle avoidance in a demo abstract, as you can see in the video below:

The next thing we did was to collect a dataset – we flew with different combinations of decks (flow-deck v2, AI-deck, our custom multi-zone ToF deck) and sometimes even tracked by a vicon system. Those recordings amount to an extensive dataset with depth images, RGB images, internal state estimation and the position and attitude ground truth.

We then fed the recorded data into a python simulation to develop an obstacle avoidance algorithm. We focused on only the ToF data (we are not fusing with the camera in this project, we just provide the data for future work). We aimed for a very efficient solution – because we want it to run on-board, on the STM32F405, with low latency and without occupying too many resources. Our algorithm is very lightweight but highly effective – we divide the FoV in different zones, according to how dangerous obstacles in those areas are and then use a decision tree to decide on a steering angle and velocity.

With only using up 0.31% of the computational power and 210 μs latency, we reached our goal of developing an efficient obstacle avoidance algorithm. Our system is also low-power, the power to lift the additional sensor with all accompanying electronics as well as the supply of it totals in less than 10% of the whole drone. On average, our system reaches a flight time of around 7 minutes. We refer to our preprint [3] for details on our various tests – they include flights with distances up to 212 m and 100% reliability and high agility at a low speed in an office environment.

As our paper is currently submitted but not yet accepted our code and dataset are not yet released – however, the hardware design is already accessible: https://github.com/ETH-PBL/Matrix_ToF_Drones

[1] V. Niculescu, H. Müller, I. Ostovar, T. Polonelli, M. Magno and L. Benini, “Towards a Multi-Pixel Time-of-Flight Indoor Navigation System for Nano-Drone Applications,” 2022 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), 2022, pp. 1-6, doi: 10.1109/I2MTC48687.2022.9806701.
[2] I. Ostovar, V. Niculescu, H. Müller, T. Polonelli, M. Magno and L. Benini, “Demo Abstract: Towards Reliable Obstacle Avoidance for Nano-UAVs,” 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), 2022, pp. 501-502, doi: 10.1109/IPSN54338.2022.00051.
[3] H.Müller, V. Niculescu, T. Polonelli, M. Magno and L. Benini “Robust and Efficient Depth-based Obstacle Avoidance for Autonomous Miniaturized UAVs”, submitted to IEEE, preprint: https://arxiv.org/abs/2208.12624

Using the AI deck for CRTP over WiFi (via CPX)

2022-09-05 | Marcus | 7 Comments

Before the summer vacations, I had the opportunity to spend some time working on AI deck improvements (blog post). One of the goals I set was to get CRTP over WiFi working, and try to fix issues along the way. The idea was to put together a small example where you could fly the Crazyflie using the keyboard and see the streamed image along the way. This would require both CRTP to the Crazyflie (logging and commands) as well as CPX to the GAP8 for the images. Just before heading off to vacation I managed to get the demo working, this post is about the results and som of the things that changed.

Link drivers

When using the Crazyflie Python library you connect to a Crazyflie using a URI. The first part of the URI (i.e radio or usb) selects what link driver to use for the connection. For example radio://0/80/2M/E7E7E7E7E7 selects the radio link driver, USB dongle 0 and communication at 2Mbit on channel E7E7E7E7E7.

While working on this demo there were two major things changed in the link drivers. The first one was the implementation of the serial link (serial://) which is now using CPX for CRTP to the Crazyflie. The usecase for this link driver is to connect a Raspberry Pi via a serial port to the Crazyflie on a larger platform.

The second change was to add a new link driver for connecting to the Crazyflie via TCP. Using this link driver it’s possible to connect to the Crazyflie via the network. It’s also possible to get the underlying protocol, the CPX object, for using CPX directly. This is used for communicating with for example the GAP8 to get images.

In the new TCP link driver the URI starts with tcp:// and has either an IP or a host name, followed by the port. Here’s two examples:

tcp://aideck-AABBCCDD.local:5000
tcp://196.168.0.100:5000

Comparison with the Crazyradio PA

So can WiFi be used now instead of the Crazyradio PA? Well, it depends. Using WiFi will give you larger throughput but you will trade this for latency. In our tests the latency is both larger and very random. In the demo I fly with the Flow V2 deck, which means latency isn’t that much of an issue. But if you were to fly without positioning and just use a joystick, this would not work out.

The Demo!

Below is a video of some flying at our office, to try it out yourself have a look at the example code here. Although the demo was mostly intended for improving CPX, we’ve made use of it at the office to collect training data for the AI deck.

The Crazyflie with AIdeck during over WiFI controlled flight.

Improvements

Unfortunately I was a bit short on time and the changes for mDNS discovery never made it it. Because of this there’s no way to “scan” or discover AI decks, so to connect you will need to know the IP or the host name. For now you can retrieve that by connecting to your AI-deck equipped Crazyflie with the CFclient and look at the console tab.

A part from that there’s more improvements to be made, with a better structure for using CPX (more like the CRTP stack with functions) in the library and more examples. There’s also still a few bugs to iron out, for example there’s still the improved FPS and WiFi throughput issues.

IMAV 2022

Next week from 13th to 16th of September Barbara, Kristoffer and Kimberly will be present at the international Micro Aerial Vehicle Conference and Competition (IMAV) hosted by the MAVlab of the TU Delft in the Netherlands. One of the competitions is called the nano quadcopter challenge, where teams will program a Crazyflie + AI deck combo to navigate through an obstacle field, so we are excited to see what solutions will come out of that. If any of you happens to be at the conference/competition, drop by our table to say hello!

Our Intern Marios’ Summer work on Swarms

2022-08-29 | Marios Stamatopoulos | Leave a comment

Hello world,

you had probably seen me from the last blog post when I first arrived. I spent this summer working here in Malmö and I can definitely say that it was one great, educative and fun experience. During the last three months I have been in Bitcraze, I was given the chance to work and develop applications and demos on the robotics subject I am most interested in, drone collaboration.

Centralized Swarm with Multiple Flying Copter

I initially started looking into the implemented swarm demo which had 7 drones charging wirelessly in 7 charging decks and one of them flying by executing a spiral trajectory until it has a low battery and another one takes its place. The original swarm demo was shown at several conferences before the pandemic hit, but my plan was to improve it by adding more quadcopters flying simultaneously. The biggest problem was the collisions and ground effect happening between them. In order to solve that I was based on this paper and the optimization engine OpEn. I solved the problem of all drones starting from a point and going to a final one without colliding and covering the minimum distance by transferring these constraints into a cost function of an optimization problem assuming a simplified model for the quadcopter. Its output is waypoints for each quadcopter to pass from. These waypoints are transformed into a trajectory(piecewise polynomial) by a custom trajectory generator based on linear algebra.

In this way, I made it possible to execute non-colliding trajectories for 4 quadcopters, upload and execute them. While executing the first trajectory, the next one was being calculated and uploaded assuming the goal of the previous one as starting point. In this way, I managed to have 4 Crazyflies flying simultaneously and landing when their battery was running out and the fully charged ones were taking their place. This mechanism with some modifications can be used as a path planner or a standalone trajectory generator from a future user by feeding it waypoints and time durations for each waypoint segment. You can find the source code here.

Decentralized Swarm with Multiple Flying Copters

The aforementioned setup seemed to work pretty well but there was always the need of having a central PC monitoring and taking decisions for the whole swarm. So we wanted to move the architecture to a decentralized one, of which Kristoffer did some preliminary work shown at BAMdays last year. This was made possible by utilizing the onboard peer-to-peer protocol (P2P) in collaboration with the onboard collision avoidance algorithm introduced in this PR contributed by James Preiss from the University of Southern California. All the Crazyflies share their position and state through the protocol by broadcasting them at a rate of 15 Hz.

Although there were some missed packets, they could avoid each other while flying by updating the collision avoidance algorithm which is taking action between the high level and the action commander by altering its waypoints. The decisions of which drone should take off or land are also taken in a decentralized way. Whenever one copter is about to take off it enters the corresponding state and assigns itself a randomized timeout. During this timeout, if the desired number of airborne copters is achieved it goes back to idle. If not and the timeout occurs it finally take-offs. So, despite there is not an actual common decision, the swarm can be led to simple desired states like keeping the number of the drones flying constant and executing changes between the landed, charged copters and flying ones. You can find the source code here.

Token Ring Implementation

After I finished this project and since I had some extra time left I decided to work more in the P2P protocol. The need for having a robust way to communicate between the Crazyflies and a way to verify that a packet was indeed sent was obvious. A solution to this problem was offered by Christos Zosimidis and Klaus Kefferpütz from the Cooperative Control Lab in Hochschule Augsburg, namely a token ring implementation. I would really like to thank them for this collaboration and hope for future ones as well.

Specifically, the proposed token ring protocol was implemented in a modified version of the nrf-radio firmware and the Crazyradio. This protocol assumes that each Crazyflie is a node of a network and a token is passed around giving permission to each drone that has it in its possession to broadcast data. So, each time only one Crazyflie broadcasts data which leads to fewer packet collisions and losses. It can also acknowledge that a node has received the data targeting to it and then continues to others. The interface with the protocol is being done by 2 queues (TX and RX) where the user can place data that wants to send and read the RX queue to receive. The moment that this blog post is being written only the static version of it is public in the firmware, which means that the number and the id of the Crazyflies must be defined before execution and in case a copter fails the whole network fails. Although, I am currently working on a dynamic approach that is going to solve these problems

All in all, I had a great time here in Malmo despite the fact that the Swedish summer is much colder than the warm weather I was used to in Greece. I was amazed by the way things in Bitcraze work and how the whole company operates. It was a pleasure being around so creative people and I am happy that I could help even in a small way. Thank you very much for giving me the opportunity to work with you and I hope I will keep on contributing to this amazing project in the future.

Happy hacking and fly safe!

And we are back!

2022-08-22 | Tobias | Leave a comment

After a period of bitcrazer-vacations, we are now all back at work. The summer here in Sweden has generally been great. Some of us stayed here to keep the company afloat, and some just stayed afloat on lakes or the sea. The majority vacationed inside of Sweden, but some (could you guess who?) have visited France, Italy, or Greece. We’ve been lucky with a mostly warm and sunny weather, perfect for bathing and grilling. And even though it’s nice to enjoy real summer, it’s still worrying sign though, as Europe is experiencing what could be the worst drought in 500 years.

Crazyflie 2.1 back in stock

What is also back is the Crazyflie 2.1, but back in stock, yay! After almost two weeks without any drones available for sale, we received a new batch of our quadcopter today. It should now be available in the shop, just in time for when school starts!

We got some indications the component shortage are slowly moving in the right direction so hopefully it will get easier to keep things in stock in the future. We are keeping our fingers crossed.

Bolt 1.1 ESC cable red/black switched

Unfortunately we recently found out that there has been a manufacturing error with the ESC cables that come with the Bolt 1.1. The black and red cables have been switched. Please see the image below.

With the black and red cables switched this will result in powering your ESCs with reversed polarity. This will most likely burn the MOSFET on the Bolt that controls the power to the ESC, which is the weakest link. This because the MOSFET body diodes on the ESC will conduct and make the whole ESC a short circuit. In many setups, e.g using 4in1 ESC these cables are not used though and will not cause a problem.

Switching the cables back is quite easy to do. Use a needle, tweezer or e.g. small screwdriver to open the plastic lock so the cable can be pulled out. Switch the black and red and you are done. You can double check that the colors are correct by comparing it with the Bolt 1.1 board. The plus and minus should match with the red and black as per the image below:

We are currently working with the manufacturer to get correct cables. If you got a Bolt 1.1 (anytime between June and August 2022) we can of course ship you correct cables once they are ready or give you support if you got problems with the control board. If so, please send us an email to support@bitcraze.io. Sorry for this inconvenience!