Introducing our new team member: Kimberly

2019-05-20 | Kimberly McGuire | Leave a comment

Hi, very nice to meet you all! My name is Kimberly McGuire and I would like to introduce myself as the newest member of the Bitcraze team. Although I probably have never written anything bigger than a paragraph about myself before, but I will try my best to get you to know me a little better!

The first thing you should know about me is that I’m Dutch. I grew up in the Netherlands, which is a small little country that has fought for eternity against the sea and still managed to exist. If you do visit and would appreciate some impressive pieces of Civil Engineering, don’t linger around in Amsterdam but visit the “Delta werken” in the province of Zeeland or the ‘Afsluitdijk” in the north. But if you don’t like to take your chances near our oceans, ‘Utrecht’, ‘Groningen’ and ‘Maastricht are really nice to visit too! Probably my last name already gave it away, but I’m also half-American and I visit California almost once a year.

So, let’s talk about hobbies! I have played tennis for almost all my life (started when I was just 6 years old), and I’ve played competitively in tournaments and team-based competitions. You can also find me skiing on some mountains at least once per year. In general, I really love visiting other countries and to experience the culture and, most importantly, the food. During my studies I’ve managed to go on extended stays abroad: I’ve spent a half a year in Santa Barbara, 3 months in Liverpool, a half a year in Tokyo. Talking about the latter: I’m a huge fan of pretty much everything related to Japan. All their food, Ghibli studios and other Japanese productions, the culture and people, and I do admit that I’m quite the karaoke addict. Moreover, I love going to the movies, playing videogames, and enjoying Dutch coziness, a.k.a. “gezelligheid”, with friends.

To show in a slice of my educational background: for my Bachelor studies at the Delft University of Technology, I started out at Industrial Design Engineering. Here I learned everything there is to know about product development, from analyzing market-trends, creating concepts to finally to the full production process. For my Masters I was leaning more towards the technical side. After a bridging course to Mechanical engineering I went to do an M.Sc. in BioMecanical design with a specialization in Bio-Robotics (currently known cognitive robotics). I ended developing a cerebellum-based stabilization system for a binocular robotic head (google “jaguar chicken commercial”:) ). After this, I came across a project at the MAVlab where I was allowed to work with tiny MAVs called pocket-drones, and the first thing I thought was ‘awesome!!’, however, that was before I realized that working with tiny MAVs was not a piece of cake. I’m glad that the inner-geek in me kept me excited for 4 years straight, which is a crucial element or any PhD I believe, or any job!

So now a new chapter will start for me in Sweden. I hope that I will append to Bitcraze’s vision and I’m quite sure that they will teach me how to become a better hardware and software developer. I’m excited to start working with Arnaud, Marcus, Kristoffer and Tobias and develop more things that will make any’s inner-geek jump a little from excitement. At the moment of writing, we will be on ICRA in Montreal, Canada, so jump by booth 101 to say hi!

Sneak peak of what’s coming at ICRA 2019

2019-05-13 | Tobias | 2 Comments

Only a week left until we stand in our ICRA booth in Montreal and give you a gimps of what we do here at Bitcraze. As we have been writing about earlier we are aiming to run a fully automated demo. We have been fine tuning it over the last couple of days and if something unpredictable doesn’t break it, we think it is going to be very enjoyable. For those that are interested in the juicy details check out this informative ICRA 2019 page, but if you are going to visit, maybe wait a bit so you don’t get spoiled.

Apart from the demo we are also going to show our products as well as some new things we are working on. The brand new things include:

AI-deck, Active marker deck and Lighthouse-4 deck

AI-deck: This is a collaborative product between GreenWaves Technologies, ETH Zurich and Bitcraze. It is based on the PULP-shield that the Integrated and System Laboratory has designed. You can read more about it in this blog post. The difference with the PULP-shield is that we have added a ESP32, the NINA-W102 module, so that video can be streamed over WiFi. This we hope will ease development and add more use cases.
Active marker deck: Another collaboration, but this time with Qualisys. This will make tracking with their motion capture cameras easier and better. Some more details in this blog post. Qualisys will have the booth just next to us were it will be possible to see it in a live demo!
Lighthouse-4 deck: Using the Vive lighthouse positioning system this deck adds sub-millimeter precision to the Crazyflie. This is the deck used in the demo and could become the star of the show.

Adding to the above we will of course also display our recently released products:

Crazyflie 2.1: The Crazyflie 2.1 is an improvement of the Crazyflie 2.0 but keeping backward capability.
- Better radio performance and external antenna support: With a new radio power amplifier we’ve improved the link quality and added support for dual antennas (on-board chip antenna and external antenna via u.FL connector)
- Better power button: We’ve gotten feedback that the power button breaks too easily, so now we’ve replaced with a more solid alternative.
- Improved battery cable fastening: To avoid weakening of the cables over time they are now run through a cable relief.
- Improved sensors: To make the flight performance better we’ve switched out the IMU and pressure sensor. The new Crazyflie uses the drone specialized sensor combo BMI088 and BMP388 by Bosch Sensortech.
Flow deck v2: The Flow deck v2 has been upgraded with the new ST VL53L1x which increases the range up to 4 meters
Z-ranger deck v2: The Z-ranger v2 deck has been upgraded with the new ST VL53L1x which increases the range up to 4 meters
Multi-ranger deck: The Multi-ranger deck adds VL53L1x sensors in all directions for mapping and obstacle avoidance.
MoCap marker deck: The motion capture deck with support for easily attachment of passive markers for motion capture camera tracking.
Roadrunner: The Roadrunner is released as early access and the hardware is basically a Crazyflie 2.1 without motors and up to 12V input power. This enables other robots or system to use the loco positioning system.

You can find us in booth 101 at ICRA 2019 (in Montral, Canada), May 20 – 22. Drop by and say hi, check out the products & demo and tell us what you are working on. We love to hear about all the interesting projects that are going on. See you there!

PULP-DroNet: open source and open hardware artificial intelligence for fully autonomous navigation on Crazyflie

2019-05-06 | Daniele Palossi | 3 Comments

Hi everyone, here at the Integrated and System Laboratory of the ETH Zürich, we have been working on an exciting project: PULP-DroNet.
Our vision is to enable artificial intelligence-based autonomous navigation on small size flying robots, like the Crazyflie 2.0 (CF) nano-drone.
In this post, we will give you the basic ideas to make the CF able to fly fully autonomously, relying only on onboard computational resources, that means no human operator, no ad-hoc external signals, and no remote base-station!
Our prototype can follow a street or a corridor and at the same time avoid collisions with unexpected obstacles even when flying at high speed.

PULP-DroNet is based on the Parallel Ultra Low Power (PULP) project envisioned by the ETH Zürich and the University of Bologna.
In the PULP project, we aim to develop an open-source, scalable hardware and software platform to enable energy-efficient complex computation where the available power envelope is of only a few milliwatts, such as advanced Internet-of-Things nodes, smart sensors — and of course, nano-UAVs. In particular, we address the computational demands of applications that require flexible and advanced processing of data streams generated by sensors such as cameras, which is beyond the capabilities of typical microcontrollers. The PULP project has its roots on the RISC-V instruction set architecture, an innovative academic and research open-source architecture alternative to ARM.

The first step to make the CF autonomous was the design and development of what we called the PULP-Shield, a small form factor pluggable deck for the CF, featuring two off-chip memories (Flash and RAM), a QVGA ultra-low-power grey-scale camera and the PULP GAP8 System-on-Chip (SoC). The GAP8, produced by GreenWaves Technologies, is the first commercially available embodiment of our PULP vision. This SoC features nine general purpose RISC-V-based cores organised in an on-chip microcontroller (1 core, called Fabric Ctrl) and a cluster accelerator of 8 cores, with 64 kB of local L1 memory accessible at high bandwidth from the cluster cores. The SoC also hosts 512kB of L2 memory.

Then, we selected as the algorithmic heart of our autonomous navigation engine an advanced artificial intelligence algorithm based on DroNet, a Convolutional Neural Network (CNN) that was originally developed by our friends at the Robotic and Perception Group (RPG) of the University of Zürich.
To enable the execution of DroNet on our resource-constrained system, we developed a complete methodology to map computationally-intense deep neural networks on the PULP-Shield and the GAP8 SoC.
The network outputs two pieces of information, a probability of collision and a steering angle that are translated in dynamic information used to control the drone: respectively, forward velocity and angular yaw rate. The layout of the network is the following:

Therefore, our mission was to deploy all the required computation onboard our PULP-Shield mounted on the CF, enabling fully autonomous navigation. To put the problem into perspective, in the original work by the RPG, the DroNet CNN enabled autonomous navigation of big-size drones (e.g., the Bebop Parrot). In the original use case, the computational power and memory was not a problem thanks to the streaming of images to a remote base-station, typically a laptop consuming 30-100 Watt or more. So our mission required running a similar workload within 1/1000 of the original power.
To make this work, we combined fixed-point arithmetic (instead of “traditional” floating point), some minimal modification to the original topology, and optimised memory and computation usage. This allowed us to squeeze DroNet in the ultra-small power budget available onboard. Our most energy-efficient configuration delivers 6 frames-per-second (fps) within only 64 mW (including all the electronics on the PULP-Shield), and when we push the PULP platform to its limit, we achieve an impressive 18 fps within just 3.5% of the total CF’s power envelope — the original DroNet was running at 20 fps on an Intel i7.

Do you want to check for yourself? All our hardware and software designs, including our code, schematics, datasets, and trained networks have been released and made available for everyone as open source and open hardware on Github. We look forward to other enthusiasts contributions both in hardware enhancement, as well as software (e.g., smarter networks) to create a great community of people interested in working together on smart nano-drones.
Last but not least, the piece of information you all were waiting. Yes, soon Bitcraze will allow you to enjoy of our PULP-shield, actually, even better, you will play with its evolution! Stay tuned as more information about the “code-name” AI-deck will be released in upcoming posts :-).

If you want to know more about our work:

Questions? Drop us an email (dpalossi at iis.ee.ethz.ch and fconti at iis.ee.ethz.ch)

Work on ICRA 2019 demo

2019-04-29 | Kristoffer Richardsson | Leave a comment

As mentioned earlier, we will be attending ICRA 2019 and have started to work on the demo that we will run in our booth. The main features we are going to show this time are the Lighthouse positioning deck and the Crazyflie 2.1.

One Crazyflie flying and 5 re-charging on their pads

Running a demo with flying Crazyflies at a conference, usually means a lot of work with changing and charging batteries, starting demos and so on. This takes time and leads to less time to talk to all the interesting people that visit our booth. This year we are aiming at making the demo as fully automated as possible. We will have 8 Crazyflie 2.1 with Lighthouse and Qi charger decks, each with a charging pad. A computer will orchestrate the Crazyflies and make sure one is flying at all times while the others re-charge their batteries. When the battery of the flying Crazyflie is depleted it goes back to its pad while another one takes over.

Most of the functionality is implemented in the Crazyflie firmware and it is pretty much autonomous after the trajectory is started. It will fly the trajectory over and over until it is low on battery, when it goes back to where it started from for recharging. Even though the Lighthouse positioning is very good, it sometimes slips off the charging pad when landing, so we have added a reposition feature to take off again and land if it is not charging after landing.

The orchestration computer (we call it the Control tower) is just keeping track of the state of the flying copter and starts a new one when the flying one goes back the the pad.

We are reusing the spiral trajectory from IROS last year and it has the property that it is possible to run up to 4 copters at the same time without colliding, if they are started at the correct times. There is a swarm feature in the Control tower that runs up to 4 Crazyflies with continuous replacements. The chargers are not fast enough to keep it going and it does not work as expected every time, but it is exciting with a few more Crazyflies buzzing around!

There are some bits and pieces left to implement, like crash detection, but the demo is mostly functional. If you want to play with the (slightly hackish) code, it is available at https://github.com/bitcraze/crazyflie-firmware-experimental/tree/icra-2019

You can find us in booth 101 at ICRA 2019, May 20 – 22. Drop by and say hi, check out the demo and tell us what you are working on. We love to hear about all the interesting projects that are going on. See you there!

The active marker deck for MoCap systems

2019-04-23 | Tobias | 2 Comments

As part of our collaboration with Qualisys we are helping them developing an active marker deck for their motion capture cameras. One of the major benefits with an active marker deck is that it can have an ID, thus it is much easier to track each Crazyflie in e.g. a swarm. Another benefit is an increased range compared to passive markers thanks to high power emitting IR LEDs.

Active marker deck mounted on a Crazyflie

We are currently only in the prototype stage but we have already managed to do initial fight tests so hopefully we can release it within a couple of months.

We will bring some prototypes to ICRA 2019, come and visit us and Qualisys to check the deck out.

Innovation week and first preparations for ICRA 2019

2019-04-15 | Kristoffer Richardsson | Leave a comment

We attended the Innovation Week at Lund University on Thursday last week. Primarily we wanted to talk to students and possibly find future colleagues (yes, we are hiring) but it was also a good opportunity to get some demo time with the Lighthouse positioning system.

As mentioned in an earlier blog post, we are going to ICRA in May and we have started to think about what to demo. The main feature will of course be the Lighthouse deck. The setup at Innovation week also served the purpose of a first iteration for the ICRA setup.

We reused an old cage that we created for another fair a couple of years ago, built from a garden tent. It turned out to be fairly wobbly and a bit heavy (steel tubing) considering we will bring it in our luggage to Canada. We probably have to rethink the construction a bit and see if we can change to aluminium.

We put the Lighthouse base stations on tripods, which worked like a charm in our flight lab. We found that we had a lot of problems calibrating the system, not to mention flying the Crazyflie, at the Innovation week fair though. It turned out that the floor was not as stable as one might expect and that the tripods were swaying when people walked by. We solved the problem by adding a tube to the top of tripod that was pushed against the ceiling and thus minimizing the movement. Experience from the real world is always useful!

The general idea for the demo at ICRA is to automate as much as possible to give us more time with visitors. With the high precision of the Lighthouse system, it should be easy to land the Crazyflies on Qi chargers to avoid changing batteries. We hope to set up 6-8 Crazyflies where one is always flying while the others are charging, and have the possibility to temporarily fly more Crazyflies for small swarms. It is still just ideas and we will not see the end result until we are at ICRA, but it will be fun to build!

Our flight arena is coming along

2019-04-08 | Tobias | Leave a comment

As we have written about before we moved to a new office last month. One of the major reasons was the need for a bigger flight lab. This will enable us to do better testing and improve how we can develop things, especially our positioning solutions. Even though it is not a huge place, going from 4x4x3m to 8x8x3.5m and possibly 13x9x3.5m, is a great improvement for us. We call this great playground for the Arena as this is what it feels like for us :-). Very soon we hope to have our Qualisys MoCap system, Lighthouse and the Loco positioning running, maybe even at the same time.

As a bonus this is a great pace to play HTC Vive VR games, we just need to get the wireless transmitter so we can make full use of the space!

One of the first flights with LPS Two Way Ranging

The Crazyflie MoCap marker deck is available

2019-04-01 | Tobias | Leave a comment

Today we received a bunch of MoCap marker decks which means they are now available in our shop. This is a handy deck for those that flies in a motion capture system as it is easy to create different configurations and move between Crazyflies.

The deck is designed in collaboration with Qualisys. We suggest using 6.5mm, 8mm or 9.5mm diameter reflective motion capture markers. Currently we don’t offer the markers but soon we will also offer a bundle together with markers.

Drawing images with Crazyflie and Bitcraze at ICRA 2019

2019-03-25 | Arnaud | Leave a comment

Last week we posted about painting with the Lighthouse deck. This week we continue on the same track but add a new dimension, all in our “let’s try this crazy idea” spirit. So last Friday, after having a lot of fun painting with the Crazyflie led-ring using long exposure photo and the Lightouse deck for positioning, we had one extra crazy idea. Can we use the Crazyflie to show a raster image, very much like the way a CRT monitor works by sweeping line by line and displaying the pixel color one by one, using the led-ring? Unfortunate we did not have enough time that day…

However the idea was so intriguing that Kristoffer couldn’t stop himself from writing a prototype script during the week-end. So last Monday, just after publishing the blog post, we went to the flight arena and tried it. After a couple of trial and error we found a display algorithm that showed a pretty good result:

The source for this image is this very low resolution Mona Lisa:

It was a very fun experiment, it is magic to see the Crazyflie going back and forth blinking for ~3 minutes, click on the camera and see the resulting picture. It is also a really nice way to observe the current state of the lighthouse positioning. The lines are spaced by about 3 cm and the Crazyflie is controlled using the PID controller. The controller do a decent job of keeping the Crazyflie in lines and the space seems a little bit ’tilted’.

If you are curious or if you want to try by yourself, we pushed the script in the Crazyflie-lib-python example folder.

As a side note, we will be exhibiting at the ICRA 2019 conference May 20-24, 2019 in Montreal, Canada. We will running demo of the LPS and Lighthouse (though I am not sure we can print long exposure picture, this is not so exciting to look in real-time :). We hope you would like to come and meet us there!

Release of the Lighthouse deck early access

2019-03-11 | Arnaud | 3 Comments

We have just released the Crazyflie Lighthouse deck as Early Access! It is now available in our web store.

The lighthouse deck allows the Crazyflie to estimate its position using the HTC Vive tracking base-station normally used for Virtual Reality. The positioning is done by tracking the timing of rotating infra-red laser beams emitted from the base-stations. This system has the advantages of having a very good precision and of allowing the Crazyflie to acquire its position autonomously: once the Crazyflie knows the position and orientation of the base-station, it can calculate its own position without the help of any external systems.

The release as Early Access means that we have finished the hardware and we are confident that the hardware is working properly. Though we have not yet finished all the software and firmware, by releasing the hardware early we can get the hardware into the hands of users quickly to try it out. In return we hope we can get some help making the software better.

Current state

The Crazyflie can calculate its position from the received Vive Base-Station V1 signals.
Direct line of sight should be kept to both base-stations. The Lighthouse deck has 4 receivers so in the future it will be possible to get a position from seeing only one base station.
Base-Station V2 support is still being worked-on, it will only require a software update.
The Base-station position is hard-coded in the Crazyflie and found using SteamVR. Ideally this should be sent from the ground and the Crazyflie should calculate the positions of the Base-Stations automatically.
The previous point means that a full VR system or at least two base stations and a controller or tracker is required to setup the system. In the future we hope to setup the system with only a Crazyflie and two base stations.
Since this version of the deck only has horizontal sensors, it is important that the base-stations are placed above the flight space and the Crazyflies should fly ~40cm bellow the base-stations

As long as the deck is in early access, the main documentation will be the lighthouse positioning page in the wiki. This page is going to be updated a lot in the near future and will track the progress in development.

Demo

We have written a small demo script that allows to set the position of the Crazyflie using a Vive controller. It is a good demo to experiment with the precision of the system and the ability to mix VR and Crazyflie since they are in the same tracking space:

In this demo, a python script connects to two Crazyflies and acquire the controller position using OpenVR and makes the Crazyflies take-off above the controller. Then, when the controller trigger is pushed, the setpoint to the closest Crazyflie is changed to follow the controller movement, the Crazyflies are flying autonomously only getting position setpoints from the python script. The position estimation and control is handled onboard.

We are pretty excited by this release since we think this positioning technology will be very useful for a lot of use-case. Let us know what you think and do not hesitate to contribute if you want to improve the system :).