Where do I add my code?

2021-11-22 | Kristoffer Richardsson | Leave a comment

Sometimes we get the question of where to modify or add code to change some behavior of the Crazyflie. There is no quick answer to this question but we thought that we should write a post to clear up some question marks and give a better idea of how to approach the problem.

There are quite a few repositories on the Bitcraze github page but there are two that are the main focal point for almost any Crazyflie work, those are the crazyflie-firmware and the crazyflie-lib-python. The crazyflie-firmware contains the source code (written in C) for the firmware that runs in the Crazyflie, that is the code responsible for flying, blinking LEDs, communicating with the radio, scanning sensors and so on. The crazyflie-lib-python (often called the python lib) on the other hand is running on the PC side and is the API to use to communicate with the Crazyflie from a script. The crazyflie-lib-python is also used by the python client which means that anything you see in the client can be done by a script using the python lib.

Let’s assume we have a system of one Crazyflie connected to a computer using a Crazyradio. Now we want to control the Crazyflie and make it take off for instance, how should this be done?

The easiest way would be to use the python lib. The python lib is used to communicate with the Crazyflie and we can use it to send instructions to the Crazyflie, for instance to take off or fly a trajectory. It is also possible to use the parameter framework to change values in the Crazyflie. The main way of monitoring what is going on in the Crazyflie is to use the log framework to read variables from the Crazyflie. The python lib is perfect for controlling the Crazyflie or prototype ideas as it is very fast to make changes and try things out. The best to get started with the python lib is to start from an example that already uses functionalities you want to use.

Another option is to add code in the firmware. Originally this has been quite hard since the firmware has not been initially designed to accept user code. This means that unless you want to modify an already existing code, it is quite hard to find where to add your code so that it runs in the Crazyflie firmware and you would have to make a fork of the firmware which can he hard to maintain and keep up to date in the long run. This is one of the things the out-of-tree build and the app layer is solving, it is now quite easy to add and run your own C files to the firmware in your own project without having to fork the Crazyflie firmware. There is a bunch of examples in the firmware that shows how to implement autonomous behavior as an app. The easiest is to start with the hello world example. When it comes to modifying exiting functionality in the firmware code, most of the time forking and modifying the official firmware unfortunately is the only solution. We are however working our way to make more and more of the firmware modular so that it can be expanded out of tree. For example there has been work to make an out of tree estimator possible to implement.

A prototype written as a python script is often pretty easy to move to the Crazyflie firmware. This is a good pattern when writing an application, rapid prototyping in python and then finalizing in firmware if needed. The best example of that is the push demo. It is a demo where the Crazyflie can be pushed-around with the help of the flow deck for autonomous flight and the multiranger deck for detecting obstacle/hands. We have a python cflib push demo as well as a Crazyflie firmware push demo app.

There is some support in the python lib for interacting with multiple Crazyflies and it is probably a good start point for simple swarms. For more advanced swarm work Crazyswarm may be a better option.

If you would like to see some of the process in action, we have made a workshop during our BAM days about implementing functionality both using the python lib and in the firmware as an out of tree app:

Fun Friday charging and lights

2021-11-15 | Marcus | Leave a comment

During the preparations for BAM, for example the awesome demo that Kristoffer did (link to demo), we had a lot of discussions about landing on charging pads. The Qi deck is a good solution, but the design is a bit complex and makes it hard to have other electronics pointing dowards (although we’ve seen some solutions to this). So after spending a few days thinking about this, I set out to spend a few fun Fridays looking at solutions for this. The results (so far) are detailed in this post.

Contact charger and decks

The idea I decided to try out was something we’ve been discussion on-and-off for a few years, using contact charging instead of Qi. So I revived a few of our old ideas and starting looking for parts. This is always a fun phase of an idea, the sky is the limit! I’m kind of used to electronic parts, but I find looking for mechanical parts is a lot harder. I don’t really have the vocabulary for this and all the parts look the same size in the product photos…

Finally the solution I wanted to test was using small pogo-pins. The idea is that the pogo-pins could be included into any design of new boards and easily enable charging. With the parts found and the ideas fresh again, I jumped into KiCad (5.99) and started the design. Two different designs were made, pictured below.

Why two solutions? I wasn’t sure which one would work well. I suspected the 4 segments would be better, but I was hoping for the two circles, since you would not need additional electronics to handle 5V/GND switched depending on how you landed.

Since there was two solutions for the pad, there’s also two solutions for the deck. I wasn’t sure about what pogo pins to use, so there’s two different sets of pins (which makes it a bit confusing).

Deck for segmented charger (with diode bridge on back)

Deck for circular charger (without diode bridge)

With the components and PCBs in hand I started testing, and…the results were not promising. The design was made to work with our current Qi charging pad (link). Although the alignment for Qi is important, the tolerance was not as tight as what I needed. Back to the drawing board, this time the mechanical one. I used FreeCAD to create a new version where the Crazyflie would align better. After a few iterations with the 3D printer I ended up with this:

Putting it all together, the results were promising. With the new mechanical design the area the Crazyflie can land in is larger and the alignment much better. I think it’s good enough to show that the concept can work. The next step is to look a bit more at the pins, they might be a bit too easy to damage.

The Glow deck

I mentioned above that the idea was to be able to put charging on decks with other designs on them. To test out this concept I revived another old idea, the Glow deck! The Glow deck is a design where the LEDs are visible from the side and also a bit stronger. To achieve this a diffuse lens would be mounted on the PCB.

The first version of the Glow I made was years ago. It contained a high-power LED, which was really bright…and really warm. We stopped measuring the PCB temperature at 140 C, so that wasn’t something that was going to work out. It also only had one color and I wanted more.

This time around I decided to try two different versions. The reason for the two versions was that I wanted to see if more power actually added something (except complexity and price) when it came to visual effects while flying. One version uses 4xWS2812B LEDs (the same as on the LED-deck) while the other one uses a RGBW high-power LED. I quickly ran into the same problem as with the charging deck, mechanics. I wasn’t able to find a good size of a diffuse lens. Not wanting to be blocked by this I decided on designing one myself and printing it. Not sure how the size would affect the effect I designed two versions.

Left: WS2812B version, right: high-power RGBW version

I haven’t had the time to try out the high-power LED version yet, but the results for the WS2812B version with the diffuse lens looks promising. It’s hard to see the effect in the images, but at least it shows that it’s possible to see the LED from the sides.

Conclusions

I think both the contact charging and the Glow looks promising and I’m happy with the results so far. But making one prototype is easy, making something that can be manufactured and properly sourced is different. If any of these ever turn into a product, the challenge will be finding the right pins, lens, charge base etc.

For the next step, I’m excited to see the high-power version of the Glow in action. This time around I’ve added proper dimming and temperature measurement :-) If you have any ideas, comments or questions drop a line below!

Real-time control of a swarm of Crazyflies in artificial forests

2021-11-08 | Enrica | 4 Comments

This week we have a guest blog post from Enrica Soria from the Laboratory of Intelligent Systems Faculty of Ecole Polytechnique Fédérale de Lausanne (EPFL) . Enjoy!

From Star Wars to Black Mirror, sci-fi movies predict a future where thousands of drones will fill our sky. Curving sharply around trees or soaring over buildings, they fly just like a flock of starlings. To turn this vision into a reality, real drone swarms need to increase their autonomy and operate in a decentralized fashion. In a decentralized swarm, each robot makes its own decision based only on local information. Decentralization not only allows the swarm to be more robust to the failure of single individuals, but also removes the dependency from a single computing unit, thus making the swarm more scalable in terms of size.

We at LIS (EPFL) have shown that predictive controllers can improve the safety of aerial swarms by predicting and optimizing the agents’ future behavior in an iterative process. However, the centralized nature of this method allowed us to only control five drones and prevented us from scaling up to a large number of drones. For this reason, we have worked on a novel decentralized and scalable swarm controller that allows the safe and cohesive flight of aerial swarms in cluttered environments. In our latest article, published in IEEE Robotics and Automation Letters (RA-L), we describe how we designed the controller, show its scalability in size, and demonstrate its robustness to noise. We studied the swarms’ performance and compared how it changes in two different environments: a forest and funnel-like environment.

The Crazyflie 2.1 was the perfect platform for our experiments. They are lightweight, modular, and tough. This quadcopter can survive big hits when things don’t go as planned… and, if you work on swarms, things can go wrong!

*The fleet of Crazyflies equipped with a single marker.*

With our algorithm, sixteen robots were able to fly through an artificial forest that we set up in our indoor motion capture arena. In our previous work, we installed four markers on each quadcopter and used the rigid body tracking from Motive (the Optitrack software). The large volume of our experimental room required the usage of big markers for long-distance detection, which added considerable weight to the drone. Hence, in our new work, we use a single marker per drone. Tracking is supported by the ‘crazyswarm’ package and communication with the entire swarm only requires two radio links. However, despite our model being decentralized, in our implementation robots relay the information to an external brain, which does the computations for them. In the future, all the necessary code will be embedded onboard, removing the dependency on external infrastructure.

*Our predictive swarm of Crazyflies flying among obstacles in our indoor experimental room.*

Video about the article

This work is a step forward towards the fully autonomous deployment of drone swarms in our cities. By enabling safe navigation in cluttered environments, drone fleets will be able to integrate with conventional air traffic, search for missing people, inspect dangerous areas, transport injured people to hospitals quicker, and deliver important packages right to our doors.

For further details, check out our article here!

Improvements in documentation

2021-11-01 | Jonas Danielsson | 2 Comments

We have recently worked on functionality in our web site to generate documentation from source in a few ways that we hope will improve the quality as well as simplify maintenance. We have already written a bit about the log and param documentation in the crazyflie-firmware repository, but we now also added and API reference in the python library as well as generating a list of publications related to the Crazyflie.

The Log and Param documentation

Earlier this year we worked on generating documentation for Log and Params from doxygen comments. We will not dig deeper into this here, but you can read more about it in this blog post. The latest version is available on our web in the repository documentation for the crazyflie-firmware: logs and params

An API reference for our Python library

The Crazyflie Python library is what you might use to create programs that interact with the Crazyflie, for instance to manage small swarms. For a while we have had introductory documentation, and step-by-step guides to show you how to perform, what we think are, common tasks.

However along the way we have also added comments and examples to our code. And this, combined with the way we have structured the library actually enables us to automagically generate reference API documentation. That is, something that shows you everything that is possible to do with the library, all modules and classes, all methods and constants that the library offers.

After some recent work this is now happening and the documentation will now get generated each time we deploy our website!

The API reference documentation can be found in the repository documentation for the library. Please check it out! And be picky, complain where the documentation is lacking! Or if the formatting seems weird! We are trying to get the hang of this and we need you to push us!

Managing publications

Some years ago we started to add publication that are related to the Crazyflie to our Research page, we hope that it might be inspiring to read about all the awesome things that the Crazyflie is used for.

Until now it has been a simple list in markdown but with an increasing number of publications it has become harder and harder to maintain it, finally we have put the work into generating the list from a BibTeX (.bib) file instead. One advantage of the new solution is that BibTeX is a well known format to the research community with lots of tools around to manage BibTeX files while another improvement is that the list will be formatted in a consistent manner (which was not always the case earlier).

If you want to add a publication to the page, simply update the .bib file with your data and create a pull request with the changes. We will merge when appropriate and the publication will become visible on the web after the next deploy, usually within a few days.

Behind the scenes

The code for generating documentation from source tends to spread out over multiple repositories and creates some complexity with a multitude of tools for different languages. It should not be necessary to understand the details and we hope the system will be easy to use for contributors to the code base.

Any questions or comments are welcome.

Happy coding!

BAM days are a wrap !

2021-10-25 | Barbara | Leave a comment

The last week was epic. We had 3 days of our online conference, the BAM days – I’m sure you’ve heard of them by now.

We are really happy with how everything went down. During those 3 days, 142 people attended, which is a highest number than we could have expected. The Welkom platform we used was stellar, allowing us to use Mibo rooms for very fruitful discussions after each talk.

Quiz and community Q&A

We took the opportunity to talk to our community, which is something we didn’t have the opportunity to do in a long time. Your insights and feedback were greatly appreciated and we have a lot to think about in the next coming weeks on how to best use all the remarks we got.

We also had a short quiz about Bitcraze, and we were quite impressed with how you performed ! And interesting to note that the hardest question for you was how many decks we sell (it’s 16, if you want to cheat on our hypothetical next quiz)

As I’ve mentioned, after each event we gathered in Mibo rooms. Even though attendance there was not as high as we would have liked, we still got quality time with community members, speakers, collaborators, even first-timers that were interested in the Crazyflie. We really love this platform, making us feel almost like meeting in real life. We even had some karaoke in Mibo during the closing party (which MAY be a good excuse to end the day for those who listened)

Content

Our external speakers presented a lot of interesting work. It was a great pleasure and honor to welcome every one of them as they explained their latest work. I have to admit that it’s rewarding to see such smart people doing awesome and cool research with our products.

We did our share too, with workshops and demos. Kristoffer’s autonomous demo using distributed consensus required a lot of work but worked perfectly in the end. Here is a small excerpt:

What now?

Now, we’re feeling as everyone is feeling the day after a party: exhausted, happy, and wondering what to do next. Hopefully we have some plans for that !

If you missed the conference, we created a Youtube playlist where you can watch everything that you missed. During the next few days, we’ll update the event page with BAM’s presentations too so you will have the opportunity to catch up. Some of our workshops will also turn into tutorials or documentations of some kind, but we’re still just cleaning up.

We are so happy with how everything went that we are already thinking about a future BAM. This one was exceptional, of course, since it was at first to celebrate our 10 year anniversary (and I have to admit that we’re all a little bit tired after 3 intense days), so we’re not going to be able to top that. But we are considering making BAM a fixed point in our agenda (and yours, let’s hope). We don’t how, we don’t know when, but one thing is sure: BAM is just beginning.

Kimberly on a different continent

On a totally different note, Kimberly is flying to the US this week: if any of you America-based wants to grab the occasion to have a more time-zone appropriate conversation with one of us, you will have a few weeks to make it possible!

Before BAM days

2021-10-18 | Barbara | Leave a comment

I’m sure you know about it by now, but as a reminder, the BAM days are this week !
We’re all are really excited about it – and, let’s be honest, a little nervous too as the day grows nearer.

We are really happy to see that so many of you plan on joining us. We have more than 100 people registered to the event, which is actually the highest estimate we were hoping for. By the way, if you are registered to the event, dismiss the Event Brite email you got on Sunday and prefer the direct Welkom link you’ve been sent: the process is much easier if you use the latest. If you did sign up, but haven’t received an email with any link (even after checking your spamfolder) , please send us an email at contact@bitcraze.io!

Our workshops are all ready and we’re proud to present you our selected subjects. Each one of us will get the opportunity to talk about an area he cares about, so if you attend the whole event you will get also the chance to meet all of us. Of course, the social fika will also be the perfect place to meet us, so don’t hesitate to jump in the Mibo room we’ve set up!

There are also keynote speakers that we’re honored to host, close collaborators whose work we admire. It will be really interesting to hear from them, whether about Motion Capture or swarms.

There are been a few changes on the agenda:

On Wednesday morning, we’ll have a second session of the quiz. Those that missed it on Tuesday evening can then also get a chance of winning the one and only Goldenflie. It will be followed by a community Q&A: if you have any questions unanswered, it will be the time to ask them! In the afternoon, Arnaud will talk about Rust and the App layer after Greenwaves’s talk (and not on the morning as announced before).
If you’re interested in swarms, Friday morning will be perfect for you as we have a double feature: Wolfgang will present his CrazySwarm and Jonas will follow with a presentation about using the CFlib to fly a swarm. Afterwards, they will have a panel discussion on swarm management.

There is also a demo programmed on Friday. We haven’t told much about it yet because we wanted to be sure it was possible to do before we promised anything, but I can tell now that on Friday, you will be able to see an autonomous swarm making distributed decisions. It’s the first time we’re trying something like that and you don’t want to miss it !

All the info is here.

Just be aware that during the BAM days, we won’t be as reactive on answering our emails or the forum, as most of us will be on the platform, but we’ll still be shipping orders.

See you tomorrow !

See you at BAM days!

2021-10-11 | Barbara | Leave a comment

The BAM days are coming up !

Only 8 days left before the first Bitcraze-organized conference ever. We are really excited about it, and this coming week is pretty much dedicated to organize the conference. We’re working hard on getting the best experience possible for there 3 days, both for us and for you. That means collaborating with great speakers, creating awesome workshops, but also think about the best way to come together – one of the main purpose behind the BAM days.

It’s been a couple of months since we decided that we wanted to organize our own conference. The main idea behind it was that we missed going to conferences, meeting people and talking with a lot of interesting persons.

Bitcraze turning 10 felt like a big occasion that we wanted to celebrate. And indeed we did, internally, but we wanted to share this accomplishment with you too. An online conference seemed like the best idea. So when time came to choose a platform for it, we looked for something which put emphasis on social experience. We finally went with the Welkom platform, from the Netherlands. It’s easy and natural to use, with a simple interface, and lots of options for us to organize different activities.

The Welkom platform we’ll use hosts two different Mibo rooms. It’s a spatial chat, where your avatar can run around a 3D world, discovering different environments. Your camera is your head, so you’ll need a webcam to join. There are some fun activities around, plenty interactive stuff, from chairs to basketballs. We have used it to for our coffee breaks when we were working from home, and had fun using this world: we’ll be happy to meet you there and hope you’ll find it fun too.

The plan is to have between each talk or workshop at least 30 minutes in Mibo to talk to the speakers, take a coffee break, discuss the exciting things that’s been happening, and generally catch up in ways we haven’t be able to the past year.

At the end of each day, we’ve programmed social events to gather together. On Tuesday, we’ll host a special quizz. Telling you its content may be a bit cheating, but if we’ve read our blogposts regularly, the questions should not seem hard to you. For the winner, a once-in-a-lifetime reward: a GoldenFlie ! You will be able to put it in your chimney mantle with your others awards, or actually use it – because, yes, it flies!

The second day, Wednesday, we’ll have a roundtable with Q&A. A perfect occasion for us to get feedback from you, and for you to ask us all the questions you have been burning to ask us.

The last day (Thursday), we’ll talk a little about Bitcraze and its future: where it is headed. We’ll then have a last party – in Mibo of course.

All the program and other useful information is in our event page. We really hope you’ll take some time to come by and say hello !

IROS 2021: Crazyflie’s adventures in Robotic Research

2021-10-04 | Kimberly McGuire | Leave a comment

Last week it was time for the International Conference on Intelligent Robots and Systems (IROS), which is one of the biggest yearly robotics conferences! In previous editions (see 2018 blog post, 2019 event page), we joined IROS’ exhibition and showed an autonomous demos for everybody to enjoy during the coffee breaks. Then… as everybody knows… the Covid-19 pandamic hit and we had to cancel our plans to go to the 2020’s Las Vegas edition. This year’s IROS was supposed to be held in Prague, but was very early announced that this was going to be an online edition.

Eventhough we considered joining IROS’s online exhibition, after our decision to organize the Bitcraze Awesome Meetup (BAM) days as a celebration to our 10 year anniversary (see this blo gpost), we wouldn’t have enough time to prepare for both. However, we still signed up for the conference itself, so we could see how and where the Crazyflie is used in the robotics community! So this blog post will give a list and small overview of the Crazyflie-fueled research with some highlights.

Crazyflie as a Demonstration Tool in Research

A common use-case that we saw is to use the Crazyflie as a research demonstration tool. In the case of designing algorithms for quadcopters, many of the researchers we heard from or talked to told us that there is significant added value to demonstrate their ideas on a real platform. Then it can be truly evaluated with real environmental issues and platform dynamics. For this, the majority of the papers have used the ROS package developed for the CrazySwarm project but some researchers has gone as far as implementing it fully on the Crazyflie’s MCU [6, 8]!

We also noticed the large spread of the Crazyflie-featured papers throughout the sessions of the conference. A bunch of the papers could obviously be found in Aerial System sub-sessions as in ‘Perception and Autonomy’ [1], ‘Applications’ [3], ‘Mechanics and Control’ [4 , 5], but many were spread out in areas like Reinforcement Learning [2], Localization [6] and Collision avoidance [7]. There were also some papers to be found in the Swarm Robotics session [8, 9], as well as in Distributed Robot Systems [10].

Note that there is usually quite some overlap between the different sessions, but it is still very exciting to see that the Crazyflie being used in so many fields!

Highlights and the Community

We saw lots of awesome applications but there are a few that we really like to highlight! There was a presentation of an actual solar-powered Crazyflie [2] which can literally fly forever. This research was done by the same group at the University of Washington that also was featured in a blog post last year, and who is responsible for the Smellicopter. Also, our jaw literally dropped at the sight of the 4 Crazyflie-fueled mega-copter by the University of California, Los Angeles [4, 5]. Last but not least, the fully onboard autonomous gas-seeking swarm, Sniffybug [8], from the MAVLab (TU Delft), never stops to amaze us. They wrote a blog post about their work on our website this summer!

We are hoping for more interesting blog posts related to the papers in the list below in the future. We also would like to invite all the researchers, who have been working with the Crazyflie, to join us for discussions at the Community Q&A at the BAMdays. Most of the amazing aspects of the Crazyflie in terms of autonomy, localization and swarming have all been contributions of the research community in the past. So join the discussion to continue that path in order to bring the Crazyflie to a new level!

List of IROS 2021 Papers featuring the Crazyflie

Here is a list of all the Crazyflie-related papers we could find, but please let us know if we are missing any!

[1] Target-Visible Polynomial Trajectory Generation within an MAV Team Yunwoo Lee, Jungwon Park, Boseong Jeon and H. Jin Kim
- Lab for Autonomous Robotic Research (LARR), Seoul National University
- Video
[2] Inclined Quadrotor Landing using Deep Reinforcement Learning Jacob E. Kooi and Robert Babuska
- Cognitive Robotics, Delft University of Technology
- ArXiv Preprint
- Video
[3] Toward battery-free flight: Duty cycled recharging of small drones Nishant Elkunchwar, Suvesha Chandrasekaran, Vikram Iyer and Sawyer B. Fuller
- Department of Mechanical Engineering, University of Washington
[4] An Over-Actuated Multi-Rotor Aerial Vehicle with Unconstrained Attitude Angles and High Thrust Efficiencies Pengkang Yu, Yao Su , Matthew J. Gerber, Lecheng Ruan and Tsu-Chin Tsao
- University of California, Los Angeles
- Paper IEEExplore
[5] Nullspace-Based Control Allocation of Overactuated UAV Platforms Yao Su, Pengkang Yu, Matthew J. Gerber, Lecheng Ruan and Tsu-Chin Tsao
- University of California, Los Angeles
- Paper IEEExplore
[6] A Computationally Efficient Moving Horizon Estimator for Ultra-Wideband Localization on Small Quadrotors Sven Pfeiffer, Christophe de Wagter and Guido C.H.E. de Croon
- MAVlab, Delft University of Technology
- Paper IEEExplore
[7] A Scalable Distributed Collision Avoidance Scheme for Multi-agent UAV systems Bjorn Lindqvist, Pantelis Sopasakis and George Nikolakopoulos
- Robotics and AI team, Luleå University of Technology
- Video
- ArXiv Preprint
[8] Sniffy Bug: A Fully Autonomous Swarm of Gas-Seeking Nano Quadcopters in Cluttered Environments
- Bardienus P. Duisterhof Shushuai Li Javier Burgues, Vijay Janapa Reddi and Guido C.H.E. de Croon
- MAVlab, Delft University of Technology
- Video playlist
- ArXiv Preprint
[9] micROS.BT: An Event-Driven Behavior Tree Framework for Swarm Robots Yunlong Wu, Jinghua Li, Huadong Dai, Xiaodong Yi, Yanzhen Wang and Xuejun Yang
- Artificial Intelligence Research Center, National Innovation Institute of Defense Technology, Beijing
[10] Neural Tree Expansion for Multi-Robot Planning in Non-Cooperative Environments Benjamin Riviere, Wolfgang Honig, Matthew Anderson and Soon-Jo Chung
- Aerospace laboratories, Caltech
- Video
- ArXiv Preprint

Update 21-10-6

[11] Trust your supervisor: quadrotor obstacle avoidance using controlled invariant sets Luigi Pannocchi, Tzanis Anevlavis, Paulo Tabuada
- University of California, Los Angeles
[12] Continuous-time Gaussian Process Trajectory Generation for Multi-robot Formation via Probabilistic Inference Shuang Guo , Bo Liu , Shen Zhan , Jifeng Guo and Changhong Wang
- Harbin institute of Technology
[13] Non-Prehensile Manipulation of Cuboid Objects Using a Catenary Robot Gustavo A. Cardona , Diego S. D’Antonio , Cristian-Ioan Vasile and David Saldana
- Lehigh University
- Video

Our new intern: Rik

2021-09-27 | Rik | 2 Comments

Hello everyone, my name is Rik and I’m Bitcraze’s new intern. I took up this internship as part of my MSc studies at the faculty of Aerospace Engineering at the Delft University of Technology. Over the past year I have worked with the AI-deck as part of my thesis work at the MAVLab, and I’m looking to contribute to Bitcraze with my practical knowledge of the platform.

For my MSc thesis I have primarily worked on the design of very small optical flow CNNs, capable of running on the GAP8 SoC, using a neural architecture search (NAS). I have implemented a pipeline on the AI-deck, feeding a stream of camera images into the CNN to determine optical flow on-board. One of the last remaining goals of my thesis work is to design an application which utilizes the resulting dense optical flow. In the meantime, the NAS is ever running to find the best possible architecture.

With my practical knowledge I hope to contribute to making the AI-deck an easier platform to work with. Of course, working at Bitcraze is a great learning opportunity, too. I’m already learning a lot about embedded systems and programming. After several years of studying, it’s great to get my first relevant working experience. And maybe most important of all, so far it has been a lot of fun working at Bitcraze and I expect to have a lot more of it. And yes, the falafel is really good.

Do AI decks dream of tutorials?

2021-09-20 | Kimberly McGuire | Leave a comment

The AI-decks are back in stock! Also, last week we had our quarterly meeting, where we plan our focus for the next quarter. As it is also the start of the fiscal year, we also take this opportunity to update our 1 year and 3 year plans as well. We have a big plans coming up, but one of the important focuses that we will have this year, is to get the AI-deck out of early-access!

But what would be necessary for such a task? The AI-deck is one of the most complicated boards we have worked with, so do we have to evaluate its ‘out-of-readiness’ along the same standards than any of our other products.

Mega AIdeck Tutorial

So one of our idea is to be able to achieve a state of the AIdeck in order to write a mega AIdeck tutorial series. This implies that we are able to show how somebody could go from a datasets all the way to a flying aideck-crazyflie combo. Such a series could consist of the following topics:

How to go from a dataset of images to a (Deep) Neural Network
Testing the DNN on the computer with the Image WiFi examples
Converting the neural network to Tensorflow Light (with basics on Edge AI and quantizing neural networks)
AIdeck basics (How to access the camera, how to add the network to the cluster, how to send commands)
Build and flashing the AIdeck and testing it out in the hand
Attach the AIdeck to the crazyflie, make an app-layer application to fly and react on the image input.

From the first look of it, this sounds like it should be easy to do right? Actually, there are still much to be done in order to make this tutorial possible.

Replumbing the Communication

One of the more challenging aspects of the AIdeck as it now is, is that users need to buy a JTAG-enabled programmer in order to flash the GAP8 and the NINA module of the AI-deck. That is the reason why currently the AIdeck has these 2 x 10 pin jtag connectors attached, but ideally we would want to get rid of it completely. This means is that we need to have over air flashing of the GAP8’s binary and that the intercommunication of the NINA and AIdeck will become even more important.

Moreover, the communication protocol from the GAP8 to the STM32 of the Crazyflie is currently very basic, as of right now, it is only possible to send single characters. It might work in some situations, but what if you would like to send an array of values back to the Crazyflie, like the collision probability & steering angle like in PULP platform’s implementation of Dronet? And, would we like to keep on using two UART serial ports or perhaps just relay both NINA and GAP8 communication all through one? The later will make it easier for us to maintain the crazyflie-aideck communication but can perhaps introduce communication delays.

These are just a slice of the type of re-plumbing work for the AIdeck before we can even start our dream tutorial series, but at least it will give you an idea of what we are dealing with :)

Rik the Intern

From this week we have the honor of hosting Rik Bouwmeester for a couple of months. He is currently doing his Master Thesis at the MAVlab from the faculty of Aerospace Engineering of the TU Delft. Since he has experience of working the AIdeck before, he will be able to provide us with some user perspective and help us with the above mentioned issues. You can expect a blogpost from him soon!