Before the summer vacations, I had the opportunity to spend some time working on AI deck improvements (blog post). One of the goals I set was to get CRTP over WiFi working, and try to fix issues along the way. The idea was to put together a small example where you could fly the Crazyflie using the keyboard and see the streamed image along the way. This would require both CRTP to the Crazyflie (logging and commands) as well as CPX to the GAP8 for the images. Just before heading off to vacation I managed to get the demo working, this post is about the results and som of the things that changed.
Link drivers
When using the Crazyflie Python library you connect to a Crazyflie using a URI. The first part of the URI (i.e radio or usb) selects what link driver to use for the connection. For example radio://0/80/2M/E7E7E7E7E7 selects the radio link driver, USB dongle 0 and communication at 2Mbit on channel E7E7E7E7E7.
While working on this demo there were two major things changed in the link drivers. The first one was the implementation of the serial link (serial://) which is now using CPX for CRTP to the Crazyflie. The usecase for this link driver is to connect a Raspberry Pi via a serial port to the Crazyflie on a larger platform.
The second change was to add a new link driver for connecting to the Crazyflie via TCP. Using this link driver it’s possible to connect to the Crazyflie via the network. It’s also possible to get the underlying protocol, the CPX object, for using CPX directly. This is used for communicating with for example the GAP8 to get images.
In the new TCP link driver the URI starts with tcp:// and has either an IP or a host name, followed by the port. Here’s two examples:
tcp://aideck-AABBCCDD.local:5000
tcp://196.168.0.100:5000
Comparison with the Crazyradio PA
So can WiFi be used now instead of the Crazyradio PA? Well, it depends. Using WiFi will give you larger throughput but you will trade this for latency. In our tests the latency is both larger and very random. In the demo I fly with the Flow V2 deck, which means latency isn’t that much of an issue. But if you were to fly without positioning and just use a joystick, this would not work out.
The Demo!
Below is a video of some flying at our office, to try it out yourself have a look at the example code here. Although the demo was mostly intended for improving CPX, we’ve made use of it at the office to collect training data for the AI deck.
The Crazyflie with AIdeck during over WiFI controlled flight.
Improvements
Unfortunately I was a bit short on time and the changes for mDNS discovery never made it it. Because of this there’s no way to “scan” or discover AI decks, so to connect you will need to know the IP or the host name. For now you can retrieve that by connecting to your AI-deck equipped Crazyflie with the CFclient and look at the console tab.
A part from that there’s more improvements to be made, with a better structure for using CPX (more like the CRTP stack with functions) in the library and more examples. There’s also still a few bugs to iron out, for example there’s still the improved FPS and WiFi throughput issues.
IMAV 2022
Next week from 13th to 16th of September Barbara, Kristoffer and Kimberly will be present at the international Micro Aerial Vehicle Conference and Competition (IMAV) hosted by the MAVlab of the TU Delft in the Netherlands. One of the competitions is called the nano quadcopter challenge, where teams will program a Crazyflie + AI deck combo to navigate through an obstacle field, so we are excited to see what solutions will come out of that. If any of you happens to be at the conference/competition, drop by our table to say hello!
you had probably seen me from the last blog post when I first arrived. I spent this summer working here in Malmö and I can definitely say that it was one great, educative and fun experience. During the last three months I have been in Bitcraze, I was given the chance to work and develop applications and demos on the robotics subject I am most interested in, drone collaboration.
Centralized Swarm with Multiple Flying Copter
I initially started looking into the implemented swarm demo which had 7 drones charging wirelessly in 7 charging decks and one of them flying by executing a spiral trajectory until it has a low battery and another one takes its place. The original swarm demo was shown at several conferences before the pandemic hit, but my plan was to improve it by adding more quadcopters flying simultaneously. The biggest problem was the collisions and ground effect happening between them. In order to solve that I was based on this paper and the optimization engine OpEn. I solved the problem of all drones starting from a point and going to a final one without colliding and covering the minimum distance by transferring these constraints into a cost function of an optimization problem assuming a simplified model for the quadcopter. Its output is waypoints for each quadcopter to pass from. These waypoints are transformed into a trajectory(piecewise polynomial) by a custom trajectory generator based on linear algebra.
In this way, I made it possible to execute non-colliding trajectories for 4 quadcopters, upload and execute them. While executing the first trajectory, the next one was being calculated and uploaded assuming the goal of the previous one as starting point. In this way, I managed to have 4 Crazyflies flying simultaneously and landing when their battery was running out and the fully charged ones were taking their place. This mechanism with some modifications can be used as a path planner or a standalone trajectory generator from a future user by feeding it waypoints and time durations for each waypoint segment. You can find the source code here.
Decentralized Swarm with Multiple Flying Copters
The aforementioned setup seemed to work pretty well but there was always the need of having a central PC monitoring and taking decisions for the whole swarm. So we wanted to move the architecture to a decentralized one, of which Kristoffer did some preliminary work shown at BAMdays last year. This was made possible by utilizing the onboard peer-to-peer protocol (P2P) in collaboration with the onboard collision avoidance algorithm introduced in this PR contributed by James Preiss from the University of Southern California. All the Crazyflies share their position and state through the protocol by broadcasting them at a rate of 15 Hz.
Although there were some missed packets, they could avoid each other while flying by updating the collision avoidance algorithm which is taking action between the high level and the action commander by altering its waypoints. The decisions of which drone should take off or land are also taken in a decentralized way. Whenever one copter is about to take off it enters the corresponding state and assigns itself a randomized timeout. During this timeout, if the desired number of airborne copters is achieved it goes back to idle. If not and the timeout occurs it finally take-offs. So, despite there is not an actual common decision, the swarm can be led to simple desired states like keeping the number of the drones flying constant and executing changes between the landed, charged copters and flying ones. You can find the source code here.
Token Ring Implementation
After I finished this project and since I had some extra time left I decided to work more in the P2P protocol. The need for having a robust way to communicate between the Crazyflies and a way to verify that a packet was indeed sent was obvious. A solution to this problem was offered by Christos Zosimidis and Klaus Kefferpütz from the Cooperative Control Lab in Hochschule Augsburg, namely a token ring implementation. I would really like to thank them for this collaboration and hope for future ones as well.
Specifically, the proposed token ring protocol was implemented in a modified version of the nrf-radio firmware and the Crazyradio. This protocol assumes that each Crazyflie is a node of a network and a token is passed around giving permission to each drone that has it in its possession to broadcast data. So, each time only one Crazyflie broadcasts data which leads to fewer packet collisions and losses. It can also acknowledge that a node has received the data targeting to it and then continues to others. The interface with the protocol is being done by 2 queues (TX and RX) where the user can place data that wants to send and read the RX queue to receive. The moment that this blog post is being written only the static version of it is public in the firmware, which means that the number and the id of the Crazyflies must be defined before execution and in case a copter fails the whole network fails. Although, I am currently working on a dynamic approach that is going to solve these problems
All in all, I had a great time here in Malmo despite the fact that the Swedish summer is much colder than the warm weather I was used to in Greece. I was amazed by the way things in Bitcraze work and how the whole company operates. It was a pleasure being around so creative people and I am happy that I could help even in a small way. Thank you very much for giving me the opportunity to work with you and I hope I will keep on contributing to this amazing project in the future.
After a period of bitcrazer-vacations, we are now all back at work. The summer here in Sweden has generally been great. Some of us stayed here to keep the company afloat, and some just stayed afloat on lakes or the sea. The majority vacationed inside of Sweden, but some (could you guess who?) have visited France, Italy, or Greece. We’ve been lucky with a mostly warm and sunny weather, perfect for bathing and grilling. And even though it’s nice to enjoy real summer, it’s still worrying sign though, as Europe is experiencing what could be the worst drought in 500 years.
Crazyflie 2.1 back in stock
What is also back is the Crazyflie 2.1, but back in stock, yay! After almost two weeks without any drones available for sale, we received a new batch of our quadcopter today. It should now be available in the shop, just in time for when school starts!
We got some indications the component shortage are slowly moving in the right direction so hopefully it will get easier to keep things in stock in the future. We are keeping our fingers crossed.
Bolt 1.1 ESC cable red/black switched
Unfortunately we recently found out that there has been a manufacturing error with the ESC cables that come with the Bolt 1.1. The black and red cables have been switched. Please see the image below.
With the black and red cables switched this will result in powering your ESCs with reversed polarity. This will most likely burn the MOSFET on the Bolt that controls the power to the ESC, which is the weakest link. This because the MOSFET body diodes on the ESC will conduct and make the whole ESC a short circuit. In many setups, e.g using 4in1 ESC these cables are not used though and will not cause a problem.
Switching the cables back is quite easy to do. Use a needle, tweezer or e.g. small screwdriver to open the plastic lock so the cable can be pulled out. Switch the black and red and you are done. You can double check that the colors are correct by comparing it with the Bolt 1.1 board. The plus and minus should match with the red and black as per the image below:
We are currently working with the manufacturer to get correct cables. If you got a Bolt 1.1 (anytime between June and August 2022) we can of course ship you correct cables once they are ready or give you support if you got problems with the control board. If so, please send us an email to support@bitcraze.io. Sorry for this inconvenience!
There are some nice and exciting improvement in the CF-client that we worked on during the summer months! First of all we worked on a toolbox structure, where every tab can be reconfigured as a toolbox as well, allowing it to be docked to the sides of the window. Secondly we have added a new geometry estimation wizard for Lighthouse systems to support multi base-station estimation. Finally we have added a new tab for PID controller tuning, mainly intended for the Bolt.
New tabs, toolboxes and wizards for the CFclient
Toolboxes in the CFclient
Everyone who used the CFclient has experienced the tabs before. Anytime you want to configure the lighthouse system, setup plotting or look at the parameter states, you switch to the appropriate tab to perform your desired action. This is all fine, but sometimes it can be useful to see the contents of two tabs at the same time, maybe you want to watch the graphing of a log variable at the same time as you change a parameter. This is what the combined tab/toolbox feature adds! Any tab can now be converted into a toolbox that can be docked to the side of the window.
Plotter tab with parameter toolbox
In the example above the plotter is displaying the estimated position of a Crazyflie with a Flow deck, while the parameter window is opened as a toolbox. The “motion.disable” parameter was just set to true and we can see that the kalman estimator gets into trouble when it no longer gets data from the flow deck.
To switch from tab to toolbox mode, go to the View/Toolboxes menu and select the window that you want to show as a toolbox. In a similar way, use the View/Tabs menu to turn it back to a tab.
Even though all tabs can be turned into toolboxes, some of them might still look better as tabs due to their design. We hope to be able to improve the design over time and make them more toolbox friendly, contributions are welcome!
Lighthouse Geometry Estimation Wizard
In a blogpost of almost a half year ago, we presented a new multi base station geometry estimation method that enabled the user to include more than 2 base station for flying a Crazyflie. This heavily increases the flight area covered by the base station V2s, as technically it should be able to handle up to 16!
New geometry estimation dialog
However, up until this summer it has been in experimental mode as we weren’t so sure as how stable this new estimation method is, so the only way to use it was via a script in the Crazyflie python library directly, and not from the CFclient. Since we haven’t heard of anybody having problems with this new experimental feature, we decided to go ahead to make a nice multi base station geometry estimation wizard in the CFclient’s Lighthouse tab.
This wizard can be accessed if you go to the lighthouse tab-> ‘manage geometry’ and press ‘Estimate Geometry’. We had to make it a wizard as this new method requires some extra intermediate steps compared to the previous, to ensure proper scaling, ground plane setting and sweep angle recording. If you are only using 2 base stations this seems like extra effort, where you only had to put the Crazyflie on the ground and push a button, but if you compare flight performance of the two methods, you will see an immediate difference in positioning quality, especially around the edges of your flight area. So it is definitely worth it!
First page of the wizard
We will still provide the “simple” option for those that want to use it, or want to geometry estimate only one base-station, as we don’t have support for that for this new estimator (see this issue). In that case, you will have to install the headless version of opencv separately like ‘pip3 install opencv-python-headless’. We will remove this requirement from the cflib itself for the next release as there are conflicts for users who has installed the non-headless opencv on their system, like for the opencv-viewer of the AI-deck’s wifi streamer for instance.
PID tuning tab
The PID controller tuning tab
And last but not least, we introduced an PID tuning tab in a PR in the CFClient! And of course… also available as a toolbox :) This is maybe not super necessary for the Crazyflie itself, but for anyone working with a custom frame with the Bolt or BigQuad deck this is quite useful. Tuning is much handier with a slider than to adjust each parameter numerically with the parameter tab. Also if you are just interested of what would happen if you would increase the proportional gain of the z-position controller of the crazyflie, this would be fun to try as well… but of course at your own risk!
If you are happy with your tuned PID values, there is the “Persist Values” button which will store the parameters in the EEPROM memory of the Crazyflie/Bolt, which means that these values will persist even after restarting the platform. This can be cleared with the ‘Clear persisted values’ button and you can retrieve the original firmware-hardcoded default values with ‘Default Values’ button. Please check out this blogpost to learn more about persistent parameters.
Try it out for yourself!
This client has not been released yet but you can already go ahead and try these new features out for yourself. Make sure to first install the client from source, and then install the CFlib from source, as an update of both is necessary. Also update the crazyflie-firmware to the latest development branch via these instructions, especially if you want to try out the new LH geometry wizard.
And of course, don’t forget to give us feedback on discussions.bitcraze.io or to make an issue on the cfclient, cflib or crazyflie-firmware github repositories if you are hitting a bug on your machine and you know pretty precisely where it comes from.
Unmanned Aerial Vehicles (UAVs) have garnered much attention from both researchers and engineers in recent decades. Aerial robots in general are classified into mainly three categories: fixed wings, rotary wings and flapping wings.
Fixed wings are one of the most common aerial vehicles as it has relatively higher power efficiency and payload capacity than other types, thanks to their big and highly customizable wing. But this also leads to a bigger footprint and usually the lack of ability for Vertical Taking Off and Landing (VTOL). Rotary wings generally include helicopter and multirotors (such as quadrotors), and they have recently become increasingly popular in our daily lives. Easily achieving great performance in attitude and position control, rotary wings are widely applied in many fields. Flapping wing robots take inspirations from small flapping insects (such as Harvard Robobee) or birds (Purdue Hummingbird Robot).
Fig: A simple prototype of SAM from SUTD with Crazyflie Bolt.
Monocopters are largely inspired from the falling motion of maple seeds, and they are relatively much simpler to build as compared to its counterparts. They can keep a relative smaller footprint and achieve decent control performance although they are highly underactuated. The Single Actuator Monocopter (SAM) has the ability to VTOL, perform 3D trajectory tracking as well as maintain high hovering efficiency. With those advantages, rapid developments have been made in recent years such as the Foldable Single Actuator Monocopter (F-SAM) and Modular Single Actuator Monocopter (M-SAM) from Engineering Product Development (EPD) of Singapore University of Technology and Design (SUTD).
Taking inspiration from nature – Samara inspired monocopter
A descending samara or maple seed, is able to passively enter auto-rotation motion and stabilize its flight attitude, helping to slow down its descent speed and travel further for better survival of the species. This natural behavior attracts interests from scientists and researchers. With previous studies, we learnt that this passive attitude stability is mainly guaranteed by mass distribution (Center of Mass) and wing geometry (Center of Pressure) as well as the rotation motion.
A maple seed inspired Single Actuator Monocopter (SAM).
The SAM is designed to be very close in its mechanical make-up to its natural sibling, having a large single wing structure and a smaller, denser ‘seed’ structure. A single motor with propeller is installed on the leading edge, parallel to the wing surface. Comparing with flight dynamics of the original maple seed, SAM has extra torques and force caused by the spinning propeller, including a reaction torque and thrust directly from propeller, as well as an extra torque caused by precession motion. As a result, the balance of the combined forces and torques allows SAM to enter a new equilibrium condition while still retaining the passive attitude stability.
Development of monocopters
The research on monocopters can be traced back to a long time ago. Here are some examples of different types of air frame to roughly introduce their developments. An air-frame called Robotic Samara [1] was created in 2010, which has a motor to provide rotational force, a servo to control collective pitch of the wing, a winged body fabricated by carbon fiber, and a lipo battery. In the following year, Samarai MAV [2] was developed by following the mass distribution of a natural maple seed. To achieve the control, a servo is equipped to regulate the wing flap. In 2020, a single actuator monocopter was introduced with a simplified air-frame [3]. The main structure is made by laminated balsa wood while the trailing edge of the wing is made by foam for better mass distribution. By making use of the passive attitude stability, only one actuator is required to control the position in 3D space. Based on which, F-SAM [4] and M-SAM [5] were developed in 2021 and 2022 respectively.
SAM with foldable wing structure (F-SAM).
A Modular SAM (M-SAM) with Crazyflie Bolt
Thanks to its easy implementation and reliable performance, we use the Crazyflie Bolt as the flight controller for M-SAM. Like other robotic systems, the ground station is integrated with motion capture system (position and attitude feedback for both control and ground truth) and a joystick (control reference directly generated by user) is responsible for sending filtered state feedbacks and control references or control signal directly to flight controller. This is realized by employing the Crazyradio PA under the Crazyflie-lib-python environment. Simple modifications from the original firmware were made to map from the control reference to motor command (a customized flight controller).
A diagram shows how Crazyflie Bolts work in M-SAM project.
Another advantage of using Crazyflie Bolt in M-SAM project is its open source swarm library. Under the swarm environment, SAMs can fly in both singular and cooperative configurations. With simple human assistance, two SAMs can be assembled into cooperative configuration by making use of a pair of magnetic connectors. The mid-air separation from cooperative configuration to singular configuration is passively triggered by increasing the rotating speed until the centrifugal force overcomes the magnetic force.
Modular Single Actuator Monocopters (M-SAM), which is able to fly in both singular and cooperative configuration.
Potential applications
What kinds of applications can be achieved with the monocopter aerial robotic platform? On the one hand, many applications are limited by the nature of self-rotation motion. On the other hand, the passive rotating body also offers advantages in some special scenarios. For example, SAM is an ideal platform for LIDAR application, which usually requires the rotating motion to sense the environment around. Besides, thanks to simple mechanical design and cheap manufacturing cost, SAM can be designed for one time use such as light weight air deployment or unknown, dangerous environments.
An example [6] shows the potential applications of a rotating robot with camera.
Reference
[1] Ulrich, Evan R., Darryll J. Pines, and J. Sean Humbert. “From falling to flying: the path to powered flight of a robotic samara nano air vehicle.” Bioinspiration & biomimetics 5, no. 4 (2010): 045009.
[2] Fregene, Kingsley, David Sharp, Cortney Bolden, Jennifer King, Craig Stoneking, and Steve Jameson. “Autonomous guidance and control of a biomimetic single-wing MAV.” In AUVSI Unmanned Systems Conference, pp. 1-12. Arlington, VA: Assoc. for Unmanned Vehicle Systems International, 2011.
[3] Win, Luke Soe Thura, Shane Kyi Hla Win, Danial Sufiyan, Gim Song Soh, and Shaohui Foong. “Achieving efficient controlled flight with a single actuator.” In 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), pp. 1625-1631. IEEE, 2020.
[4] Win, Shane Kyi Hla, Luke Soe Thura Win, Danial Sufiyan, and Shaohui Foong. “Design and control of the first foldable single-actuator rotary wing micro aerial vehicle.” Bioinspiration & Biomimetics 16, no. 6 (2021): 066019.
[5] X. Cai, S. K. H. Win, L. S. T. Win, D. Sufiyan and S. Foong, “Cooperative Modular Single Actuator Monocopters Capable of Controlled Passive Separation,” 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 1989-1995, doi: 10.1109/ICRA46639.2022.9812182.
[6] Bai, Songnan, Qingning He, and Pakpong Chirarattananon. “A bioinspired revolving-wing drone with passive attitude stability and efficient hovering flight.” Science Robotics 7, no. 66 (2022): eabg5913.
Now it is time to give a little update about the ongoing ROS2 related projects. About a month ago we gave you an heads-up about the Summer ROS2 project I was working on, and even though the end goal hasn’t been reached yet, enough has happened in the mean time to write a blogpost about it!
Crazyflie Navigation
Last time showed mostly mapping of a single room, so currently I’m trying to map a bigger portion of the office. This was initially more difficult then initially anticipated, since it worked quite well in simulation, but in real life the multi-ranger deck saw obstacles that weren’t there. Later we found out that was due to this year old issue of the multi-ranger’s driver incapability to handle out-of-range measurements properly (see this ongoing PR). With that, larger scale mapping starts to become possible, which you can see here with the simple mapper node:
If you look at the video until the end, you can notice that the map starts to diverge a bit since the position + orientation is solely based on the flow deck and gyroscopes , which is a big reason to get the SLAM toolbox to work with the multi-ranger. However, it is difficult to combine it with such a sparse ‘Lidar’ , so while that still requires some tuning, I’ve taken this opportunity to see how far I get with the non-slam mapping and the NAV2 package!
As you see from the video, the Crazyflie until the second hallway. Afterwards it was commanded to fly back based on a NAV2 waypoint in RViz2. In the beginning it seemed to do quite well, but around the door of the last room, the Crazyflie got into a bit of trouble. The doorway entrance is already as small as it is, and around that moment is also when the mapping started to diverge, the new map covered the old map, blocking the original pathway back into the room. But still, it came pretty close!
The diverging of the map is currently the blocker for larger office navigation, so it would be nice to get some better localization to work so that the map is not constantly changed due to the divergence of position estimates, but I’m pretty hopeful I’ll be able to figure that out in the next few weeks.
Crazyflie ROS2 node with CrazySwarm2
Based on the poll we set out in the last blogpost, it seemed that many of you were mostly positive for work towards a ROS2 node for the Crazyflie! As some of you know, the Crazyswarm project, that many of you already use for your research, is currently being ported to ROS2 with efforts of Wolfgang Hönig’s IMRCLab with the Crazyswarm2 project. Instead of in parallel creating separate ROS2 nodes and just to add to the confusion for the community, we have decided with Wolfgang to place all of the ROS2 related development into Crazyswarm2. The name of the project will be the same out of historical reasons, but since this is meant to be the standard Crazyflie ROS2 package, the names of each nodes will be more generic upon official release in the future.
To this end, we’ve pushed a cflib python version of the crazyflie ros2 node called crazyflie_server_py, a bit based on my hackish efforts of the crazyflie_ros2_experimental version, such that the users will have a choice of which communication backend to use for the Crazyflie. For now the node simply creates services for each individual Crazyflie and the entire swarm for take_off, land and go_to commands. Next up are logging and parameter handling, positioning support and broadcasting implementation for the CFlib, so please keep an eye on this ticket to see the process.
So hopefully, once the summer project has been completed, I can start porting the navigation capabilities into the the Crazyswarm2 repository with a nice tutorial :)
ROScon talk
As mentioned in a previous blogpost, we’ll actually be talking about the Crazyflie ROS2 efforts at ROScon 2022 in Kyoto in collaboration with Wolfgang. You can find the talk here in the ROScon program, so hopefully I’ll see you at the talk or the week after at IROS!
Advancements in technology have made quadrotor drones more accessible and easy to integrate into a wide variety of applications. Compared to traditional fixed-wing aircraft, quadrotors are more flexible to design and more suitable for motioning, such as statically hovering. Some examples of quadrotor applications include photographers using mounting cameras to take bird’s eye view images, and delivery companies using them to deliver packages. However, while being more versatile than other aerial platforms, quadrotors are still limited in their capability due to many factors.
First, quadrotors are limited by their lift capacity, i.e., strength. For example, a Crazyflie 2.1 is able to fly and carry a light payload such as an AI deck, but it is unable to carry a GoPro camera. A lifter quadrotor that is equipped with more powerful components can transport heavier payload but also consumes more energy and requires additional free space to operate. The difference in the strength of individual quadrotors creates a dilemma in choosing which drone components are better suited for a task.
Second, a traditional quadrotor’s motion in translation is coupled with its roll and pitch. Let’s take a closer look at Crazyflie 2.1, which utilizes a traditional quadrotor design. Its four motors are oriented in the same direction – along the positive z-axis of the drone frame, which makes it impossible to move horizontally without tilting. While such control policies that convert the desired motion direction into tilting angles are well studied, proven to work, and implemented on a variety of platforms [1][2], if, for instance, we want to stack a glass filled with milk on top of a quadrotor and send it from the kitchen to the bedroom, we should still expect milk stains on the floor. This lack of independent control for rotation and translation is another primary reason why multi-rotor drones lack versatility.
These versatility problems are caused by the hardware of a multi-rotor drone designed specifically to deal with a certain set of tasks. If we push the boundary of these preset tasks, the requirements on the strength and controllability of the multi-rotor drone will eventually be impossible to satisfy. However, there is one inspiration we take from nature to improve the versatility in the strength of multi-rotor drones – modularity! Ants are weak individual insects that are not versatile enough to deal with complex tasks. However, when a group of ants needs to cross natural boundaries, they will swarm together to build capable structures like bridges and boats. In our previous work, ModQuad [3], we created modules that can fly by themselves and lift light payloads. As more ModQuad modules assemble together into larger structures, they can provide an increasing amount of lift force. The system shows that we can combine weak modules with improving the versatility of the structure’s carrying weight. To carry a small payload like a pin-hole camera, a single module is able to accomplish the task. If we want to lift a heavier object, we only need to assemble multiple modules together up to the required lift.
Improving controllability
On a traditional quadrotor, each propeller is oriented vertically. This means the device is unable to generate force in the horizontal direction. By attaching modules side by side in a ModQuad structure, we are aligning more rotors in parallel, which still does not contribute to the horizontal force the structure can generate. That is how we came up with the idea of H-ModQuad — we would like to have a versatile multi-rotor drone that is able to move in an arbitrary direction at an arbitrary attitude. By tilting the rotors of quadrotor modules and docking different types of modules together, we obtain a structure whose rotors are not pointing in the same direction, some of which are able to generate a force along the horizontal direction.
H-ModQuad Design
H-ModQuad has two major characteristics: modularity and heterogeneity, which can be indicated by the “Mod” and “H-” in the name. Modularity means that the vehicle (we call a structure) is composed of multiple smaller modules which are able to fly by themselves. Heterogeneity means that we can have modules of different types in a structure.
As mentioned before, insects like ants utilize modularity to enhance the group’s versatility. Aside from a large number of individuals in a swarm that can adapt to the different scales of the task requirement, the individuals in a colony specializing in different tasks are of different types, such as the queen, the female workers, and the males. The differentiation of the types in a hive helps the group adapt to tasks of different physical properties. We take this inspiration to develop two types of modules.
In our related papers [4][5], we introduced two types of modules which are R-modules and T-modules.
Fig 2. Major components of an H-ModQuad “T-module” we are using in our project. We use Bitcraze Crazyflie Bolt as the central control board.
An example T-module is shown in the figure above. As shown in the image, the rotors in a T-module are tilted around its arm connected with the central board. Each pair of diagonal rotors are tilted in the opposite direction, and each pair of adjacent rotors are either tilting in the same direction or in the opposite direction. We arrange the tilting of the rotors so that all the propellers generate the same thrust force, making the structure torque-balanced. The advantage of the T-module is that it allows the generation of more torque around the vertical axis. One single module can also generate forces in all horizontal directions.
An R-module has all its propellers oriented in the same direction that is not on the z-axis of the module. In this configuration, when assembling multiple modules together, rotors from different modules will point in different directions in the overall structure. The picture below shows a fully-actuated structure composed of R-modules. The advantage of R-modules is that the rotor thrusts inside a module are all in the same direction, which is more efficient when hovering.
Structure 1: Composed of four types of R-modules.
Depending on what types of modules we choose and how we arrange those modules, the assembled structure can obtain different actuation capabilities. Structure 1 is composed of four R-modules, which is able to translate in horizontal directions efficiently without tilting. The picture in the intro shows a structure composed of four T-modules of two types. It can hover while maintaining a tilting angle of up to 40 degrees.
Control and implementation
We implemented our new geometric controller for H-ModQuad structures based on Crazyflie Firmware on Crazyflie Bolt control boards. Specifically, aside from tuning the PID parameters, we have to change the power_distribution.c and controller_mellinger.c so that the code conforms to the structure model. In addition, we create a new module that embeds the desired states along predefined trajectories in the firmware. When we send a timestamp to a selected trajectory, the module retrieves and then sends the full desired state to the Mellinger Controller to process. All modifications we make on the firmware so that the drone works the way we want can be found at our github repository. We also recommend using the modified crazyflie_ros to establish communication between the base station and the drone.
Videos
Challenges and Conclusion
Different from the original Crazyflie 2.x, Bolt allows the usage of brushless motors, which are much more powerful. We had to design a frame using carbon fiber rods and 3-D printed connecting parts so that the chassis is sturdy enough to hold the control board, the ESC, and the motors. It takes some time to find the sweet spot of the combination of the motor model, propeller size, batteries, and so on. Communicating with four modules at the same time is also causing some problems for us. The now-archived ROS library, crazyflie_ros, sometimes loses random packages when working with multiple Crazyflie drones, leading to the stuttering behavior of the structure in flight. That is one of the reasons why we decided to migrate our code base to the new Crazyswarm library instead. The success of our design, implementation, and experiments with the H-ModQuads is proof of work that we are indeed able to use modularity to improve the versatility of multi-rotor flying vehicles. For the next step, we are planning to integrate tool modules into the H-ModQuads to show how we can further increase the versatility of the drones such that they can deal with real-world tasks.
Reference
[1] D. Mellinger and V. Kumar, “Minimum snap trajectory generation and control for quadrotors,” in 2011 IEEE International Conference on Robotics and Automation, 2011, pp. 2520–2525.
[2] T. Lee, M. Leok, and N. H. McClamroch, “Geometric tracking control of a quadrotor uav on se(3),” in 49th IEEE Conference on Decision and Control (CDC), 2010, pp. 5420–5425.
[3] D. Saldaña, B. Gabrich, G. Li, M. Yim and V. Kumar, “ModQuad: The Flying Modular Structure that Self-Assembles in Midair,” 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 691-698, doi: 10.1109/ICRA.2018.8461014.
[4] J. Xu, D. S. D’Antonio, and D. Saldaña, “Modular multi-rotors: From quadrotors to fully-actuated aerial vehicles,” arXiv preprint arXiv:2202.00788, 2022.
[5] J. Xu, D. S. D’Antonio and D. Saldaña, “H-ModQuad: Modular Multi-Rotors with 4, 5, and 6 Controllable DOF,” 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 190-196, doi: 10.1109/ICRA48506.2021.9561016.
In the first years that I started at Bitcraze I’ve been focused mostly on embedded development and algorithmic design like the app layer, controllers and estimators and such, however recently I started to be quite interested in the robotic integration between the Crazyflie and other (open-source) projects and users. This means that I’ll be dwelling more often in the space between Bitcraze and the community, which is something that I do really enjoy I noticed during the Grand Tour. It also initiated my work with simulators which I think would be very useful for the community too. The summer fun project that I’ve been now working on is to integrate the Crazyflie with ROS2 to integrate standard navigational packages, which will be the topic of this blogpost!
ROS2 Crazyflie Node
So first I worked on the ROS2 node that actually communicates with the Crazyflie directly. I think many of you are familiar with the USC’s CrazySwarm project, of which the ROS2 variant, CrazySwarm2, is already available for most functionalities. Even though the name says CrazySwarm, this can be very easily used for only one Crazyflie too. The CrazySwam2 is currently under more development by the IMRClab of TU Berlin, but please take a look if you want to give it a go!
For now while Crazyswarm2 is still under development, I used the Bitcraze Crazyflie python library to make a more hackish node that just publishes exactly the information I want. I am focusing on the scenario with the STEM ranging bundle, aka the Crazyflie + Flowdeck (optical flow + distance sensor) + Multi-ranger (5 x distance sensors) combo, where the node logs the multi-ranger data and the odometry from the Flowdeck with the Crazyradio and outputs that into necessary /scan and /odom topics. Moreover, it also outputs several tf2 transforms that makes it possible to either visualize it in RVIZ and/or connect it to any other packages and it should react to incoming twist messages as well.
Development with a Simulator
And of course… I went in head first and connected it directly with the SLAM toolbox. I have worked with ROS1 in the past, but I had my first experience working with that package in the course: Build Mobile Robots with ROS2 (by Weekly Robotic Newsletter’s Mat Sadowski), so I couldn’t wait to try it on a real platform like the Crazyflie. However, tuning this was of course more work than I thought, as the map that I got out of it first was mostly a sparse collection of dots. Of course the SLAM toolbox is meant for lidars and not something that provided sparse range distances like the Multiranger. Then I decided to take one or two steps back, and first connect a simulator to make tuning a bit easier.
Luckily, I’ve already started to look at simulators, and was quite far in the Webots integration of the Crazyflie. Actually… Webots’ next release (2022b) will contain a Crazyflie as standard! Once it is out, I’ll write a blogpost about that separately :). As luck has it, Webots also has good ROS2 integration as well, and even won the ‘Best ROS Software’ award by The Construct’s ROS awards! Another reason is that I wanted to try out a different simulator for ROS2 this time to complement what I’ve learned in the ROS2 course I mentioned earlier.
So I used the webots driver node to write a simulated Crazyflie that should output the same information as the real Crazyflie node, so that I can easily hack around and try out different things without constantly disturbing my cats from their slumber :). Anyway, I won’t go into to the simulator too much and save that for another blogpost!
Simple Mapping
I decided to also take another additional step before going full SLAM, which is to make a simple mapper node first! This takes the estimated state estimate of the Crazyflie and the Multiranger’s range values and it creates an occupancy grid type map of it. I do have to give kudos to the Marcus’ cflib Pointcloud script and Webot’s simple mapper example, as I did look at them for some reference. But still with the examples, integration and connecting the dots together is quite some work. Luckily I had the simulator to try things out with!
So first I put the Crazyflie in an apartment simulator, flew around and see if any decent maps comes out of it and it seemed it did! Of course, the simulated Crazyflie’s ‘odometry’ comes from near perfect position estimate, so I didn’t expect any problems there (and in such a situation you would actually not really need the localization part of SLAM). This still needs some improvements to be done, like now range measurements that don’t see anything are excluded from drawing, but still it was pretty cool to map the virtual environment.
So it was off to try it out on a real crazyflie. In one of our meeting rooms, I had one Crazyflie take off, let it turn around with a twist message in a /cmd_vel topic and made a map of the room I was currently in. The effect of the 4 range sensors rotating around and creating a map in one go, makes me think of these retro video transitions. And the odometry drift does not seem as bad for it to be possible, but I haven’t mapped our entire office yet so that might be different!
What’s next?
So I’m not stopping here for sure, I want to extend this functionality further and for sure get it to work with the SLAM_toolbox properly! But if the simple mapper already can produce such quality, I’m pretty sure that this can be done in one way or the other. What I could also do, is first generate a simple map and already have a go at the NAV2 package with that one… there are many roads to Rome here!
Currently I’m doing my work on my personal Github account in the crazyflie_ros2_experimental repository. Everything is still very much in development, hackish and quite specific for one use case but that is expected to change once things are working better, so please check the planning in the project’s readme. In the mean time, you can indicate to us in this vote if this is an interesting direction for us to go towards. Not that it will stop me from continuing this project since it is too much fun, but it is always good to know if certain efforts are appreciated!
Last week we had the first ever Bitcraze DEV meeting! With about 10 participants, we covered a range of topics. The meeting was mostly focused around how to handle support and what the DEV meetings should be about. We also had a chance to get some feedback, and one of the points was sharing a bit more what we’re currently working on and what we might work on in the future. So in the light of that, this blogpost is about CPX (the Crazyflie Packet eXchange) protocol. We’ve mentioned CPX before (1, 2), but with this blogpost I want to share the current status and some thoughts on why we need something new.
As summer is approaching and things are winding down, I’m talking the opportunity to get back to the AI deck and CPX. The AI deck was officially released out of early access last month, but there’s still more work to be done with porting examples, adding some more functionality and increasing stability and performance.
For the AI deck we’re only supplying examples, there’s no functionality that will be used with the platform (except for the WiFi connection maybe). This is in contrast to for instance the Flow deck, where there’s a specified functionality the user can use and that should work. So in order to move forward I came up with a little demo that I want to get working during the summer. The goal is to make an application where I can fly around the Crazyflie with the keyboard and get a video stream back. To achieve this I’m using the Flow deck together with the AI deck and using WiFi for both CPX and CRTP (to send command and to get images and logging).
Why we need something new
I’ve written a post about CPX in the past (link) where I detailed the issues we are trying to solve. But in short we needed was a protocol that …
… could be routed though intermediaries to reach it’s destination
… could handle high transfer rates with large amounts of data as well as small messages
… could handle different memory budgets
… doesn’t drop data along the way if some parts of the system is loaded
As the Crazyflie echo system grows and becomes more complex we need new tools to work with it. When CRTP was implemented many years ago, the complexity we have today wasn’t something we could imagine. The Crazyflie had the only MCU and the hardware on the decks were used directly from it. Now we have multiple decks with more complex systems on them: AI deck (2 MCUs), Active marker deck (1 MCU) and the Lighthouse deck (1 FPGA). Looking forward these more complex decks might increase in the future. With more and more functionality in the Crazyflie and resources occupied, like DMA channels and pins, some functionality might need to move further out onto the decks.
For each deck new protocols are implemented and specific code is needed in the Crazyflie to handle it. Some things also become complex, like getting printouts from the different MCUs on the decks. So for the AI deck we wanted to test something new and more generic to see if it would be something we could use more in the future to talk directly to different MCUs in the system.
Will CPX replace CRTP? Probably not. We’re not sure what solution we will land in, but I think CPX is a good step in the right direction.
Current status
Back to my little demo. To reach the goal there’s a few things which needs to be fixed:
crazyflie-firmware/#1065: When starting to run CRTP over CPX (via WiFi) I’ve noticed that the UART2 driver was too slow, loading the system too heavily and creating problems down the line. So this is being worked on, and at the same time the old SYS-link over UART2 implementation is being moved to CPX instead.
aideck-esp-firmware/#12: We’ve had reports of intermittent performance issues for WiFi, which is also effecting.
Aside from the issues there’s also a few other features that are being added:
CRTP over CPX: Since I already have a connection for the images I also want to use this for controlling the Crazyflie. The latency is too high for controlling roll/pitch/yaw in real-time, but in my case I have the Flow deck for position control
CPX over CRTP: Although not part of the demo, this is interesting to look at for the future. One example is that right now we have an implementation where the Crazyflie firmware has a special implementation for the WiFi credentials. If we would like to set it from the ground we would first have to do CRTP to the Crazyflie, re-package it and then send it via CPX to the ESP32 on the AI deck. Instead I would like to send it via CPX directly from the ground, saving us extra work and complexity in the Crazyflie
Using Zeroconf/mDNS for finding AI decks: With this changes it will be possible to connect to the Crazyflie via the client, so we need a way to find the AI decks. For this Zeroconf/mDNS has been added, so AI decks will be automatically discovered on the local network.
The current status can be seen in the following draft PRs: crazyflie-firmware/#1068 and crazyflie-lib-python/#342. Note that until these are real PRs (not draft) they are not useful, so don’t try to use them yet.
Our Ultra Wide Band (UWB) based positioning system, the Loco Positioning System, has been around for a long time and is still going strong! In this post we will tell you a bit about how it works (for those that don’t know about it yet), what research that is on-going in the field and new developments.
Crazyflie with Loco deck
Basics
UWB is using high frequency, low power, wide band radio where one of the most important properties is that it is possible to detect when a packet is received with very high accuracy. Combining this with very high frequency clocks, opens up the possibility to measure the time it takes for a radio packet to travel from a transmitter to a receiver. Since radio waves propagates with the speed of light in air we can convert the time into distance, and this is the basic idea in UWB positioning.
Not only is it possible to measure the timing of transmissions, the packets can also contain data, like in other radio standards. This property is extensively used to include time stamps of when a packet is sent, and also for instance the time stamp of when the transmitter received other packets or the position of an anchor.
This sounds pretty straight forward, but there are (of course) some complications. We will mention some of them but not go into the details.
Reflections – radio waves bounce around on walls and objects. Luckily, the nature of UWB actually uses this to its advantage and works better indoors than out side.
The clocks in the transmitter and receiver are not synchronized – the Time Of Flight can unfortunately not simply be measured by subtracting reception time from transmission time as the time stamps originate from two different clocks. The problem can be solved by sending some more packets back and forth though.
Packet collisions – two transmitters can not send at the same time, one or both packets will be lost. Transmissions must be scheduled or packet loss must be handled.
Obstacles – obstacles between the transmitter and receiver changes the transmission time.
Antennas – the propagation time through the antenna is substantial and changes depending on the angle to the transmitter/receiver.
Radio interference – other radio sources may interfere with the UWB radio signals and add noise or packet loss.
Modes
The Loco Positioning System can run in two fundamentally different modes: Two Way Ranging (TWR) and Time Difference of Arrival (TDoA).
Two Way Ranging (TWR)
In TWR the Crazyflie measures the distance to one anchor at a time, over and over again. Each measurement in initiated by the Crazyflie and requires 4 messages to be sent between the Crazyflie and the anchor, two request-response pairs. The position is estimated by pushing the measured distances into the kalman estimator.
This mode only supports one Crazyflie, but has the advantage of being very robust and also works pretty well some distance outside the system.
Time Difference of Arrival (TDoA)
In TDoA the setup is different, the anchors are transmitting packets while the Crazyflie is passively listening to the traffic. From the received information it is unfortunately not possible to measure the distance to the anchors, but what we can get is the difference in distance to two anchors. For example, we might know that we are 0.54 meters closer to anchor 3 than anchor 6, or similar. It is possible to calculate the position from this information and similarly to TWR the measurements are pushed to the kalman estimator for further processing.
This mode supports unlimited numbers of Crazyflies (swarms) but is less robust compared to TWR, especially outside the system. TDoA is similar to how GPS works.
Research
There are many researchers that use the Loco System, some use it as a positioning system and investigate topics like path planning or similar, while some others are looking at different questions related to the UWB positioning itself. We will not try to mention everyone, we probably only know of a small fraction of what is going on (please tell us!), but would like to point out two areas of research.
The first is related to improving the estimated position by handling measurement errors and the environment in a better way. Examples of this is to compensate for differences in reception angle or handling of obstacles in the space. We would like to mention Wenda Zhao’s work at the Dynamic Systems Lab, University of Toronto. He has contributed the robust TDoA implementation in the kalman estimator (blog post) as well as a public TDoA data set.
The second is inter drone ranging, that is measuring the distance between drones as an addition to, or instead of drone-to-anchor measurements. Examples in this are are the work by Dr Feng Shan at School of Computer Science and Engineering Southeast University, China (blog post) and professor Klaus Kefferpütz, Hochschule Augsburg, work on “Crazyflie quadcopter in decentralized swarming” as presented on the BAM days last year.
Experimental functionality
Even though there has not been a lot of code committed lately in our repositories related to the Loco Positioning System, it has been simmering in the background. We would like to mention what is cooking in the pots and some of the stuff that has been discussed or tested.
System size
An 8 anchor Loco Positioning System can cover a flight space of around 8×8 meters, but from time to time we get the question of larger systems. TDoA3 was designed with this in mind and supports up to 255 anchors, which in theory would make it possible to build larger systems. This functionality was implemented 4 years ago but we never really tested it(!). Finally we collected all anchors in the lab an set up 20 anchors in the same system, and it worked! This should make it possible to extend systems to at least 15×15 meters, but maybe even more with some clever radio cell planing.
Another possibility to enlarge a system is to tweak the radio settings to make them reach longer. There is a “Longer range” mode in TDoA3 that lowers the bit rate, but again it has not really been verified. This was also tested in the latest Loco frenzy and with some minor modifications it worked the way we hoped, with 20 anchors! The tests mainly verified that the anchors play nicely together, and we are not sure about the maximum range (to be tested) but we believe distances of up to 40 meters between anchors is possible. To use this feature you should make sure to use the latest firmware for the Loco Nodes as well as the Crazyflie.
The two features mentioned above should hopefully make it possible to go big and we hope it could be used for shows for instance.
TDoA3 hybrid mode
If one looks at the messages sent in a TDoA system, the anchors are actually doing TWR with each other, while the Crazyflie(s) are just listening to the traffic and that the possibility to extract the position is a nice “side effect”. Now imagine if the Crazyflies were to send some messages from time to time, then they could act as “dynamic” anchors, or do inter-drone ranging with each other. This is something we call TDoA3 hybrid mode.
Currently there is no official implementation of the Hybrid mode, only some experimental hacks. Some researchers have done their own implementations, but we hope, at some point, to generalize the functionality and integrate it into the firmware.
Read more
If you are interested to read more about positioning and the Loco system, you can take a look at the following link list.
Summer is coming and with that vacations, yeay! There will always be someone at the office to help you if you need help, and we will handle shipping through out the summer, but it might take a bit longer than usual.
