This week we have a guest blog post from Anirudha Majumdar from Mechanical & Aerospace Engineering, Princeton University. Enjoy !
Overview
The course “Introduction to Robotics” at Princeton (MAE 345 / ECE 345 / COS 346) introduces students to fundamental theoretical and algorithmic principles in robotics. We use the Crazyflies to bring the excitement of drones to students in their very first introductory course in robotics. The course is primarily aimed at undergraduate students in their third and fourth years, and also offers a graduate-level track. In Fall 2021, approximately seventy undergraduate students and ten graduate students enrolled in the course from a variety of different majors: Mechanical and Aerospace Engineering, Computer Science, Electrical and Computer Engineering, Mathematics, Operations Research and Financial Engineering, Civil and Environmental Engineering, and Architecture.
The course covers the following technical topics:
- Feedback Control;
- Motion Planning;
- State estimation, localization, and mapping;
- Computer vision and learning;
- Broader topics: Robotics and the law, ethics, and economics.
One of the primary aims of the course is to allow students to obtain hands-on experience implementing various algorithms on a hardware platform. For this, we use the Crazyflie 2.1 quadrotors with a camera attachment (Fig. 1). The Crazyflie is an ideal hardware platform for our course due to its light weight, reliability, safety, and open-source code. Throughout the semester, students work in teams to implement different algorithms from the course:
(i) linear quadratic regulator (LQR) for stabilizing the drone
(ii) rapidly-exploring randomized trees (RRTs) for motion planning through a forest of PVC pipe obstacles
(iii) the Lucas-Kanade optical flow algorithm
(iv) object-following with neural networks.
In the final project, students implement algorithms for autonomous vision-based navigation through novel (i.e., a priori unknown) obstacle environments (Fig. 2). Below, we describe the modified Crazyflies used for the course, along with the hands-on projects in more detail.
Open-source course materials and website
We have made the course materials (lecture notes, slides, and assignments) freely available through our course website. The assignments include theory questions, programming assignments in Python, and hardware implementation with the Crazyflie. All programming and hardware assignments are freely-available through GitHub. We hope that the course materials will be useful to instructors at other institutions in getting the next generation of engineers excited about robotics!
Hardware components and lab spaces
The feedback control and motion planning hardware assignments can be completed with the following parts list:
- Crazyflie 2.1 drone
- Flow deck v2
- Crazyflie PA radio
- Spare parts (if necessary)
For the vision-based assignments, we attach cameras to the drones, which transmit images in real-time to a receiver unit plugged into to a laptop. This requires the following additional parts:
We set up three netted arenas for students to complete hardware assignments. Each arena is approximately 12 ft x 8 ft in size; two of the arenas are shown in Fig. 3.
Crazyflie assignments
Lab 1: Linear Quadratic Regulator (LQR)
In the first hardware lab, students implement and tune a Linear Quadratic Regulator (LQR) controller to make the drone hover. This builds on the theoretical foundations of feedback control covered in the first module of the course. Implementing LQR control on a physical system allows students to get a sense for the “reality gap”, i.e., assumptions made by the theory that are not perfectly satisfied in real life (e.g., linear dynamics, perfect state estimation, and perfect knowledge of the dynamics). This assignment makes use of a modified version of the Crazyflie firmware that allows us to upload different controller gains (obtained using LQR) and run these on the drones. Code for this assignment is available here.
Lab 2: Rapidly-Exploring Randomized Trees (RRTs)
In the second lab, students implement the RRT algorithm for motion planning. Each netted arena is set up with PVC pipe obstacles (similar to Fig. 2). Students measure the obstacle locations and radii and use the RRT algorithm to find a collision-free path from a starting location to a goal location. The Crazyflie then uses its waypoint tracking capabilities to execute the motion plan. Code for this assignment is available here.
Lab 3: Lucas-Kanade optical flow
In the computer vision module of the course, we first introduce students to “classical” vision approaches (before discussing modern deep learning techniques). One of the primary algorithms we discuss in this module is the Lucas-Kanade algorithm for computing optical flow (which has many applications such as computing time-to-collision). Students record a video using the drone’s on-board camera attachment and process this offline to compute optical flow. Code for this assignment is available here.
Lab 4: Object tracking
In the computer vision module of the course, we introduce students to deep learning-based approaches to vision. As a demonstration of the power of deep learning, we use neural networks to perform real-time object detection. Specifically, students use neural networks trained on the Coco dataset to detect a person (or an object such as a cup) using the drone’s camera, and use the resulting bounding box to command the drone in a way that makes it follow the person or object. The neural networks can be run on standard laptop computers at >= 30Hz, allowing the drone to follow the person or object in real-time. Code for this assignment is available here.
Final project: Vision-based obstacle avoidance and target-seeking
In the final project for the course, students implement vision-based navigation and target-seeking using the Crazyflie drones. Teams must program the drones to autonomously navigate through a forest of PVC pipes, whose locations are not known beforehand. Students utilize different computer vision algorithms (e.g., edge detection, contour finding, etc.) to detect obstacles and program strategies to avoid these obstacles. The goal is to navigate to the other side of the flying arena and land in front of a target object (in the form of a book). Again, teams use different approaches (e.g., neural networks) to find the book and then land in front of it. Further details and starter code for the project can be found here. See above for a video.
Acknowledgements
The following teaching staff have contributed greatly to the development of the materials for this course: Vincent Pacelli, Julienne LaChance, Jon Prevost, Alec Farid, Meghan Booker, Lena Rosendahl, David Snyder, Allen Ren, and Eric Lepowsky.