How I made a drone autopilot in less than 100 lines of Python code

I have a DJI Tello drone. It is an affordable drone with wifi, camera, and a quite simple SDK that allows to develop programs for multiple purposes. In this post I explain how I made a python script that detects an object using color segmentation¹ and drives the drone in front of the object.

This code will work with other drones, but you must replace the references to DJITelloPy² with another library compatible with your model of drone.

The script shown in this post is in Github: 2c-AutopilotByColorSegmentation.py. Feel free to fork my Github repository.

The code was ran on a computer with Windows 10 1909, Python 3.9.0, and OpenCV 4.4.0

How to detect the object

Color segmentation is a technique that detects object contours using a range of colors. You need an object with homogeneous color different from the color of other objects in the background. I used an orange power bank.

Colors must be in HSV³ format. In my script the range is defined in the variables LOWER and UPPER. Adjust these values with the colors of your object.

First call to the function cv2.inRange() that marks the pixels that match de desired color range. Next call to cv2.findContours() to get the coordinates, width, and height of the object.

How to get a path to the object

Now that We have the coordinates of the object we can drive the drone to the object. To do this I use 3 type of movements: Forward and Backward, Up and Down, and Yaw (Rotate). I set velocity for each type of movement using these algorithms:

• Forward and Backward: Start with the value of DESIRED_OBJECT_SIZE (The value is the size from the left upper corner coordinate to the right lower corner), then subtract the distance between the left upper corner of the object and the right lower corner and multiply that result to the value of MAX_SPEED_FORWARDBACK. A positive result means the object is far, and the drone will approach to it. Negative result will cause the drone to go backward. Value equal or near cero makes the drone maintain the distance.
• Up and down: First calculate the Y coordinate of the center of the object, then subtract the Y coordinate of the center of the frame, multiply that result to the value of MAX_SPEED_UPDOWN, and multiply the result by -1. A positive result means the object is in the lower part of the image and the drone needs to move up. Negative will make the drone go down. With a value near or equal to zero the drone maintains the altitude.
• Yaw: First calculate the X coordinate of the center of the object, then subtract the X coordinate of the center of the frame, finally multiply that result to the value of MAX_SPEED_YAW. A positive result means the object is to the left and the drone will rotate to that direction. Negative will rotate to the right. Values near or equal to cero cause the drone to not rotate.

Demo

The script connect to a Tello drone, turns on streaming, and take off. After that there is an infinite bucle that takes photos, draws a rectangle around the object and drives the drone in front of it. The script will run until you press ESC key or Ctrl+C.

Here you have a short clip of the output of the script from the drone point of view.

References:

¹Thresholding Operations using inRange: docs.opencv.org/3.4/da/d97/tutorial_threshold_inRange.html
²DJI Tello drone python interface: github.com/damiafuentes/DJITelloPy
³HSL and HSV: en.wikipedia.org/wiki/HSL_and_HSV