LEGO_EV3_Python_Examples/Object_Sorting

EV3_Python Object sorter

The following document describes a project including a program written in Python for the Lego Mindstorm EV3. The purpose of the prject is to let a robot sort cubes based on their color. Before using the program on the EV3 Brick you need to prepare it as described in the README document of the framework.

Used Hardware

The program was written for the LEGO EV3 Brick. It is the base of every Mindstorm EV3 robot. Additionally, for identification of an object / color, a camera from an external company is used. The built / used robot has two motors, one on each side, with two wheels each connected via a caterpillar track. But the program should work with all robots that have one driven wheel per side.

Camera

The camera used for this program is the Pixy2 for Lego Mindstorms. To work with the camera, you have to teach it the object you want to sort. For this, you need PixyMon on your PC. How to get PixyMon, teach an object and everything else you need to know is listed on the PixyCam Website.

• Camera: Connected to input port 1

• Taught Signature 1: orange cube, leave in sorting area

• Taught Signature 2: green cube, sort out

The camera should be mounted on top of the robot and directed downwards for good results in detection.

Motors

The robot drives forward when the wheels turn counter-clockwise. If you use a robot with a clockwise direction of rotation, you need to change the parameters given in the arguments to 'normal'. (See Additional Parameters in Initialization)

• Left Motor: Connected to output-port A
• Right Motor: Connected to output-port D

Sensors

To track the distance between the robot and the cube, a LEGO Ultrasonic Sensor is used. To identify the outer line of the sorting area, one LEGO Color Sensor is used. For keeping track of the rotation on the back side of the robot, a LEGO gyro sensor is mounted.

• Ultrasonic Sensor: Connected to input-port 2
• Color Sensor: Connected to input-port 3
• Gyro Sensor: Connected to input-port 4

Objects to sort

The sorted objects are simple cubes made of paper. You can find the tutorial for folding the cube on YouTube. To ensure the ultrasonic sensor recognizes the cube, it is 5.5 cm high and long. The two colors should be distinct from each other.

Program

The purpose of the program is to sort different objects. In detail, this means that the robot should push all green cubes out of the sorting area and leave all orange cubes inside.

The main part of the program is the class "Object_Sorter". Due to the fact that the robot gets a lot of inputs and does not need to check everything at the same time, the class contains five different state methods. The robot will switch automatically between the states when certain events happen. You can identify the current state by looking at the LEDs on the EV3-brick.

• Search: Both LEDs are black
• Get: Only the left LED is green
• Sort: Both LEDs are green
• Avoid: Both LEDs are orange
• Edge: Both LEDs are red

The different methods of the class are explained in detail below.

Initialization

During the initialization phase of the object sorter all the arguments given are assigned.

In- and Outputs:

• ultrasonic_input: Input Port of the Ultrasonic Sensor (Default:INPUT_2)
• color_input: Input Port of the Color Sensor (Default:INPUT_3)
• gyro_input: Input Port of the Gyro Sensor(Default:INPUT_4)
• camera_port: Input Port of the Pixy2 camera (Default: 1)
• leftmotor_output: Output Port of the left motor (Default:OUTPUT_A)
• "rightmotor_output: Output Port of the right motor (Default:OUTPUT_D)

Additional Parameters:

• leftmotor_polarity: Polarity (direction of rotation) of the left motor (Default:'inversed')
• rightmotor_polarity: Polarity (direction of rotation) of the right motor (Default:'inversed')
• signature_dice_remains: Number of signature of the cube that should remain inside the outline (Default:1)
• signature_dice_sorted: Number of signature of the cube that should be sorted out of the area (Default:2)
• speed_drive: Speed of motor during normal driving (Default:30)
• speed_search: Speed of motor during search mode (Default:10)
• distance_us: Detection range, where the "caught" cube should be in, relevant for the ultrasonic sensor (Default:12)
• max_reflection_outline: Maximal reflection of outline (Default:10)

Method: Search

The target of the method is to search for new cubes to sort. The robot spins around until one of five events causes a change.

Robot drives over black line: If the robot drives across the line, it switches to the state "Edge".

Robot has a cube in his arms: When the Ultrasonic Sensor detects an object in front of it, this means there is already a cube in its arms, and it switches to state "Sort".

Camera detects orange cube: If the camera finds an orange cube, it changes to the state "Avoid".

Camera detects green cube: If the camera finds a green cube, it changes to the state "Get".

Robot spun without finding something: When the robot has turned for 2 full rounds (720°) and has not detected one of the four things above, it will drive a bit forward. If it reaches the outline, it changes to the state "Edge". Otherwise, it starts the state "Search" again at the new spot.

Method: Get

The purpose of the method is to drive towards the green cube and "catch" it in its arms. To make that happen, the method checks the current position of the cube and corrects the driving direction if necessary. Due to the fact that there are several reasons why the camera does not consistently recognize the cube, the robot has a routine to find the cube again (driving backwards, turning left). If the routine does not work, the program changes back to state "Search". Otherwise, the ultrasonic sensor detects the cube in the arms, and the program changes to state "Sort", or the robot moves over the line, and it switches to state "Edge".

Method: Sort

The task of the method is to drive the cube to the outline of the sorting area and avoid the orange cubes on the way. If the camera detects an orange cube, the program switches to the state "Avoid", whereas if it doesn't, the robot continues straight ahead until the outline is reached and the program changes to the state "Edge".

Method: Avoid

The detected orange cube should be avoided by the robot. Therefore, the camera looks for the orange cube again and gets its position. If it is out of reach of the robot, the drive path will not change, but if the cube is on the way, the robot corrects its driving direction. After that, the program switches back to the state before. ("Search" or "Sort")

Method: Edge

The robot has reached the outline. It drives backward and turns 100° to the right. The program switches back to the state "Search" afterward.

Method: Run

This method only is used to start the operations of the object sorter and continue in an endless while-loop.

Problems and Improvements

Robot does not detect outer line

During programming, I was challenged by the problem that the outer line is sometimes not detected. After trying different approaches to fix the bug, I suspect that the processor of the EV3 is slowed down by the camera and, for this reason, does not detect the line when the sensor is above it.

Another reason can be that the arms project shadows or the line doesn't differ enough from the floor. Make sure that the given values and the physical structure of the robot make sense for your circumstances.

You can change all the input values such as the minimal line reflection and the driving rates with a dictionary of the arguments given to the class. (See Initialization)

Camera does not detect cubes

If the camera does not detect the cubes, there are multiple reasons. Sometimes, due to lightning changes, the camera does not identify the different colors correct. You can go back to PixyMon and see if the cubes are even detected. If not, you have to teach the object / cube to the camera again, preferably in the environment where the sorting happens. Also, there is a section on the PixyCam Website about improving the detection accuracy you should read.

Ultrasonic sensor does not detect cube

The ultrasonic sensor should be mounted as close to the floor as possible to make sure the cube gets detected. You should build the arms in such a way that they are not in the sensor's field of vision. But they should hold the cube as centered as possible in front of the robot. Make sure that you have followed all the construction advice.