====== How to Use Darwin OP 2's Computer Vision ====== **Author:** Yu Hang He **Email:** \\ **Date:** Last modified on <11/19/17> \\ **Keywords:** Darwin OP 2, C++/C Programming, Robotis OP 2, Computer Vision \\ \\ This tutorial will demonstrate how to use vision processing capability on Darwin OP 2 to detect and track color. Darwin OP 2 have computer vision that can detect multiple colors and return x and y center position of those colors. A second class will move camera to keep track of color based on x and y position. It takes approximately 2 hours to complete. \\ ===== Motivation and Audience ===== This tutorial's motivation is to demonstrate how to use computer vision on Darwin OP 2. This tutorial assumes the reader has the following background and interests: * Familiar with handling Darwin OP 2 * Familiar with C++/C programming language * Familiar with C++/C codes on Darwin OP 2 * Interested in computer vision The rest of this tutorial is presented as follows: * [[darwin_color_detection#Color_Detection | Color Detection]] * [[darwin_color_detection#Camera_Tracking | Camera Tracking]] * [[darwin_color_detection#Demonstration | Demonstration]] * [[darwin_color_detection#Final_Words | Final Words]] ===== Color Detection ===== The core of image processing on Darwin OP 2 is ColorFinder class. ColorFinder can find the center of mass of detected color and return its location. There is a [[http://support.robotis.com/en/product/darwin-op/development/tutorials/linux/image_processing.htm | tutorial]] on Robotis' online manual that can provide more insights. \\ \\ ColorFinder class requires 5 input color parameters: ''hue, tolerance, minimum saturation, minimum value, gain, and exposure''. To find the most accurate set of values for the color you want to detect, use the color_filtering program on Darwin OP 2. \\ \\ Use terminal and go to directory ''**/robotisop2/Linux/project/tutorial/color_filtering**'' \\ \\ Type in ''make'' and execute the program ''./color_filtering''. \\ \\ {{ :yuhang:darwin_sprint:make.jpg? 500 |}} \\ Once the program is running, open web browser and type in the address http://192.168.123.1:8080 \\ \\ {{ ::yuhang:darwin_sprint:web.jpg? 500 |}} \\ Now you can adjust the values until Darwin OP 2 properly detect your color. \\ \\ In this example, I am trying to detect a blue color. \\ \\ **Before adjustments** \\ \\ {{ :yuhang:darwin_sprint:yuhang:darwin_sprint:before.jpg?500 |}} \\ **Afterward** \\ \\ {{ ::yuhang:darwin_sprint:after.jpg?500 |}} \\ The adjusted values are: * ''Hue: 225'' * ''Tolerance: 15'' * ''Minimum Saturation: 45'' * ''Minimum Value: 0'' * ''Gain: 0.3'' * ''Exposure: 50.0'' ==== Programming ==== This tutorial will demonstrate how to create a new ColorFinder object and how to use it. Only show parts of code that has been modifed. \\ \\ ''main.cpp'' can be found under directory ''/robotisop2/Linux/project/demo'' \\ //declare a new object of class ColorFinder ColorFinder* finder = new ColorFinder(); //you can initiate the object with 5 input color parameters ColorFinder* green_finder = new ColorFinder(120, 45, 35, 0, 0.3, 40.0); //There are default values of color you can load from ini file. Example finder->LoadINISettings(ini, "RED"); //To get the position of center of mass of filtered pixels Point2D pos; pos = finder->GetPosition(LinuxCamera::GetInstance()->fbuffer->m_HSVFrame); //first declare an object of class Point2D //then set it equal to GetPosition function of ColorFinder object ===== Camera Tracking ===== Camera tracking can be accomplished through BallTracker class. An object of this class receive a Point2D object as input, then internally calculate pan and tilt motor movements in Darwin OP 2's head to keep track of position. There is a [[http://support.robotis.com/en/product/darwin-op/development/tutorials/linux/head_control.htm|tutorial]] on Robotis' online manual that may provide more insight. \\ \\ The goal of this demonstration is for Darwin OP 2 to keep track of center of blue and green color. In this demonstration, I created two ColorFinder objects, green_finder and blue_finder, to find the center of mass of its respective colors. Furthermore, I created two Point2D objects, green and blue, to keep track of position of colors. Finally, I used a third Point2D object, center, to store the average between green and blue position. ==== Programming ==== directory ''**/robotisop2/Linux/project/demo**'' \\ This section will only show parts of code that has been modified. //inside main loop int main() { ColorFinder* green_finder = new ColorFinder(120, 45, 35, 0, 0.3, 40.0); ColorFinder* blue_finder = new ColorFinder(225, 15, 45, 0, 0.3, 50.0); //created 2 new ColorFinder objects to find color green and blue BallTracker marker_tracker = BallTracker(); //created a new BallTracker object to use camera tracking //inside while loop while(1) { else if(StatusCheck::m_cur_mode == SPRINT) //under a new mode called Sprint { Point2D green, blue, center; //created 3 objects of Point2D class green = green_finder->GetPosition(LinuxCamera::GetInstance()->fbuffer->m_HSVFrame); blue = blue_finder->GetPosition(LinuxCamera::GetInstance()->fbuffer->m_HSVFrame); //store position of green and blue color if(green.X < 0 || blue.X < 0) center.X = -1; else center.X = (green.X + blue.X)/2; if(green.Y < 0 || blue.Y < 0) center.Y = -1; else center.Y = (green.Y + blue.Y)/2; //calculate center of blue and green marker_tracker.process(center); //use camera tracking } ===== Demonstration ===== In this demonstration, I used camera track the center between green and blue colors. \\ \\ First I used color_filtering to find 5 input parameters for green and blue colors. \\ \\ {{ ::yuhang:darwin_sprint:2_color.jpg?500 |}} \\ Then, followed the step outlined in the program section and implemented it within a new mode. \\ Here is the result. {{ youtube>4HAZPNL5eY0?large }} ===== Final Words ===== This tutorial's objective was to demonstrate how to use camera tracking and color detection on Darwin OP 2. \\ \\ For questions, clarifications, etc, Email: