====== Pegasus Cam OpenCV Examples ====== **Author:** Dylan Wallace Email: \\ **Date:** Last modified on 03/04/19 \\ **Keywords:** Pegasus Cam, OpenCv, Face Tracking, Ball Tracking \\ This tutorial shows how to implement some common OpenCV algorithms using the Pegasus Cam. The tutorial covers Aruco Marker Detection, Facial Tracking, and Ball Tracking. This tutorials will help the user to become more comfortable using the Pegasus cam for computer vision, and will demonstrate how powerful the Pegasus Cam framework is for networked computer vision. This tutorial takes around 2 hours to complete. All source code for these examples can be found on the [[https://github.com/D-Wazzle/Pegasus-Cam-CV|Pegasus Cam Github]]. \\ ===== Motivation and Audience ===== This tutorial's motivation is to demonstrate the capabilities of the Pegasus Cam for networked computer vision. This tutorial assumes the reader has the following background and interests: * Know how to use the basics of the Linux command line interface (CLI) \\ * Working knowledge of Python and C/C++ \\ * Perhaps additional background needed may include OpenCV experience \\ \\ The rest of this tutorial is presented as follows: * [[pegasus_cam_cv#aruco_marker_detection|Aruco Marker Detection]] * [[pegasus_cam_cv#face_tracking|Face Tracking]] * [[pegasus_cam_cv#ball_tracking|Ball Tracking]] * Final Words ===== Aruco Marker Detection ===== This program is built to detect and track Aruco markers within the frame. These markers are built-in to OpenCV to allow for AR applications and motion tracking. In Python, this code is very simple and effective. To start, we import necessary libraries, most important being the Aruco library. \\ import sys import cv2 import numpy as np import aruco \\ The main loop with comments can be seen below: \\ if __name__ == '__main__': # get camera parameters camparam = aruco.getParameters() camparam.readFromXMLFile(os.path.join(os.path.dirname(__file__), "dfk72_6mm_param2.yml")) # create detector and get parameters detector = aruco.MarkerDetector() params = detector.getParameters() # print detector parameters print("detector params:") for val in dir(params): if not val.startswith("__"): print("\t{} : {}".format(val, params.__getattribute__(val))) # import Pegasus Cam stream cap = cv2.VideoCapture("http://192.168.50.22:3000/html/cam_pic_new.php?time=9999999999999999999999999&pDelay=40000") # read a frame from the stream ret, frame = cap.read() # throw error if stream is bad if not ret: print("Can't open video!") sys.exit(-1) # while the stream is good while ret: # get markers in the frame markers = detector.detect(frame) # Print marker coords for marker in markers: print("Marker: {:d}".format(marker.id)) for i, point in enumerate(marker): print("\t{:d} {}".format(i, str(point))) marker.draw(frame, np.array([255, 255, 255]), 2) # calculate marker extrinsics for marker size of 3.5cm marker.calculateExtrinsics(0.035, camparam) # print("Marker extrinsics:\n{}\n{}".format(marker.Tvec, marker.Rvec)) print("detected ids: {}".format(", ".join(str(m.id) for m in markers))) # show frame cv2.imshow("frame", frame) cv2.waitKey(100) # read next frame ret, frame = cap.read() \\ This code should produce a result similar to the video below. \\ {{ youtube>aLAa_pvDLho?large }} \\ ===== Face Tracking ===== This program is made to detect faces in the frame using the Haar Cascades as the face classifier. The code is implemented in Python, and the full code with comments explaining the usage can be seen below: \\ import numpy as np import cv2 from datetime import datetime # import Haar cascades face_cascade = cv2.CascadeClassifier('/usr/local/share/OpenCV/haarcascades/haarcascade_frontalface_default.xml') eye_cascade = cv2.CascadeClassifier('/usr/local/share/OpenCV/haarcascades/haarcascade_eye.xml') # import Pegasus Cam stream cap = cv2.VideoCapture("http://192.168.50.22:3000/html/cam_pic_new.php?time=9999999999999&pDelay=40000") # while the stream is good while(cap.isOpened()): # get current time time_1 = datetime.now() print(time_1) # get current frame ret, frame = cap.read() # convert ot grayscale gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # detect faces in the frame faces = face_cascade.detectMultiScale(gray, 1.3, 5) # display bounding boxes for face features for(x, y, w, h) in faces: cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), -1) roi_gray = gray[y:y+h, x:x+w] roi_color = frame[y:y+h, x:x+w] eyes = eye_cascade.detectMultiScale(roi_gray) # dispay bounding boxes for eyes for(ex, ey, ew, eh) in eyes: cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2) cv2.imshow('frame', frame) # get elapsed time time_2 = datetime.now() dt = time_2 - time_1 print(dt.seconds*1000 + dt.microseconds/1000) # wait for ESC key to exit k = cv2.waitKey(10) if k == 27: cv2.destroyAllWindows() cap.release() cv2.destroyAllWindows() \\ After implementing the code above, you should get results similar to the video seen below. \\ {{ youtube>HBtBz4uqVCI?large }} \\ ===== Ball Tracking ===== This program is used to track a red ball within the frame of the stream. This color can be modified in the code for other applications. It should be noted that this code shows an alternative way of import the Pegasus Cam stream into OpenCV by using a parameter for the http stream on the command line. With the use of parameter files, this would allow for a "global" variable that contains the location of any Pegasus Cam streams on a network. The full code with comments explaining the usage can be seen below. \\ # USAGE # python ball_tracking.py --video http://192.168.50.22:3000/html/cam_pic_new.php?time=9999999999999999999999999&pDelay=40000 # python ball_tracking.py # import the necessary packages from collections import deque import numpy as np import argparse import imutils import cv2 # construct the argument parse and parse the arguments ap = argparse.ArgumentParser() ap.add_argument("-v", "--video", help="path to the (optional) video file") ap.add_argument("-b", "--buffer", type=int, default=64, help="max buffer size") args = vars(ap.parse_args()) # define the lower and upper boundaries of the "red" # ball in the HSV color space, then initialize the # list of tracked points redLower = (0, 100, 100) redUpper = (100, 255, 255) pts = deque(maxlen=args["buffer"]) # if a video path was not supplied, grab the reference # to the webcam if not args.get("video", False): camera = cv2.VideoCapture(0) # otherwise, grab a reference to the video file else: camera = cv2.VideoCapture(args["video"]) # keep looping while True: # grab the current frame (grabbed, frame) = camera.read() # if we are viewing a video and we did not grab a frame, # then we have reached the end of the video if args.get("video") and not grabbed: break # resize the frame, blur it, and convert it to the HSV # color space frame = imutils.resize(frame, width=600) # blurred = cv2.GaussianBlur(frame, (11, 11), 0) hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) # construct a mask for the color "red", then perform # a series of dilations and erosions to remove any small # blobs left in the mask mask = cv2.inRange(hsv, redLower, redUpper) mask = cv2.erode(mask, None, iterations=2) mask = cv2.dilate(mask, None, iterations=2) # find contours in the mask and initialize the current # (x, y) center of the ball cnts = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2] center = None # only proceed if at least one contour was found if len(cnts) > 0: # find the largest contour in the mask, then use # it to compute the minimum enclosing circle and # centroid c = max(cnts, key=cv2.contourArea) ((x, y), radius) = cv2.minEnclosingCircle(c) M = cv2.moments(c) center = (int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"])) # only proceed if the radius meets a minimum size if radius > 10: # draw the circle and centroid on the frame, # then update the list of tracked points cv2.circle(frame, (int(x), int(y)), int(radius), (0, 255, 255), 2) cv2.circle(frame, center, 5, (0, 0, 255), -1) # update the points queue pts.appendleft(center) # loop over the set of tracked points for i in xrange(1, len(pts)): # if either of the tracked points are None, ignore # them if pts[i - 1] is None or pts[i] is None: continue # otherwise, compute the thickness of the line and # draw the connecting lines thickness = int(np.sqrt(args["buffer"] / float(i + 1)) * 2.5) cv2.line(frame, pts[i - 1], pts[i], (0, 0, 255), thickness) # show the frame to our screen cv2.imshow("Frame", frame) key = cv2.waitKey(1) & 0xFF # if the 'q' key is pressed, stop the loop if key == ord("q"): break # cleanup the camera and close any open windows camera.release() cv2.destroyAllWindows() \\ After implementing the code above, you should get results similar to the video below. \\ {{ youtube>fWTrsr3X_y0?large }} \\ ===== Final Words ===== This tutorial's objective was to show application of the Pegasus Cam for networked computer vision. Complete source code as well as descriptions of the code was provided. Once the concepts were conveyed the reader should be able to create their own OpenCV algorithms with the Pegasus Cam. \\ \\ Speculating future work derived from this tutorial, includes implementing more computer vision algorithms such as people tracking, facial detection, or emotion recognition. In the big picture, the problem of low-latency networked computer vision can be solved with this tutorial. \\ \\ For questions, clarifications, etc, Email: