In this project, we use OpenCV in python to draw on the screen using a virtual pen i.e, any marker can be used to draw/erase using the technique of contour detection based on the mask of the desired colored target marker.
Note: Make sure the target maker color is not present in the background to avoid error.
Have a look at the final project in action:
The entire code for this opencv post is available at the following github Repository:
Work flow/Structure:
- STEP1: Find the HSV range of the target Marker/pen and save the values in a .npy file
- STEP 2: Use the above values to start drawing in the main code
- First, we will use color masking to get a mask of our colored pen/marker using the above HSV range.
- Next, using contour detection we detect and track the location of that pen,i.e get the x,y coordinates.
- Next, draw a line by joining the x,y coordinates of pen’s previous location (location in the previous frame) with the new x,y points.
- Add a feature to use the marker as an Eraser to erase unwanted lines.
- Finally, add another feature to clear the entire Canvas/Screen.
Step1: Find HSV color range of the marker/pen we will use
- Import the required Libraries:
import cv2
import numpy as np
- Define a capture device .Here we use our primary laptop camera(0).If using external usb camera use replace 0 with 1.
cap=cv2.VideoCapture(0)
- We create six trackbars to set the values of upper and lower limit of H,S,V respectively within a new window named “trackbar“
def nothing(x):
pass
#Create trackbar to adjust HSV range
cv2.namedWindow("trackbar")
cv2.createTrackbar("L-H","trackbar",0,179,nothing)
cv2.createTrackbar("L-S","trackbar",0,255,nothing)
cv2.createTrackbar("L-V","trackbar",0,255,nothing)
cv2.createTrackbar("U-H","trackbar",179,179,nothing)
cv2.createTrackbar("U-S","trackbar",255,255,nothing)
cv2.createTrackbar("U-V","trackbar",255,255,nothing)
- Start reading frames inside an infinite loop:
- get the updated trackbar values,
- create a mask using this range, and
- Finally, do a bitwise operation and display the result.
while True:
ret,frame =cap.read()
hsv=cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)
l_h=cv2.getTrackbarPos("L-H","trackbar")
l_s=cv2.getTrackbarPos("L-S","trackbar")
l_v=cv2.getTrackbarPos("L-V","trackbar")
h_h=cv2.getTrackbarPos("U-H","trackbar")
h_s=cv2.getTrackbarPos("U-S","trackbar")
h_v=cv2.getTrackbarPos("U-V","trackbar")
low=np.array([l_h,l_s,l_v])
high=np.array([h_h,h_s,h_v])
mask=cv2.inRange(hsv,low,high)
result=cv2.bitwise_and(frame,frame,mask=mask)
cv2.imshow("result",result)# If the user presses ESC then exit the program
- press ‘s’ in keyboard to save the current settings of HSV range values and break from the infinite loop.This will create a new file penrange.npy and exit from the program.
- Outside the loop release cap and destroy all open windows.
key = cv2.waitKey(1)
# If the user presses `s` then print and save this array.
if key == ord('s'):
thearray = [[l_h,l_s,l_v],[h_h, h_s, h_v]]
# Save this array as penrange.npy
np.save('penrange',thearray)
break
#if esc pressed exit
if key == 27:
break
cap.release()
cv2.destroyAllWindows()
STEP 2: Use the above values to start drawing in the main code
- Import the required libraries.
import cv2
import numpy as np
- Create a new class named drawingCanvas withing which we will define all the required functions.
- First, we define __init__() and initialize the required variables.
- Also, call the draw() function from here.
class drawingCanvas():
def __init__(self):
self.penrange = np.load('penrange.npy') # load HSV range
self.cap = cv2.VideoCapture(0) #0 means primary camera .
self.canvas = None #initialize blank canvas
#initial position on pen
self.x1,self.y1=0,0
# val is used to toggle between pen and eraser mode
self.val=1
self.draw() #Finally call the draw function
- Next, define the draw() function:
- read frames from the camera within an infinite loop
- flip the frame horizontally
- create a mask by calling CreateMask function
- Detect contours using the mask.
- Draw lines on the canvas.
- Finally, display the results.
- Read keyboard input ,store it in k and call takeAction() function.
- Press ‘Esc’ key to break and exit the program.
def draw(self):
while True:
_, self.frame = self.cap.read() #read new frame
self.frame = cv2.flip( self.frame, 1 ) #flip horizontally
if self.canvas is None:
self.canvas = np.zeros_like(self.frame) #initialize a black canvas
mask=self.CreateMask() #createmask
contours=self.ContourDetect(mask) #detect Contours
self.drawLine(contours) #draw lines
self.display() #display results
k = cv2.waitKey(1) & 0xFF #wait for keyboard input
self.takeAction(k) #take action based on k value
if k == 27: #if esc key is pressed exit
break
- Next , define CreateMask() function.
def CreateMask(self):
hsv = cv2.cvtColor(self.frame, cv2.COLOR_BGR2HSV) #convert from BGR to HSV color range
lower_range = self.penrange[0] #load HSV lower range
upper_range = self.penrange[1] #load HSV upper range
mask = cv2.inRange(hsv, lower_range, upper_range) #Create binary mask
return mask
- Next, define ContourDetect() function.
def ContourDetect(self,mask):
# Find Contours based on the mask created.
contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
return contours
- Next, define drawLine() function: Here based on valid current and previous marker position we draw a line by joining the two coordinates.
def drawLine(self,contours):
#if contour area is not none and is greater than 100 draw the line
if contours and cv2.contourArea(max(contours, key = cv2.contourArea)) > 100: #100 is required min contour area
c = max(contours, key = cv2.contourArea)
x2,y2,w,h = cv2.boundingRect(c)
if self.x1 == 0 and self.y1 == 0: #this will we true only for the first time marker is detected
self.x1,self.y1= x2,y2
else:
# Draw the line on the canvas
self.canvas = cv2.line(self.canvas, (self.x1,self.y1),(x2,y2), [255*self.val,0,0], 10)
#New point becomes the previous point
self.x1,self.y1= x2,y2
else:
# If there were no contours detected then make x1,y1 = 0 (reset)
self.x1,self.y1 =0,0
- Next, define display(): Here the canvas and frame with lines is displayed.
def display(self):
# Merge the canvas and the frame.
self.frame = cv2.add(self.frame,self.canvas)
cv2.imshow('frame',self.frame)
cv2.imshow('canvas',self.canvas)
- Define takeAction() to perform different operations based on keyboard input
- eraser toggle mode (Press ‘e’ to toggle). This will draw with black color when we move the marker and in a way can perform erase action on the canvas.
- canvas clear operation (Press ‘c’ ).
Finally Create instance of the class drawingCanvas.
def takeAction(self,k):
# When c is pressed clear the entire canvas
if k == ord('c'):
self.canvas = None
#press e to change between eraser mode and writing mode
if k==ord('e'):
self.val= int(not self.val) # toggle val value between 0 and 1 to change marker color.
if __name__ == '__main__':
drawingCanvas()
cv2.destroyAllWindows()
That’s all we need to make our very own virtual marker for the screen using Opencv and any marker/color object lying around.
