Image Recognition Project - Due 3/6

Do the work for this project in a repo called imgRecName1Name1

Choose two cereal boxes from the classroom pile. When your program starts up, the robot will be surrounded by cereal boxes - the user should enter which of the two cereal boxes it's looking for, and it should then turn until it sights the box, and then drive into it.

To start, you'll want clear, straight-on, well-lit, no glare pictures of your cereal boxes, which you then crop to contain only the cereal box (make sure these pictures are in your repo).

Code examples

Given an image img, here's how you use OpenCV to calculate features:

grayImg=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) #make it gray

sift=cv2.xfeatures2d.SIFT_create()

kp,des=sift.detectAndCompute(grayImg,None)

Given two sets of features, (knownImgKP,knownImgDes) and (robotImgKP,robotImgDes), where knownImg* are the features from your cropped photo, and robotImg* are from your robot's current scene, the following computes matches (the .75 is a tunable number - smaller will give you fewer better matches, larger will give you more worse matches):

bf=cv2.BFMatcher()
matches=bf.knnMatch(knownImgDes,robotImgDes,2)
good=[]
for m,n in matches: #call the first two matches m and n
  if m.distance<0.75*n.distance: #if the first match is much better than the second
    good.append([m])

To get at the X,Y values of the matches, you have to do some digging. Each element of good is now a list of size 1 of DMatch objects. The following code prints out the (x,y) values, in the robot scene, of each match:

for g in good:
  x=robotImgKP[g[0].trainIdx].pt[0]
  y=robotImgKP[g[0].trainIdx].pt[1]
  print x,y

To visualize the matches like we did in class, you can use the following to make the image here called matches to be displayed:

matches=np.zeros((1,1,3),np.uint8)
matches=cv2.drawMatchesKnn(knownImg,knownImgKP,robotImg,robotImgKP,good,matches)

Speed

Doing all of this, at frame-rate, will likely be difficult. Make sure you're thinking about what is necessary to compute every cycle, and what is unnecessary.