⬡ Hub
Skip to content

OpenCV: Feature Detection and Description

Feature detection and description are fundamental concepts in computer vision. They involve finding distinct and recognizable points or regions in an image (features) and then creating a compact numerical representation (descriptor) of these features. These descriptors can then be used to compare features between different images, enabling tasks like object recognition, image matching, stitching, and augmented reality.

OpenCV provides implementations for many classical and modern feature detection and description algorithms.

1. Corner Detection

Corners are points in an image where the intensity changes significantly in at least two directions. They are generally robust to rotation, scaling, and illumination changes.

a. Harris Corner Detector (cv2.cornerHarris())

A classic algorithm for detecting corners. It's rotation invariant but not scale invariant.

import cv2
import numpy as np
import matplotlib.pyplot as plt

# Create a dummy image with some corners
img_orig = np.zeros((200, 200, 3), dtype=np.uint8)
cv2.rectangle(img_orig, (50, 50), (150, 150), (255, 255, 255), -1) # White square
cv2.rectangle(img_orig, (10, 10), (70, 70), (255, 255, 255), -1) # Another square
cv2.circle(img_orig, (100, 100), 20, (0, 0, 255), -1) # Red circle (no corners)

img_gray = cv2.cvtColor(img_orig, cv2.COLOR_BGR2GRAY)
img_display = img_orig.copy() # For drawing corners

# Apply Harris Corner Detector
# img: Input image (grayscale, float32)
# blockSize: Neighborhood size (typically 2)
# ksize: Aperture parameter for Sobel operator (typically 3)
# k: Harris detector free parameter (0.04 to 0.06)
dst = cv2.cornerHarris(img_gray, blockSize=2, ksize=3, k=0.04)

# Dilate to mark corners clearly (for visualization)
dst = cv2.dilate(dst, None)

# Threshold for an optimal value, mark corners in red
img_display[dst > 0.01 * dst.max()] = [0, 0, 255] # Red dots

plt.figure(figsize=(8, 6))
plt.imshow(cv2.cvtColor(img_display, cv2.COLOR_BGR2RGB))
plt.title('Harris Corners')
plt.axis('off')
plt.show()

b. Shi-Tomasi Corner Detector (cv2.goodFeaturesToTrack())

An improvement over Harris corners, giving better results for tracking. It directly finds the N strongest corners.

import cv2
import numpy as np
import matplotlib.pyplot as plt

img_orig = np.zeros((200, 200, 3), dtype=np.uint8)
cv2.rectangle(img_orig, (50, 50), (150, 150), (255, 255, 255), -1)
cv2.rectangle(img_orig, (10, 10), (70, 70), (255, 255, 255), -1)
cv2.circle(img_orig, (100, 100), 20, (0, 0, 255), -1)

img_gray = cv2.cvtColor(img_orig, cv2.COLOR_BGR2GRAY)
img_display = img_orig.copy()

# Apply Shi-Tomasi Corner Detector
# maxCorners: Maximum number of corners to return
# qualityLevel: Minimum accepted quality of image corners (0-1)
# minDistance: Minimum Euclidean distance between detected corners
corners = cv2.goodFeaturesToTrack(img_gray, maxCorners=100, qualityLevel=0.01, minDistance=10)
corners = np.int0(corners) # Convert to integer coordinates

# Draw detected corners
for i in corners:
    x, y = i.ravel()
    cv2.circle(img_display, (x, y), 3, (0, 255, 0), -1) # Green dots

plt.figure(figsize=(8, 6))
plt.imshow(cv2.cvtColor(img_display, cv2.COLOR_BGR2RGB))
plt.title('Shi-Tomasi Corners')
plt.axis('off')
plt.show()

2. Feature Descriptors (Keypoint Descriptors)

Keypoint detectors (like SIFT, SURF, ORB) first detect interest points (keypoints) in an image and then compute a "descriptor" for each keypoint. This descriptor is a vector that describes the local image region around the keypoint, making it invariant to changes in scale, rotation, and illumination.

a. ORB (Oriented FAST and Rotated BRIEF)

ORB is a robust and efficient alternative to SIFT/SURF (which were patented until recently). It's a combination of FAST keypoint detector and BRIEF descriptor.

import cv2
import numpy as np
import matplotlib.pyplot as plt

# Load a sample image (e.g., a simple shape or a texture)
# For this example, let's draw a simple shape on a blank image
image1_orig = np.zeros((200, 300, 3), dtype=np.uint8)
cv2.rectangle(image1_orig, (50, 50), (150, 250), (255, 255, 0), -1) # Cyan rectangle
cv2.circle(image1_orig, (70, 70), 20, (0, 255, 255), -1) # Yellow circle

image2_orig = cv2.rotate(image1_orig, cv2.ROTATE_90_CLOCKWISE) # Rotated version
image2_orig = cv2.resize(image2_orig, (300, 200)) # Resize to match original dimensions roughly

image1_gray = cv2.cvtColor(image1_orig, cv2.COLOR_BGR2GRAY)
image2_gray = cv2.cvtColor(image2_orig, cv2.COLOR_BGR2GRAY)

# Initialize ORB detector
orb = cv2.ORB_create()

# Find keypoints and descriptors
keypoints1, descriptors1 = orb.detectAndCompute(image1_gray, None)
keypoints2, descriptors2 = orb.detectAndCompute(image2_gray, None)

# Draw keypoints on images
img1_kp = cv2.drawKeypoints(image1_orig, keypoints1, None, color=(0, 255, 0), flags=0)
img2_kp = cv2.drawKeypoints(image2_orig, keypoints2, None, color=(0, 255, 0), flags=0)

plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.imshow(cv2.cvtColor(img1_kp, cv2.COLOR_BGR2RGB))
plt.title('Image 1 with ORB Keypoints')
plt.axis('off')

plt.subplot(1, 2, 2)
plt.imshow(cv2.cvtColor(img2_kp, cv2.COLOR_BGR2RGB))
plt.title('Image 2 (Rotated) with ORB Keypoints')
plt.axis('off')
plt.show()

print(f"Keypoints detected in Image 1: {len(keypoints1)}")
print(f"Keypoints detected in Image 2: {len(keypoints2)}")

3. Feature Matching

Once you have descriptors for keypoints in two images, you can match them to find corresponding points.

Brute-Force Matcher (cv2.BFMatcher())

The BFMatcher (Brute-Force Matcher) takes the descriptor of one feature in the first set and is matched with all other features in the second set using some distance calculation.

import cv2
import numpy as np
import matplotlib.pyplot as plt

# Using the same image1_orig, image2_orig, keypoints1, keypoints2, descriptors1, descriptors2 from above

# Create a Brute-Force Matcher object
# cv2.NORM_HAMMING is used for ORB (binary descriptors)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True) # crossCheck=True for only best matches

# Match descriptors
matches = bf.match(descriptors1, descriptors2)

# Sort them in the order of their distance (best matches first)
matches = sorted(matches, key=lambda x: x.distance)

# Draw top N matches
img_matches = cv2.drawMatches(image1_orig, keypoints1, image2_orig, keypoints2, matches[:20], None, flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)

plt.figure(figsize=(15, 8))
plt.imshow(cv2.cvtColor(img_matches, cv2.COLOR_BGR2RGB))
plt.title('ORB Feature Matching')
plt.axis('off')
plt.show()

print(f"Number of matches found: {len(matches)}")

Further Topics:

  • SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features): Historically very popular, but were patented. SIFT is now patent-free.
  • AKAZE, BRISK, KAZE: Other feature detectors and descriptors.
  • FLANN-based Matcher: Faster matcher for larger datasets.
  • Homography: Estimating transformations between images based on feature matches.
  • RANSAC: Robust method for estimating model parameters from data containing outliers.
  • Object Tracking: Using detected features to track objects across video frames.

Feature detection and description, combined with matching, are cornerstones of many advanced computer vision applications, allowing systems to understand spatial relationships and recognize objects in dynamic environments.