Keypoints and Feature Descriptors in OpenCV

The field of computer vision is constantly growing. Not only in active research, but also in real inventions and applications, computer vision and image processing have gained a lot of traction in recent years.

When somebody is interested in developing applications in this area, the place to go for almost 10 years has been the OpenCV library. With the aim of providing real-time computer vision and image processing capabilities, OpenCV has become the standard in research, hobby programming, and commercial products.

One of the strengths of OpenCV is the broad built-in functionality for feature detection, feature description, and feature matching. The term keypoint is often used interchangeably with feature. Feature detection is critical for applications such as image stitching. In image stitching, one combines multiple pictures from several viewpoints into a single comprehensive image. More recently, the same concept has been transferred from 2D to 3D, where one tries to stitch 3D point clouds together instead of plain images.

OpenCV is a huge help in solving such tasks, and the developer community has made this easier by introducing common interfaces for feature detectors, feature descriptor extractors, and feature matchers.

Especially for developers starting in computer vision, it can be time-consuming to read all relevant papers that introduce different algorithms and techniques for setting up a complete pipeline:

feature detector -> feature descriptor -> feature descriptor matcher

Despite the help of common interfaces in OpenCV, it can still be hard to get the broader picture of what each algorithm solves in the keypoint pipeline, and what it does not. Some algorithms like SIFT can be used for both feature detection and feature description, while others like FAST only work as keypoint detectors.

I had the same issues, so I decided it could be helpful to list algorithms and use cases available in OpenCV for image registration.

Legend

• 1 = keypoint detector and descriptor extractor
• 2 = only keypoint detector
• 3 = only keypoint descriptor
• 4 = descriptor extractor
• 5 = descriptor matcher
• o = free to use
• x = non-free (patented)
• e = experimental (v3.1.0)
• n = non-experimental (v3.1.0)

Algorithms in OpenCV (Image Registration)