Moiré Vision: A Signal Processing Technology Beyond Pixels Using the Moiré Coordinate

Nowadays, many computer vision (CV) techniques require high-precision feature point detection in calibration targets. Checkerboards or circle grids are common patterns used to infer the locations of the points directly from the image. However, such approaches can lead to degradation in sub-pixel accuracy for low-resolution or sub-sampled sensor data (e.g., Bayer pattern). This paper proposes a target-based calibration technique applicable to various conventional CV methods, leveraging the moiré phenomena. We use a moiré pattern that is created between the digital camera’s pixels and the high-frequency texture on the calibration target. Even a tiny movement of the target or camera leads to a significant motion of the moiré, acting as a magnifier of the target texture for estimating accurate location of targets or camera. In contrast to previous moiré approaches that employ frequency domain analyses, we introduce the moiré coordinates and derive them from the difference between the sensor and target texture position. In the moiré coordinate, each moiré pattern is undistorted. Unlike moiré in standard images, each moiré pattern has almost the same scale and shape in these coordinates. Utilizing this property, we propose a straightforward algorithm to detect and handle correspondence in the moiré peaks in image and texture location, treating them as feature points of calibration target. In addition, the moiré vision can magnify pixels, which means the pattern in the moiré coordinate visualizes the pixel sensitivity with high spatial resolution. To validate the proposed approach, we present an empirical evaluation using simulated and real Bayer sensor patterns. We show the advantages of the proposed method in terms of accuracy and identify promising future directions.