Thermal Camera, with3D Software Lens

Chating with AI on a topic to figure out if it would be posible to build a thermal camera with an single mlx90614 dci temperature sensor. And optimize the resolution by means of mathematical computations produced the folowing result:

The 3D Software Lens: Building a Single-Pixel Thermal Camera with ToF Laser and Richardson-Lucy Deconvolution

When engineers set out to build a DIY thermal imaging camera, they usually resort to expensive microbolometer arrays or cheap, low-resolution thermopile grids. But what if you could combine the absolute temperature accuracy of an industrial pyrometer with high-resolution imaging—without spending thousands of dollars on custom Germanium optics? The answer lies in a powerful synergy of analog precision, a Time-of-Flight (ToF) laser sensor, and astronomical mathematics: the Richardson-Lucy algorithm.

Modern thermal cameras are highly impressive, but for critical electronics applications, they suffer from fundamental drawbacks. They rely on microbolometers: focal plane arrays consisting of thousands of microscopic, temperature-sensitive resistors on a single silicon die. Because no two pixels are perfectly identical, they suffer from Fixed Pattern Noise (FPN). Furthermore, these sensors drift constantly due to their own self-heating, requiring frequent recalibration via a noisy mechanical shutter that freezes the image frame every few seconds.

A single-pixel thin-film thermopile is completely immune to these issues. Operating on the Seebeck effect, this instrument measures a direct, analog voltage and exhibits extreme long-term stability. Because a single sensor scans the entire scene, spatial uniformity is perfect—eliminating pixel-to-pixel variations, crosstalk, and delivering superior absolute temperature accuracy. The catch? It only captures a single point at a time through a relatively wide, conical field of view.

The classic workaround is mechanical pan/tilt scanning. However, to resolve a sharp image naturally, the Field of View (FOV) must be heavily restricted. Physical apertures or pinhole masks block so much incoming infrared energy that the remaining signal gets completely drowned out by the noise floor. Furthermore, true Synthetic Aperture techniques (like those used in radar) are impossible here, as passive thermal radiation is completely incoherent and lacks phase information.

Our solution? We keep the sensor aperture wide open to maximize signal strength, utilize a ToF laser sensor to map the exact distance per pixel, and leverage the Richardson-Lucy algorithm to act as a dynamic, depth-dependent "software lens" to mathematically reconstruct a razor-sharp image.

+------------------+      +------------------+      +------------------+

|  Infrared Source | ---> |Thermopile Sensor | ---> |   Low-Noise LNA  |

+------------------+      | (MLX90614-DCI)   |      | (AD8237, G=100)  |

                          +------------------+      +------------------+

                                   ^                         |

                                   | (Pan/Tilt)              v

+------------------+      +------------------+      +------------------+

|   Laser Distance | ---> |  ToF Laser Sensor| ---> |    24-bit ADC    |

|   (3D Surface)   |      |    (VL53L1X)     |      |    (ADS1220)     |

+------------------+      +------------------+      +------------------+

                                   ^                         |

                                   |                         v

                          +--------------------------------------------+

                          |      Microcontroller (Teensy / ESP32)      |

                          +--------------------------------------------+

                                   |

                                   v (USB Serial: Data + Distance per pixel)

                          +--------------------------------------------+

                          |     PC / Raspberry Pi (Python + 3D-RL)     |

                          +--------------------------------------------+

As a thermopile surveys a scene, its sensitivity profile projects outwards as a three-dimensional cone. In signal processing, the cross-section of this cone is defined as the Point Spread Function (PSF).

If a warm object is positioned 1 meter away, the sensor's physical footprint on that object is relatively small. If the object sits at 3 meters, the footprint triples in size, resulting in three times the blur. A standard software deblurring filter that assumes a uniform, static blur profile will completely fail in a real-world environment containing depth variations (such as a chair in the foreground against a distant wall).

By mounting a multizone ToF sensor (such as the VL53L8CX) coaxial to the thermopile, we can map the precise distance (Z-coordinate) of every single pixel simultaneously with its temperature. This maps our processing domain away from static 2D filtering into the realm of Space-Variant 3D Deconvolution.

We utilize a high-grade thermopile capsule (or the raw analog output of a Melexis MLX90614-DCI). The minute microvolt signals (uV} are routed directly to an AD8237 precision instrumentation amplifier. Featuring a zero-drift architecture, this chip continuously cancels out its own input offset voltages via an internal clocking mechanism.

                       +5V (Ultra-Low-Noise LDO: LT3042)

                         |

                       [100nF]

                         |

  Thermopile+ ───[ 1k ]───┤7 (+IN)

                          |               8 (OUT)

                          |  AD8237      ├───> To AIN0 (ADS1220)

  Thermopile- ───[ 1k ]───┤6 (-IN)       |

                          |               |

             REF (2.5V) ──┤1 (REF)        |

                          |               |

                          |     [R_FB]    |

                          5 (FB) ──┴──[R_G]───┴─── Gnd

  

  * Gain = 1 + (R_FB / R_G). Configured for G = 100.

The amplified signal is digitized by an ADS1220, a 24-bit Delta-Sigma ADC. This ultra-high resolution allows us to resolve thermal deltas as fine as 0.01 °C.

The sensor payload (thermopile + VL53L8CX ToF) is carried by a rigid pan-tilt assembly driven by two NEMA 17 stepper motors paired with 1:14 planetary gearboxes. This completely isolates the system from mechanical backlash.

The motors step at precise 0.5° increments per pixel, whereas the thermopile exhibits a Full Width at Half Maximum (FWHM) acceptance angle of 5°. This massive oversampling guarantees a 90% spatial overlap between adjacent measurements. The microcontroller (e.g., an ESP32) orchestrates a strict stop-and-go sampling cadence: step the gantry, pause for 40 ms (allowing structural vibrations to settle and the thermopile membrane to stabilize), poll the ToF distance over I2C/SPI, sample the ADC, and stream the formatted packet to the host PC.

On the host PC, we reconstruct two matching matrices: a matrix of raw, blurred thermal intensities (g) and a matrix of precise spatial distances (Z) per pixel coordinate.

The classic Richardson-Lucy algorithm recovers a degraded image through an iterative feedback loop governed by a static PSF (h):

            f(k+1 = f(k) ) x (  (g/( f(k) * h) * h^T)        (* = convolution)

In our 3D implementation, h is no longer treated as a constant matrix. For every individual pixel coordinate (x,y), we compute the localized spatial variance Sigma()  of the Gaussian PSF based on the actual ToF measurement Z(x,y):

       Sigma(x,y) = (FWHMrad x z(xy)) / (2.3548 x StepSize_meters) 

                        (FWHM = Full Width at Half Maximum) 

 

Because the sensor sweeps along a spherical trajectory, scanning a flat wall directly ahead results in non-uniform distances—the corners of the room are physically further away than the center. The ToF sensor catches this geometric inflation natively. Using basic trigonometry (sin/cos), the software normalizes this projection distortion, scaling the operational PSF footprint dynamically as the target boundary recedes.

The following Python script demonstrates how to utilize a ToF distance matrix to apply a dynamically shifting PSF throughout the deconvolution routine.

Python

import numpy as np

import matplotlib.pyplot as plt

from scipy.ndimage import gaussian_filter

from skimage import restoration

 

def get_local_psf(distance_mm, fwhm_deg=5.0, step_size_deg=0.5, matrix_size=21):

    """

    Computes a localized PSF matrix based on the ToF distance.

    Greater distances yield a larger sigma (increased blur footprint) in pixel space.

    """

    # While the physical angular cone remains constant, the projected spot size

    # scales linearly with target distance.

    scaling_factor = distance_mm / 1000.0  # Scale to meters

    

    sigma_deg = fwhm_deg / 2.3548

    sigma_pixels = (sigma_deg / step_size_deg) * scaling_factor

    

    # Bound sigma to avoid zero-division or errors at extreme macro distances

    sigma_pixels = max(sigma_pixels, 0.5)

    

    psf = np.zeros((matrix_size, matrix_size))

    psf[matrix_size // 2, matrix_size // 2] = 1.0

    psf = gaussian_filter(psf, sigma=sigma_pixels)

    return psf / psf.sum()

 

# ==========================================

# PROCESSING PIPELINE OVERVIEW

# ==========================================

 

# Instantiate matrices (e.g., a 64x64 scan array)

# raw_thermal_matrix = ... (Incoming raw ADC datasets)

# tof_distance_matrix = ... (Incoming ToF ranging data in mm)

 

# To optimize computational complexity in space-variant deconvolution, 

# we partition the depth map into localized distance bands.

# Below is the conceptual workflow for an extracted depth region:

 

mean_distance = np.mean(tof_distance_matrix)

dynamic_psf = get_local_psf(mean_distance, fwhm_deg=5.0, step_size_deg=0.5)

 

# Execute the iterative deblurring routine using the depth-aware cone profile

sharpened_image = restoration.richardson_lucy(raw_thermal_matrix, dynamic_psf, num_iter=20)

 

 

By shifting the burden of optical convergence entirely from physical lenses over to Computational 3D Imaging, smart sensor fusion can bypass traditional hardware limitations. Integrating a ToF laser sensor elevates a classic single-pixel scanner into a robust, radiometrically precise thermal imaging device that inherently understands depth of field. Armed with a single thermopile capsule, a low-cost ToF chip, and a few lines of iterative matrix math, you can assemble a thermal imager that rivals expensive commercial core matrices in spatial uniformity and absolute accuracy.