Accelerating 4D Hyperspectral Imaging via Neural Representation and Adaptive Sampling

Teaser: Hyper-spectrum reconstruction through neural rendering

We sparsely sample the 4D hyper-spectrum from ultrafast molecular interactions and use a physics-informed neural representation to reconstruct dense spectra — achieving up to 32× speedup in data acquisition.

Abstract

High-dimensional hyperspectral imaging (HSI) enables the visualization of ultrafast molecular dynamics and complex, heterogeneous spectra. However, applying this capability to spatially-resolved two-dimensional infrared (2DIR) spectroscopy necessitates prohibitively long data acquisition, driven by dense Nyquist sampling requirements and the need for extensive signal accumulation.

We introduce a physics-informed neural representation that efficiently reconstructs dense spatially-resolved 2DIR hyperspectral images from sparse measurements. A multilayer perceptron (MLP) learns the continuous mapping between sub-sampled 4D coordinates and their spectral intensities, faithfully recovering both oscillatory and non-oscillatory spectral dynamics. Complementing this, we develop a loss-aware adaptive sampling algorithm that progressively focuses measurements on high-uncertainty regions during data collection.

Experimental results demonstrate high-fidelity spectral recovery using only 1/32 of the sampling budget, reducing total experiment time by up to 32×. This framework offers a scalable solution for any hypercube experiment, including multidimensional spectroscopy and hyperspectral imaging.

Overview

Spatially-resolved 2DIR and the four-dimensional measurement space.

What is 4D 2DIR?

Spatially-resolved 2D infrared spectroscopy extends the standard HSI framework into the nonlinear regime, encoding molecular couplings and ultrafast dynamics across four acquisition dimensions:

Coherence time (t₁): Pump–pump delay; Fourier transformed to yield the pump frequency axis ω₁.
Population time (t₂): Waiting time between pump and probe; encodes vibrational relaxation and dynamics.
Detection frequency (ω₃): Natively resolved by a grating spectrometer.
Spatial axis (x): Line-scan position via a monochromator slit.

While ω₃ and x are multiplexed on the detector, t₁ and t₂ must be exhaustively scanned — making them the bottleneck our method targets.

System diagram of spatially-resolved 2DIR setup

System diagram showing the pump-probe geometry. Pump pulses (controlled via pulse shaper) and probe pulses are overlapped on the sample. The emitted signal is spatially and spectrally dispersed onto a focal-plane array.

Illustration of four acquisition dimensions in 4D 2DIR

Four-dimensional measurement space. Each 2DIR frame is parameterized by (ω₁, ω₃), while temporal and spatial variations are encoded along population time t₂ and spatial position x.

Method

Physics-informed neural field reconstruction with frequency-domain regularization.

MLP-based neural representation architecture

Neural representation model. Population branch: a coordinate-based MLP directly maps 4D coordinates to intensities for smooth t₂ dynamics. Coherence branch: the MLP outputs frequency-domain spectra; an Inverse Real Fourier Transform (IRFT) recovers time-domain estimates for loss computation, exploiting the MLP's low-frequency bias as a natural regularizer for oscillatory signals.

Key design choices

Coordinate-based MLP: A 4-layer, 64-neuron network with ReLU activations maps normalized (x, t₁, t₂, ω₁, ω₃) coordinates to spectral intensities — no predefined grid, resolution-independent.
IRFT-based optimization for coherence: Rather than fitting rapidly oscillating t₁ signals directly, the MLP learns smoother frequency-domain spectra. An IRFT converts predictions back to the time domain for sparse-measurement supervision.
Statistical moment constraints: Spatial moment-matching losses (mean position μ, standard deviation σ) preserve spatially resolved dynamics alongside intensity reconstruction.
Monotonicity & smoothness regularization: Hinge-type penalties encourage physically consistent, monotonically evolving spatial variance across population time.

2D projection of 4DIR data for evaluation

4D-to-2D projection. A bounding box is defined around the dominant spectral peak. Intensities within the half-maximum neighborhood are averaged, reducing the 4D spectra to a 2D spatial-temporal profile M(x, t₂) used for moment-matching and evaluation.

Results

Neural reconstruction from sparse measurements across all four acquisition dimensions.

Reconstruction across repeated experiments. Our method consistently recovers high-fidelity spectra across five independently collected datasets, demonstrating robustness to noise and measurement variability.

Reconstruction results along population time t2

Population time (t₂) reconstruction. Sparse sampling along the smooth t₂ axis; the MLP interpolates unsampled dynamics faithfully.

Reconstruction results along coherence time t1

Coherence time (t₁) reconstruction. IRFT-based optimization recovers rapidly oscillating coherence dynamics from highly undersampled measurements.

Coherence-axis comparison against baselines. Our method outperforms compressed sensing and low-rank alternatives, especially at low sampling rates where oscillatory signals are difficult to recover.

Performance vs. sampling rate (coherence axis). Quantitative comparison showing our advantage grows at lower budgets.

Reconstruction under different accumulation counts

Effect of signal accumulation count r. The neural representation denoises effectively even at low r, further reducing experiment time.

Ablation study. Removing the IRFT branch, moment-matching loss, or regularization terms each degrades reconstruction quality, validating the contribution of each component.

Adaptive Sampling

Loss-driven progressive measurement allocation along the population time axis.

Algorithm overview. Starting from an initial sparse set of measurements, the MLP is trained and per-time reconstruction errors are computed. The interval with the highest average prediction error is identified and its midpoint is selected as the next acquisition coordinate — focusing effort where uncertainty is highest.

How it works

Initialize: Begin with a coarse uniform grid of t₂ samples.
Train & evaluate: Fit the MLP to current measurements; compute the per-sample reconstruction loss L(t_2,k) by aggregating error across spatial and frequency dimensions.
Select next sample: Find the time interval with the highest average error; query the experiment at its midpoint.
Repeat: Update the model with the new measurement and iterate until the budget is exhausted.

This interpretable, greedy strategy requires no external training data and naturally adapts to the dynamics of each individual sample.

Sampling strategy comparison. Adaptive sampling allocates measurements to the most informative regions, yielding better coverage of dynamic events compared to uniform or random strategies.

Reconstruction results with adaptive sampling

Adaptive sampling reconstruction results. Under the same total measurement budget, adaptive sampling produces higher-fidelity reconstructions of population-time dynamics than uniform or random baselines.

BibTeX

@article{ho2026neural4dir,
  title={Accelerating 4D Hyperspectral Imaging through Physics-Informed Neural Representation and Adaptive Sampling},
  author={Ho, Chi-Jui and Bhakta, Harsh and Wei, Xiong and Antipa, Nicholas},
  year={2026}
}