High-throughput intensity diffraction tomography with a computational microscope

Introduction

Quantitative characterization of thick biological samples is a challenging task. Unstained biological cells appear transparent when imaged under a standard brightfield microscope, since only the sample’s absorption information is directly visible. Though techniques based on exogenous labels (e.g. dyes and fluorophores) have been developed , they suffer from the need for staining or labeling using external contrast agents which may alter cellular behavior . Here, we develop a new label-free phase tomography technique, which provides 3D cellular information with intrinsic structural sensitivity. Our technique is fast, motion-free, and easy to implement with a computational microscope platform, in which a standard commercial microscope is modified with an LED array source .

3D phase microscopy techniques can be largely categorized into two classes, interferometry based and intensity-only methods. The most widely used interferometry-based technique is optical diffraction tomography (ODT). In ODT, images are first taken interferometrically to directly record both phase and amplitude information of the scattered field. Next, a tomographic reconstruction algorithm is devised to recover the sample’s 3D refractive index distribution. The interferograms are taken by using either a separate reference path or a common-path interferometer attached to an existing tomography setup . Various tomographic measurement schemes have been developed, including projection measurement by rotating the sample mechanically or with an optical tweezer , varying the illumination angles with a tilting mirror or a spatial light modulator , and through-focus measurement with a mechanical stage . Due to the need for interferometry, ODT typically requires additional specialized and expensive hardware, which is not always compatible to standard microscopes. Naturally, one would prefer a technique capable of leveraging the ubiquitous microscopy platforms with minimal hardware modifications. To this end, there has been a continued interest in intensity-only phase tomography techniques, which perform 3D phase imaging without interferometry.

Intensity diffraction tomography (IDT) refers to a class of 3D imaging techniques that employ tomographic phase reconstruction from intensity-only measurements. One IDT approach combines a defocus-based phase contrast technique and diffraction tomography model to recover 3D phase. The measurement involves taking multiple defocused images while rotating the sample or the illumination. Recent works further incorporate partially coherent illumination using symmetric or asymmetric light source to achieve up to 2$\times$ resolution improvement in the recovered phase. However, changing focus not only requires mechanical scanning, but also increases the acquisition time and data size, both of which are undesirable for high-throughput applications.

An alternative IDT approach extracts 3D phase information using angled illumination without mechanical scanning . In , 3D phase contrast is computed with an algorithm combining lightfield refocusing and differential phase contrast ; however, the results suffer from low-resolution as diffraction effects are neglected. In , a multislice model is proposed to incorporate diffraction and multiple forward scattering, and achieves high-resolution 3D recovery. However, since the multislice model is nonlinear, it necessitates an iterative reconstruction algorithm, which is non-ideal for time-constrained applications. In , an iterative algorithm combining Fourier ptychography and the first Born approximation was proposed. However, their model ignored the interference term between the scattered and unscattered fields. Here, we show that this term is the primary source of 3D phase contrast in IDT measurements.

In this work, we develop a novel linear IDT model that relates the sample’s 3D permittivity contrast to intensity measurements using angled illumination (Fig. 1). Previous efforts formulate the phase-intensity mapping in the 3D Fourier space; this approach unfortunately suffers from stringent sampling requirement for the measurement, resulting in hundreds of defocused images needed in practice. In addition, the reconstruction requires computation and memory intensive 3D deconvolution. Our approach overcomes all these limitations by employing a slice-based framework. The 3D sample is first modeled as a series of 2D slices along the axial direction. We then derive the slice-wise (2D) phase and absorption transfer functions (TF) at different depths for each illumination angle. We show that this framework enables flexible and efficient data acquisition, allowing using arbitrary patterning of the illumination angles and much fewer images required compared to other techniques . Our model fully accounts for the interference between the scattered and unscattered fields. The linearization is achieved via the first Born approximation that considers single scattering and neglects higher order nonlinear effects. Our linear model enables non-iterative 3D reconstruction directly from intensity-only measurements. We demonstrate volumetric reconstructions with closed-form Tikhonov-regularized solutions, implemented using a computation and memory efficient 2D FFT-based algorithm. We demonstrate our technique on both stained and unstained thick biological samples. The imaging performance and limitation is further investigated in the presence of strong multiple scattering. Experiments demonstrate that our technique is robust even for samples with large permittivity contrast.

Theory and method

We employ the first Born approximation to model the light-sample interaction. Given the incident field $f_{i}$, the total field $f$ after propagating through the sample is given by

where $\ast$ denotes 2D convolution. For notational simplicity, we have neglected the microscope’s magnification. The total field at the front focal plane ($z=0$) is $f(\vec{x},0|\vec{u}_{i})=f_{i}(\vec{x},0|\vec{u}_{i})+f_{s}(\vec{x},0|\vec{u}_{i})$, in which the scattered field is simplified using the Weyl expansion of the Green’s function as

where $\widetilde{\cdot}$ represents the 2D Fourier transform (FT); $\widetilde{\Delta\epsilon}(\vec{u},z)$ is the 2D FT of the slice $\Delta\epsilon(\vec{x},z)$ at depth $z$; $\mathscr{F}^{-1}\{\cdot\}$ denotes the 2D inverse FT (IFT); $\vec{u}$ is the transverse frequency variable, and $\eta(\vec{u})=\sqrt{k^{2}-|\vec{u}|^{2}}$ the axial frequency. (3) calculates the total scattered field as the coherent superposition of the scattered field from each slice; inter-slice scattering is ignored as a consequence of the first Born approximation.

Importantly, (2) contains the interference information between the unscattered and scattered field, and can be expanded into four terms: $I=I_{i}+I_{ss}+I_{is}+I_{si}$. $I_{i}=S(\vec{u}_{i})|P(-\vec{u}_{i})|^{2}$ is the constant background intensity. $I_{ss}=|f_{s}(\vec{x},0|\vec{u}_{i})\ast h(\vec{x})|^{2}$ represents squared modulus of the scattered field, which is negligible if the permittivity contrast is small, and can thus be dropped . Hence, the phase and absorption information is mainly contained only in the two cross-terms, $I_{is}=[f_{i}(\vec{x},0|\vec{u}_{i})\ast h(\vec{x})]^{*}[f_{s}(\vec{x},0|\vec{u}_{i})\ast h(\vec{x})]$ and $I_{si}=I^{*}_{is}$. This is also highlighted by the Fourier spectrum of the intensity measurement in Fig. 1. The relation between the sample’s permittivity contrast and the intensity spectrum is thus linear; and the phase and absorption terms are decoupled:

2 Inverse problem

The reconstruction algorithm combines all the intensity images to estimate the sample’s complex permittivity contrast (i.e. phase and absorption) in 3D. Since our forward model enables inferring the full field information, the reconstruction essentially stitches all the Fourier components coherently, akin to the 3D synthetic aperture . To facilitate efficient reconstruction, we propose a slice-wise deconvolution algorithm, in which the phase and absorption are reconstructed slice-by-slice.

Results

Our system consists of a Nikon TE 2000-U microscope with a programmable red (central wavelength 630nm) LED array source (specifications same as ) placed 79mm above the sample [Fig. 1(a)]. An sCMOS camera (Pco.Edge 5.5) is used for image acquisition, which is synchronized in real-time with the LED source via a microcontroller. Since our technique completely removes any mechanical scanning, the acquisition speed is only limited by the camera’s frame rate (up to 50Hz). To achieve the incoherent resolution limit, we acquire data using brightfield LEDs fully covering the NA of the objective used in each experiment. Angle-varying intensity images are captured by sequentially turning on one LED at a time. We conduct experiments using 10$\times$ (0.25 NA, CFI Plan Achro) and 40$\times$ (0.65 NA, CFI Plan Achro) microscope objectives (MO). For the 10$\times$ MO, 89 LEDs are within the brightfield region; whereas 697 LEDs for the 40$\times$ MO. We only use a small subset of the LEDs, as detailed in each experiment section. Different LED sampling patterns are explored and their reconstruction results are compared experimentally, demonstrating the flexibility in data acquisition enabled by our IDT framework. In all the images, the reconstructed permittivity contrast has been converted to the real (phase) and imaginary (absorption) part of the refractive index. The background medium index for all the biological samples are assumed to be 1.33 (i.e. close to water).

We first demonstrate our technique to image a stained spirogyra sample (Fisher Scientific S68786) using the 10$\times$ MO. The sample contains both highly absorbing features (e.g. chloroplasts) and “phase” features (e.g. filaments) orientated in 3D, as seen in the full field-of-view (FOV) brightfield image in Fig. 2(a). We used all of the 89 brightfield images to perform the IDT reconstruction. To demonstrate the best possible axial resolution, the slice spacing is set to be $5\mu m$ in the reconstruction, corresponding to approximately 2$\times$ oversampling axially. We have reconstructed 25 phase and absorption slices equally spaced between $-20\mu m$ and $200\mu m$. The number of images used is about twice the number of unknowns, the underlying linear problem is thus over-determined, albeit ill-conditioned due to the missing axial frequency information that in turn sets the axial resolution.

The reconstruction performance is illustrated on features with different length scales. In Fig. 2(b), we zoom in on a region containing four clustered spirogyras. Our reconstruction demonstrates that axial sectioning can be successfully obtained using our IDT algorithm - features at different depths are clearly distinguished in the reconstruction. In Fig. 2(c), we turn to a spiral structure on a single spirogyra. The helical structure that presents much finer axial features, is successfully reconstructed. Finally, we look at the filaments in Fig. 2(d), which is a “phase sample” that does not provide high contrast in the brightfield image. Using our technique, the filaments are clearly resolved with high contrast in the phase reconstruction. As expected, the absorption reconstruction does not provide much contrast.

2 Imaging of unstained dense cell clusters

Next, we image unstained MCF-7 cancer cells fixed in formaldehyde solution using the 40$\times$ MO (Fig. 3). The sample contains both monolayer cells and dense cell clusters. A qualitative visualization of the sample is performed via phase contrast (PhC) mode (MO: 0.65 NA, CFI Plan Achro), shown in Fig. 3(a). Features imparting longer optical path delay generally produce darker contrast in the PhC image. Halos are present at the boundaries of thick cell regions.

We first use 153 images to reconstruct 22 phase and absorption slices with $1\mu m$ spacing from $-6\mu m$ to $15\mu m$. To demonstrate the versatility of our IDT technique, Fig. 3(b) shows the phase reconstructions of a few cell regions covering both thin and thick features, along with their PhC images for comparison. Subcellular features are correctly reconstructed, matching the PhC images. Features appear darker contrast in PhC are reconstructed with higher refractive index values.

Next, we zoom-in on a region containing a dense cluster of 5 cells with partial axial overlapping [Fig. 3(c)]. Phase reconstructions are shown at 6 slices along with the corresponding PhC images, captured by mechanically adjusting the focus. Since both techniques use high angle illumination that results in similar Fourier coverage, we expect that they should provide similar lateral resolution and axial sectioning capability. This can be qualitatively verified in Fig. 3(c).

Finally, we demonstrate the flexibility and robustness of our technique when different illumination patterns and number of LEDs are used for data acquisition. In Fig. 3(d), we investigate illumination strategies using symmetric and pseudorandom patterns. Each symmetric pattern uses LEDs that are equally spaced along both azimuthal and radial directions. Each pseudorandom pattern is designed such that each quadrant contains the same number of randomly located LEDs. In all cases, we reconstruct the same 22 phase and absorption slices (i.e. in-total 44 unknown slices). With 37 images, the number of measurement is slightly smaller than the number of unknowns. Although the problem is under-determined, the reconstruction only degrades slightly as compared to the ones from using more images (e.g. 105 and 153). When we further reduce the number of images, the reconstruction further degrades, but is still able to recover cellular features. Using the pseudorandom pattern does not make significant difference when using a large number of LEDs (e.g. 105 and 153). As the number of LEDs is reduced, the pseudorandom pattern is observed to achieve slightly better results, likely because it can provide more uniform Fourier coverage.

3 Imaging of strongly scattering sample

Finally, we evaluate our technique when imaging a strongly scattering sample, which contains both highly absorbing and strong phase features with high permittivity contrast. The goal is to investigate the performance of our technique when multiple scattering effects become strong. The influence of multiple scattering to ODT has been investigated extensively . A general observation is that the unaccounted-for multiple scattering leads to under-estimation of the permittivity contrast. The stronger the multiple scattering, the more severe the under-estimation . In IDT, the data is further limited by the intensity-only information. Our forward model only considers single scattering; intensity variations due to multiple scattering essentially act as non-random noise in the inversion, so they can only be partially suppressed by the Tikhonov regularization.

The sample consists of an absorption target (58-198, Edmund Optics) placed underneath a phase target (QPT, Benchmark Technologies), separated axially by $\sim 790\mu m$ [Fig. 4(a)]. The absorption target contains chromium patterns on a glass substrate. The phase target consists of structures of different sizes and heights with refractive index 1.52, which are raised above a glass substrate. All the experiments are performed in air, so the permittivity contrast is large (¿1), violating the requirement of the first Born approximation. To investigate how the reconstruction degrades as the strength of scattering increases, we perform controlled experiments on phase patterns with increasing height (50nm, 100nm, and 200nm), while keeping the same region of the absorption target underneath. To further investigate the influence of sample depths, we repeat the experiments at three different focal planes. The first dataset is taken with the phase target in focus [(A) in Fig. 4]; the second with focus 300nm below (A) [(B) in Fig. 4]; the third with the absorption target in focus [(C) in Fig. 4]. In each experiment, we take 89 brightfield images with the 10$\times$ MO. A sample brightfield image from the on-axis LED is shown in Fig.4(b).

To quantitatively validate these results, we perform simulations on a similar two-layer sample [Fig. 4(d)]. The multislice model is used to simulate the intensity measurements since it can accurately model multiple scattering at this length scale . We then use our IDT model to perform the reconstruction. As seen in the recovered images, the amount of height under-estimation in each case matches well with the corresponding experimental results. A similar level of background noise is also observed.

Conclusion

We have demonstrated a new computational microscopy technique that enables scan-free, 3D phase and absorption reconstruction using intensity-only measurements. Our slice-based intensity diffraction tomography framework allows flexible illumination patterning for data acquisition and a direct inversion algorithm that has both low computational complexity and is memory efficient. We have experimentally validated this technique on dense 3D biological samples. Although our model only accounts for single scattering, the reconstruction is robust for samples with high permittivity contrast. The effects of multiple scattering are examined via both simulation and experiment, and their results have been found to be in good agreement. The system is simple, and built directly on a commercial microscope with an LED array to enable rapid illumination-angle scanning. This is particularly attractive for the wide adoption of our system to existing microscopy facilities and may open up many biomedical imaging applications.

Funding

National Science Foundation (NSF) Industry/University Cooperative Research Center for Biophotonic Sensors and Systems (IIP-1068070).

Acknowledgments

The authors thank Ziji Liu for the help with the LED array.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

References