2.1.

Volumetric PACT with High-Speed and Isotropic Resolutions

PAT is inherently capable of volumetric or three-dimensional (3D) imaging, benefiting from the time-resolved detection of the acoustic waves that provide the depth information of the targets. For PAM, in which a single laser pulse generates a 1D depth-resolved image, 2D raster scanning is employed to obtain a 3D image; for PACT with a 1D transducer array, in which a single laser pulse generates a 2D cross-sectional image, orthogonal scanning along the elevational direction is needed to obtain a 3D image. We will discuss the new developments in PAM in a later section, but here we focus on volumetric PACT. While traditional volumetric PACT has been widely used for functional brain imaging, small-animal whole-body imaging, and breast cancer diagnosis in humans, the major drawbacks include the long imaging time needed for mechanical scanning and the anisotropic spatial resolution (much worse elevational resolution) determined by the cylindrical focusing of the transducer elements.

To accelerate the speed and improve resolution symmetry of volumetric PACT, recent efforts have concentrated on applying 2D ultrasound transducer arrays coupled with high-power laser sources and 3D image reconstruction. For PACT with a 2D transducer array, a single laser shot can theoretically generate a 3D image,^42,43 and the resolutions can be nearly isotropic at the center of the FOV or the well-resolved FOV. In practice, however, 2D transducer arrays typically lack enough active elements to satisfy the spatial Nyquist sampling over a large volume, limited by the transducer fabrication complexity and the number of data acquisition channels. Thus, multiplexed data acquisition (electronic scanning)⁴⁴ and rotational scanning⁴⁵ are typically needed to improve the spatial sampling density. Moreover, because repeated wide-field illumination may cause tissue damage due to accumulated heating,⁴⁶ the optical fluence ($\mathrm{J}/{\mathrm{m}}^2$) per pulse and the average fluence rate ($\mathrm{W}/{\mathrm{m}}^2$) on the tissue surface need to be carefully controlled.⁴⁶

Different types of 2D transducer arrays have been explored for volumetric PACT, mostly based on piezoelectric materials, as summarized in Table 1. To maximize the detection aperture, several groups have explored the feasibility of spherical ultrasound transducer arrays, usually with the transducer elements sparsely distributed over the array surface.^47,49,50 Compared with the planar 2D array,^53,54 the spherical array can provide higher spatial sampling density around its center volume and better visualization of 3D structures with different orientations. Matsumoto et al.⁴⁷ developed a volumetric PACT system using a sparse hemispherical detector array that is scanned in a spiral pattern [Fig. 2(a)], which can provide detailed blood oxygenation images on the breast skin surface but suffers from slow imaging speed and low penetration depth [Figs. 2(c) and 2(d)]. Similarly, Schoustra et al.⁵⁰ upgraded the Twente Photoacoustic Mammoscope using 12 arc-shaped transducer arrays arranged over a hemi-spherical surface, which can provide 3D vascular images of healthy breasts within four minutes. To accelerate the imaging speed, the Razansky group has developed a volumetric PACT system using a 2D ultrasonic transducer array with 256 elements densely arranged on a partial cup, which has achieved a 3D imaging rate of $\sim 50\mathrm{Hz}$.⁴⁸ This system has been implemented in both desktop and handheld configurations^55,56 and has been applied to capture in real time the heart beating of a mouse and the neuronal activities of a swimming zebrafish and a GCaMP-expressing mouse brain.^42,55,57 However, without performing additional scanning, such a transducer arrangement results in a small well-resolved FOV ($\sim 4\text{-}\mathrm{mm}$ diameter) and can only be used to study small animal organs, such as the heart and brain. Moreover, the limited view aperture (i.e., $<2\pi$ steradian solid angle) can further compromise the image quality when a higher imaging speed is required.

Table 1

Comparison of representative 2D ultrasound arrays in volumetric PACT.

Fig. 2

Volumetric PACT with 2D ultrasound array. (a) Schematic of the hemispherical array with 512 elements sparsely arranged over the sensor surface.⁴⁷ (b) Schematic of the quad-arc array with 1024 elements densely arranged along four arcs (with a separation of 90 deg) that are mounted on a hemispherical surface.⁴⁹ (c)-(d) Projection images of the human breast vascular oxygenation obtained by the volumetric PACT in (a), with a total scanning time of 120 s. (e)–(f) Projection images of the human breast vasculature obtained by the volumetric PACT in (b), with a total scanning time of 10 s. Adapted with permissions from Refs. 47 and 49.

To simultaneously improve the spatial sampling and imaging speed over a large FOV, the Wang group has reported a novel design with a quad-arc-shaped 2D transducer array, which has 1024 elements and one-to-one mapped signal amplification and data acquisition [Fig. 2(b)].⁴⁹ By rotating the quad-arc-shaped array by 90 deg, the volumetric PACT system can provide a large well-resolved FOV ($\text{diameter}>100\mathrm{mm}$) and $\sim 2\pi$ steradian solid angle, with nearly isotropic resolution of 370 to $390\mu \mathrm{m}$. It takes only 2 to 10 s to generate a volumetric image, depending on the targeted FOV, which is much faster than the previously reported systems. The newly developed volumetric PACT system has been applied for imaging a human breast within a single breath hold [Figs. 2(e) and 2(f)]. So far, this is the volumetric PACT system with the largest well-resolved FOV and the highest speed. Nevertheless, the imaging speed can be further improved by adopting pulsed lasers with a higher repetition rate ($>10\mathrm{Hz}$) as well as faster rotational scanning stages. Meanwhile, functional and molecular imaging capability remains to be demonstrated with high-speed, wavelength-tunable light sources. However, to comply with the laser safety standard,⁴⁶ a higher laser repetition rate would lead to a lower maximum permissible exposure ($\mathrm{mJ}/{\mathrm{cm}}^2$) on the tissue surface, and thus a lower signal-to-noise ratio. In other words, a higher imaging speed in volumetric PACT may come at the cost of the final image quality and penetration depth.

The imaging characteristics of several representative volumetric PACT systems are summarized in Table 2.

Table 2

Comparison of representative volumetric PACT systems.

In addition to 2D ultrasound transducer arrays based on piezoelectric materials, optical ultrasound detectors, such as the Fabry–Perot interferometer,^58,59 micro-ring resonator,⁶⁰ and Bragg grating fiber,^61–63 have been actively explored for volumetric PACT. Compared with piezoelectric transducers, optical ultrasound detectors often have smaller size ($<100\mu \mathrm{m}$), larger detection bandwidth and receiving angle, higher detection sensitivity per unit area, better transmittance of the PA excitation light, and simpler PA signal readout, all of which can help improve the resolution and penetration depth of volumetric PACT. We will discuss the developments in optical ultrasound detectors in a later section.

2.2.

High-Speed Photoacoustic Microscopy over a Large FOV

Biological functions occur on a wide range of temporal and spatial scales, which requires imaging technologies to provide best-matched imaging speeds and FOVs. For example, a single neuron action potential lasts for 1 to 2 ms along a $10\text{-}\mu \mathrm{m}$-diamter axon, neurovascular coupling happens within hundreds of milliseconds over a functional circuit with a $100\text{-}\mu \mathrm{m}$ radius, and the resting-state functional connectivity between the brain’s sub-regions occurs within tens of seconds over a millimeter-level radius. Configured to work in 1D, 2D, or 3D imaging modes, different implementations of PAT offer a wide range of imaging speeds with associated tradeoffs.⁶⁴ In this section, we will focus on new developments in PAM that can offer high imaging speed, large FOV, and functional imaging capability.

For PAM, different scanning mechanisms can be employed according to the desired imaging speeds.⁸ Unlike confocal or two-photon microscopy, PAM does not require depth scanning for 3D imaging due to its time-resolved acoustic detection. When high-speed imaging is needed in OR-PAM, the focused excitation laser beam can be raster-scanned within the acoustic focal spot ($\sim 50\mu \mathrm{m}$ in diameter), which largely confines the FOV to single vessels.^65,66 Alternatively, cylindrically focused or unfocused acoustic detection can enlarge the FOV—up to $\sim 40\mathrm{mm}$ in diameter as demonstrated thus far—at the expense of detection sensitivity.^67,68 In order to achieve a high detection sensitivity over a large FOV, it is critical to maintain the confocal alignment of the optical excitation and acoustic detection. Recently, 1D or 2D water-immersible resonant MEMS (microelectromechanical systems) scanning mirrors that confocally steer both the excitation laser beam and the emitted acoustic beam⁶⁹ have achieved a 2D imaging rate of 500 Hz and a 3D imaging rate of $\sim 1\mathrm{Hz}$, with a moderate FOV of $\sim 3\times 3{\mathrm{mm}}^2$ and uncompromised detection sensitivity.^70–72 By using a pulse-width-based, single-wavelength method or a Raman-shifter-based, two-wavelength method, MEMS-scanning OR-PAM can monitor the change in blood oxygenation of mouse brain in vivo.^71,73 However, it is challenging for the resonant MEMS scanning mirrors to provide a larger FOV without sacrificing the system’s detection sensitivity, and the scanning range drops sharply when the scanning frequency deviates from the resonant frequency.

To address the tradeoff between the scanning speed and scanning range of MEMS scanners, a recent work by Lan et al.⁷⁴ has reported the use of a water-immersible polygon mirror scanner in OR-PAM that has achieved a 2D imaging rate of 1.2 kHz over a 12-mm scanning range and a 3D imaging rate of 1 Hz over a $12\times 12{\mathrm{mm}}^2$ FOV [Fig. 3(a)]. The polygon scanner with six facets is driven by a rotational DC motor, with each rotation providing six repeated 2D scans. Unlike the resonant MEMS mirror, the polygon scanner can maintain its large scanning range at different scanning frequencies, which is critical for imaging large organs, such as the blood oxygenation change of the whole mouse cortex [Fig. 3(b)]. By combining the polygon scanner with a Raman-shifter-based, two-wavelength laser, Chen et al.⁷⁶ have demonstrated high-speed functional imaging of the hemodynamic response of the entire mouse ear to epinephrine, a commonly used vasoconstrictor. Nevertheless, one drawback of the polygon scanner is the lack of adjustment of its scanning range. Once the optical path is constructed, the scanning range is determined and difficult to change, which poses a waste of scanning time on small targets. Different scanning mechanisms used in high-speed OR-PAM are compared in Table 3.

Fig. 3

High-speed PAM with novel scanning and non-scanning approaches. (a) Schematic of the high-speed PAM using a water-immersible polygon scanner, in which a single rotation of the polygon scanner provides six repeated 2D scans.⁷⁴ UT, ultrasound transducer. (b) High-speed imaging of the mouse brain under hypoxia challenge obtained by the system in (a), showing reduced blood oxygenation. (c) Schematic of the high-speed PAM using wide-field light illumination and a single-element ultrasound detector through an ergodic relay.⁷⁵ (d) High-speed tracking of arterial pulse wave obtained by the system in (c), showing the heated blood (coded in color) flowing in the vessels. Adapted with permissions from Refs. 74 and 75.

Table 3

Comparison of scanning mechanisms in OR-PAM.

For AR-PAM, the imaging speed is mainly limited by the mechanical scanning speed and the pulse repetition rate of the high-pulse-energy laser, the latter of which is limited by laser safety on the tissue. In AR-PAM, mechanical scanning by a step motor or a voice-coil scanner can be used, with a scanning step size $\sim 10$ times that used in OR-PAM.⁸ A 2D imaging rate of 40 Hz has been achieved by AR-PAM over a scanning range of $\sim 9\mathrm{mm}$, sufficient to capture the oxygenation dynamics in a mouse heart within a heartbeat.⁸¹ Recently, the 2D water-immersible MEMS scanning mirrors have also been adapted to improve the imaging speed of AR-PAM by $\sim 10$-fold, with an FOV of $2\times 2.5{\mathrm{mm}}^2$.^82–84 When integrated with additional mechanical scanning to “stitch” the MEMS scanning area, AR-PAM can image a $30\times 30{\mathrm{mm}}^2$ area within 70 s.

While the above fast-scanning–based approaches have significantly improved the imaging speed of PAM systems, they are fundamentally limited by the laser’s pulse repetition rate when the spatial Nyquist sampling needs to be satisfied. This limitation is particularly true for high-speed OR-PAM, which often requires a small scanning step size of $<2\mu \mathrm{m}$. For example, for the recently published polygon-scanner–based PAM system,⁷⁴ the laser’s maximum pulse repetition rate is 800 kHz and the B-scan (i.e., the fast-scanning axis) rate can reach as high as 2000 Hz over a 10-mm scanning range. However, to satisfy the Nyquist sampling theorem, the B-scan rate is limited to only 200 Hz if the FOV is kept the same, much lower than the maximal achievable speed. One way to increase the scanning speed over a large FOV is to increase the scanning step size at the cost of effective spatial resolution. Sparse sampling has thus become a necessary compromise when imaging speed needs to be increased.⁸⁵

To relax the requirement on the laser’s pulse repetition rate, one solution is non-scanning PA imaging based on an ergodic relay, which can simultaneously encode all of the PA signals from a large FOV according to their unique time-delay characteristics.^75,86–88 In a recent work, Li et al.⁷⁵ demonstrated a high-speed implementation referred to as photoacoustic topography through an ergodic relay (PATER). In PATER, for each single excitation laser pulse, the encoded PA signals can be detected in parallel via a single-element ultrasound transducer and then decoded mathematically to reconstruct a 2D projection image [Fig. 3(c)].⁷⁵ With a point-by-point scanning calibration step, PATER has demonstrated a topographic frame rate of 2 kHz over a field of view of $6\times 7.5{\mathrm{mm}}^2$, and has been applied to image the blood pulse wave velocity and track the circulation of melanoma cells in the mouse brain [Fig. 3(d)]. Because no optical or acoustic beam scanning is needed in PATER, the imaging speed is essentially limited by the acoustic transit time within the ergodic relay. Nevertheless, the current calibration method lacks the depth information and thus only topographic images can be provided.

Limited by slow imaging speed and bulky system size, desktop PAM is mostly applied on small animals under anesthesia or human subjects with the targeted region fixed (e.g., arm, hand, or finger) to minimize the motion artifacts. Enabled by the elevated imaging speed and the resultant high imaging throughput, it has become possible to implement miniaturized PAM to image otherwise challenging targets prone to motion artifacts, such as brain functions of freely moving animals, longitudinal monitoring of rare circulating tumor cells of melanoma patients, and skin cancer screening of difficult regions such as the neck and back. In recent years, various PAM systems have been developed for handheld,^89–91 wearable,^92–94 and even head-mounted applications,^95,96 thanks to the advances in high-speed scanning methods. All these technical innovations have allowed the miniaturization of PAM systems without sacrificing the imaging performance. For example, to capture normal brain functions, it is critically important to record the neural activities in freely behaving animals with high resolution and high throughput. Chen et al.⁹⁵ have reported a wearable PAM system that is small enough to be mounted on the head of a freely moving rat. A miniaturized MEMS scanning mirror provided high-speed, high-resolution imaging of the brain’s hemodynamic activities during and post ischemia challenge. Remarkably, the motion artifacts were negligible during the 90-min imaging time.

2.3.

Optical Detection of the Ultrasound Pressure

Piezoelectric ultrasound transducers still largely dominate the PAT technologies due to their wide availability, high detection sensitivity, low fabrication cost, and ease of use. However, optical ultrasound sensors have their unique advantages for PAT and have gained more momentum in recent years.²¹ Unlike ultrasonography, PAT does not need ultrasound transmission, and the PA signals are usually broadband, so optical sensors can be used in receiving-only mode, taking full advantage of their small size, large receiving angle, wide detection bandwidth, strong responsivity in the low frequency band, and good compatibility with PA light path. More importantly, the detection sensitivity of optical sensors usually has less dependence on the sensor size, which leads to better sensitivity than piezoelectric transducers of the same size, especially at higher frequencies ($>2.5\mathrm{MHz}$).⁹⁷ Practically speaking, the optimal size of piezoelectric transducers used in PACT is equivalent to a half-wavelength on the FOV boundary. Further size reduction provides no clear benefit in spatial sampling density or receiving angular directivity.¹⁰ Therefore, when comparing the performance of optical ultrasound sensors with piezoelectric transducers, we suggest half-wavelength sized piezoelectric transducers shall provide a fair comparison unless the application is inherently space-constrained. For example, the optical sensors’ high detection sensitivity with a small form factor is particularly attractive for endoscopic and wearable PAT implementations, in which the working space is extremely limited. A thorough comparison of optical sensors and piezoelectric transducers can be found in the review article by Wissmeyer et al.²¹

So far, there have been two types of optical sensors demonstrated in PAT technologies: interferometric sensors and refractometric sensors. Taking advantage of the optical and acoustic interactions in the PA effect, these optical sensors often probe a single step in the PA signal generation and propagation process. The interferometric sensors typically have better detection sensitivity than the refractometric sensors.²¹ The above-mentioned Fabry–Perot interferometer, micro-ring resonator, and Bragg grating fiber are all interferometric sensors that target the last step in the PA signal propagation and have been applied in volumetric PACT and/or PAM as point-like detectors. Refractometric sensors often exploit the earlier steps in PA signal generation, such as the photothermal or thermoelastic effect in the tissues or coupling medium, and detect the change in probing light beam’s transmission, reflection, or deflection.^98–100 Such changes, however, are usually small. While interested readers are referred to the comprehensive review article on the optical sensors in PAT technologies,²¹ we would like to discuss three important limitations of optical sensors—speed, scalability, and stability—as well as highlight some new studies aiming to address these limitations.

For the PACT systems based on a planar Fabry–Perot interferometer, the nearly isotropic spatial resolutions, approximately defined by the optical probing beam size, can be well maintained with 2D dense spatial sampling over the entire FOV.⁵¹ However, the imaging speed is traditionally limited by the point-by-point raster scanning of the probing beam and the low pulse repetition rate of the PA excitation laser at 50 Hz.^51,101 A more recent work by the UCL group has demonstrated a 32-fold higher imaging speed by employing a total of eight parallel probing beams scanning simultaneously over the sensor [Figs. 4(a) and 4(b)].¹⁰² A customized, high-speed laser (200 Hz) also helps to improve the imaging speed. Such a speed-up strategy, however, has drastically increased the system’s complexity and cost, and the high-speed laser has relatively low pulse energy. Wide-field detection of the interference pattern on the sensor surface, using time-gated light illumination and a high-speed CCD camera, can potentially speed up the imaging as well.^103,104

Fig. 4

PAT with optical ultrasound sensors. (a) Schematic of a PACT system using a Fabry–Perot interferometer with eight parallel probing beams.¹⁰² (b) A 3D human palm image obtained the system in (a) with a total imaging time of 10 s. (c) Schematic and photograph of a micro-ring-resonator using silicon photonic technology.⁶⁰ (d) The optical transmission spectrum of a multiplexed micro-ring-resonator array with 10 sensors. (e) Representative 3D image of three stacked polyamide sutures obtained by the micro-ring resonator in (a). Adapted with permissions from Refs. 60 and 102.

Other optical ultrasound sensors, including polymer micro-ring⁶⁰ and Bragg-grating fiber,⁶¹ have recently been demonstrated as point-like detectors in PAT, often with sensor sizes that are orders of magnitude smaller than their piezoelectric counterparts. However, one major obstacle met by these optical sensors is the extreme difficulty in scaling up the production while maintaining consistent optical properties, such as the optical resonant wavelength, Q factor, and transmission efficiency. Unlike piezoelectric materials that allow the manufacturing of high-density arrays, it is difficult for optical sensors to be multiplexed. Slight fabrication inaccuracy, such as of the Fabry-Perot polymer’s thickness or the micro-ring’s diameter, would drastically change its operating parameters. This is particularly problematic for volumetric PACT, which requires parallel signal detection to improve the imaging speed. To address this issue, Westerveld et al.⁶⁰ have developed a new micro-ring-resonator using silicon photonic technology [Fig. 4(c)]. As a proof of concept, a total of ten resonators can be fabricated onto a single optical bus waveguide [Fig. 4(d)]. This CMOS-compatible fabrication process may provide a viable path for scaling the optical sensor to a 2D array for high-speed volumetric PACT [Fig. 4(c)].

Another significant drawback of optical ultrasound sensors, particularly the interferometric sensors, is low stability in the biological environment. For example, the micro-ring resonator is sensitive to contamination on the sensor surface (e.g., dust, body fluid, or blood stain), which induces scattering and absorption loss, and the Fabry–Perot interferometer is sensitive to the environmental temperature drift, which changes the thickness and refractive index of the polymer spacer. Such instability in the biological environment often leads to fast degradation of the sensor sensitivity and prevents the use in longitudinal in vivo studies. To address this issue, Li et al.¹⁰⁵ have developed a micro-ring resonator by soft nanoimprinting lithography, which has significantly improved stability for in vivo applications. The micro-ring resonator is encapsulated by a protection layer made of both optically and acoustically transparent polydimethylsiloxane (thickness $5\mu \mathrm{m}$). By isolating the micro-ring and waveguide from the potential contaminants (e.g., blood), the micro-ring resonator has demonstrated impressively stable performance when implanted on a mouse cortical surface for 28 days. Similarly, Westerveld et al.⁶⁰ recently demonstrated a micro-ring resonator using a thin layer of acoustic membrane to isolate the ring structure from the environment, which can potentially improve the sensor’s stability in water. To overcome the thermal stability of the Fabry–Perot interferometer, Chen et al.¹⁰⁶ have incorporated an additional heating light source at 650 nm into the interferometer, which can modulate the polymer spacer’s thickness and thus compensate for the temperature-induced resonant spectral shift. Such thermal compensation can be performed in real time by a closed-loop feedback.

2.4.

Deep Learning Enhanced Image Reconstruction and Processing

Like many other technologies, PAT’s developments have been incorporating the fast-evolving deep learning enabled by the prevalence of graphical processing unit (GPU) capabilities.^107–116 Deep learning is well-suited for addressing some long-standing challenges of PAT, such as improving ill-posed reconstruction, removing limited-view artifacts, denoising channel data, improving diffraction-limited spatial resolution, and upsampling sparse input data. Many of these efforts have proven to be promising when traditional solutions either fail or make only incremental progress. There have been several excellent reviews on the history and status of deep learning technologies in PAT,^35,36,117 to which we refer interested readers. A detailed comparison of different deep learning approaches in PAT can be found in the review article by Gröhl et al.³⁶ Here we will highlight several of the most exciting advances.

There is a clear difference between the deep learning formulation in PACT and PAM. In PACT, many challenges arise from solving the inverse problem, mostly with partial and/or sparse detection geometries.¹¹⁸ Deep learning in PACT can be used as (i) a pre-processing or post-processing step in the image reconstruction, (ii) replacement of the traditional image reconstruction altogether, or (iii) one integrated step in the iterative reconstruction. For example, Gutta et al. used a fully connected deep neural network (FC-DNN) as a pre-processing step to correct the sonograms acquired by each transducer channel and broaden the bandwidth of the received channel data.¹¹⁹ Davoudi et al.¹²⁰ used a fully convolutional neural network (U-Net) to reconstruct the PACT data obtained by a ring array with limited view or sparse sampling, which resulted in improvements in both spatial frequency coverage and the final image quality [Figs. 5(a) and 5(b)]. For PACT with a linear transducer array, a stabilized generative adversarial network (GAN) model with gradient clipping has been employed as a post-processing step, which can reduce the limited-view and limited-bandwidth reconstruction artifacts of in vivo data [Fig. 5(c)].¹²¹ Another key area of research is integrating the deep learning into the PA forward operator for iterative–based image reconstruction, as demonstrated by Hauptmann et al.^122,123 and Bioink et al.¹²⁴ However, these iterative methods can be time–consuming.

Fig. 5

Limited-view PACT improved by deep learning methods. (a) Whole-body PACT images of a mouse and the corresponding spatial frequency spectra obtained by a ring-array system with a 360 deg or 60 deg detection angle range. The images were reconstructed using the traditional back projection method.¹²⁰ (b) A U-Net–based deep learning method was used to reconstruct the PACT image with a 60 deg detection angle. (c) A comparison of PACT images obtained by a linear array before and after a GAN method was used to reduce the limited-view artifacts.¹²¹ Adapted with permissions from Refs. 120 and 121.

Unlike PACT, PAM does not require inverse reconstruction, so deep learning models can directly map time-resolved input signals to output images, and improve imaging speed, signal-to-noise ratio, and spatial resolution. One of the major utilizations of deep learning in PAM is to improve sparsely sampled images, thereby shortening image acquisition time without substantially degrading image quality. For example, DiSpirito et al.¹²⁵ have developed a modified fully dense U-Net architecture (FD U-Net), and demonstrated the feasibility to recover microvessels in the mouse brain by acquiring only 2% of the pixels required by the Nyquist sampling. For situations that lack ground truth data for model training, Vu et al.¹²⁶ have proposed an innovative method that iteratively refines undersampled PAM images using a deep learning prior. This work is of particular interest because it does not require training on a large PAM dataset with ground truth. Deep learning has also recently been used by Song et al.¹²⁷ to improve PAM images with extremely low excitation laser energy. Most recently, Sharma and Pramanikhave¹²⁸ developed an FD U-Net to enhance the lateral resolution of AR-PAM, especially in the out-of-focus regions.

Notably, deep learning methods have also been investigated for improving quantitative PAT, which has been difficult for deep-seated targets due to spectral coloring. Deep learning approaches have been developed to either better estimate the optical fluence at different wavelengths or completely replace the traditional spectral unmixing algorithms. For example, a sequential-learning recurrent neural network has been used to predict eigen-fluence maps in deep tissue,¹²⁹ which were subsequently used for linear unmixing of the oxy- and deoxy-hemoglobin concentrations.¹²⁹ In another work, Gröhl et al.¹³⁰ applied a fully connected neural network on multi-spectral PA images, which improved the quantification accuracy of blood oxygenation estimations on phantoms and in vivo porcine brain. Further, Bench et al.¹³¹ applied a 3D encoder-decoder style neural network to predict volumetric blood oxygenation; however, this methodology has not yet been adapted to in vivo data due to the complexity of tissue’s optical properties.

Nevertheless, one obstacle to the broad adoption of deep learning in PAT is the heavy reliance on simulation data and the lack of large, open-source repositories of in vivo data. The gap between simulation data and in vivo data makes model extrapolation to in vivo applications difficult. Potential solutions to address this obstacle are for the community to (i) create a large, open-source repository of various in vivo training examples or (ii) improve the quality of simulation data to better mimic in vivo cases. Ultimately, the incorporation of deep learning into PAT requires the training of robust models that can readily adapt to a variety of in vivo conditions—many of which, such as sparsely-sampled, limited-view, and limited-bandwidth detection, are in non-ideal environments.