Volumetric Light-field Encryption at the Microscopic Scale

We report a light-field based method that allows the optical encryption of three-dimensional (3D) volumetric information at the microscopic scale in a single 2D light-field image. The system consists of a microlens array and an array of random phase/amplitude masks. The method utilizes a wave optics model to account for the dominant diffraction effect at this new scale, and the system point-spread function (PSF) serves as the key for encryption and decryption. We successfully developed and demonstrated a deconvolution algorithm to retrieve spatially multiplexed discrete and continuous volumetric data from 2D light-field images. Showing that the method is practical for data transmission and storage, we obtained a faithful reconstruction of the 3D volumetric information from a digital copy of the encrypted light-field image. The method represents a new level of optical encryption, paving the way for broad industrial and biomedical applications in processing and securing 3D data at the microscopic scale.

3D focal stacks across the entire volume of a 3D object. In the early demonstrations of light-field imaging systems, the use of ray-optics models for reconstruction inevitably compromised the lateral and angular (i.e. axial) resolution [32][33][34][35] . Recently, wave-optics models have been developed for light-field reconstruction, which mitigates the resolution loss through PSF deconvolution, thereby significantly advancing high-resolution 3D light-field imaging [36][37][38] . The algorithm reported in this work integrates the wave-optics models of both a microlens array and an array of random phase masks. We demonstrated that such an encryption system can faithfully recover both multiplexed discrete data and continuous volumetric data. Furthermore, we showed that volumetric information can be accurately recovered from a series of binary images digitally converted from a 2D light-field image, which will potentially be useful for the storage and transmission of large amounts of volumetric data. The principle can be flexibly extended to other digital systems, in which intrinsically 3D data or stacks of 2D data can be digitally processed in a similar manner. We anticipate that this light-field method will advance the development of optical encryption methodologies for securing, storing and transmitting volumetric data and have new applications in industry and the life sciences. Fig. 1a, the light-field based encryption system is established by placing a microlens array at the intermediate image plane of a wide-field microscope 34 . The microlens array is chosen to match the numerical aperture and magnification of the objective lens. This consideration allows the images formed by each individual lenslet to tile with each other without overlapping or leaving gaps on its back focal plane, where the camera sensor is placed. Each lenslet generates a perspective-view of the object, which propagates angular information onto different regions of the camera. Next, an array of random phase masks, each identical in size to a lenslet, is attached to the microlens array. Random phase masks are typically utilized in optical encryption as a pivotal data security feature. This random modulation of the light field can be experimentally realized using a phase plate or a spatial light modulator, located on a plane that is conjugated to the microlens array. With this optical design, both the spatial intensity and the angular (i.e. axial) information of the volumetric 3D data are encrypted after passing through the optical components. It is then recorded as a 2D light-field image on the camera.

Experimental and Computational Encryption Model. As shown in
To decrypt the data, we thus need to take into account the light-field information originating from both the microlens array and the random phase masks. For the microlens array, a traditional image retrieval method would generate a low-resolution, perspective-view of the object by tiling those pixels that correspond to the same incidence angle, and a conventional 2D image by summing the pixels behind each lenslet. However, in this case, the axial resolution compromises the lateral resolution, which is substantially limited by the lenslet size. We utilized a wave-optics model that overcomes this resolution limitation by considering the diffraction of light (i.e. the PSF) in the formation of the image through the microlens array and the random phase masks. It should be noted that though there are various types of random phase masks which could be used for the method, using the form of an array of masks, each matching a lenslet, ensures that the known phase information of the masks can be readily integrated into the same wave-optics model as is used for the microlens array. This integrated model also considers any cross talk generated by the random masks. Hence, the universal PSF, composed of both optical parts in the light-field system, becomes the key by which the original volumetric information is retrieved (see Methods for more details).
Mathematically, the decryption is an inverse problem to recover the radiant intensity at each point in the 3D volume, denoted with g, using the recorded camera image O. As shown in Fig. 1b, the two pieces of information satisfy the relationship O = Hg, where the measurement matrix H is determined by the PSF, and its elements h kj represent the projection of the light arriving at pixel O(j) on the camera from the k th calculated volume g(k) in the object space. To obtain g, the inverse problem thus becomes: The operator diag() diagonalizes a matrix. This expression is a modified deconvolution algorithm 36,37,39 based on the Richardson-Lucy iteration scheme 40 .

3D Data Encryption and Decryption.
To validate the light-field encryption model, we created a volumetric 32-μ m × 32-μ m × 50-μ m object space that consists of volumetric images of the letters 'S' , 'B' and 'U' , with the same 2-μ m thickness, located at − 60 μ m, − 34 μ m and − 10 μ m in the axial dimension, respectively, as shown in Fig. 2a. The imaging and encryption system is primarily composed of a 20× , 0.5NA objective, a microlens array with a 150-μ m pitch and 3-mm focal length, located at the intermediate image plane, an array of random phase masks (pixel size = 10 μ m) with the same pitch, and a CCD camera with a 10-μ m pixel size. An emission wavelength of 532 nm from the objects is considered. The 3D image data was encrypted and captured by the camera as a 2D light-field image, as shown in Fig. 2b. It should be noted that even without the random phase masks, the microlens array redistributes and projects the 3D image-forming light, protecting the original information from visual scrutiny and thus providing a basic level of data security. Next, the encrypted image was analyzed using the wave-optics based deconvolution algorithm, where the object space was sectioned into 128 × 128 × 26 volumes with a lateral (in x and y) interval of 0.25 μ m and an axial (in z) interval of 2 μ m. These interval values were empirically chosen to work with the resolution of the objective and the computational capacity available to execute the algorithm. Figure 2c shows the successful restoration of the volumetric images using the correct decryption key, i.e. the correct PSF of the system. We also examined the occlusion attack, where 25% of the image area was blocked (Fig. 2d). Figure 2e shows that the method can still recover the volumetric objects ('U' , 'B' , 'S') with moderately increased noise level. It is also seen that when the similarly occluded area was further increased (e.g. to 37.5%), the 3D object cannot be reconstructed faithfully. In addition, when the values of the elements in such a PSF key are randomly modulated by 5%, the reconstruction becomes noticeably degraded and inaccurate (Fig. 2f). This demonstrates that volumetric 3D data can be encrypted into a single 2D light-field image, and that by using the correct PSF key, the original information can be retrieved with high accuracy and sensitivity.
Multiplexed Optical Processing. The light-field method allows the encryption of information in an additional axial dimension without increasing the size and complexity of the recorded data. This capability not only increases the information density, allowing volumetric data to be multiplexed and encrypted in a 2D image, but also suggests a means for the direct encryption of a continuous 3D object (as opposed to the discrete volumetric objects in Fig. 2). In this Section, to further test the multiplexing capability of the light-field encryption system, i.e. the robustness against crosstalk effects, we positioned three 2-μ m thick objects, 'deer' , 'dog' and 'bird' , at distances of − 60 μ m, − 34 μ m and − 10 μ m in the object space, respectively. In addition, a 3D object, 'bunny' , with the size of 30 μ m × 30 μ m × 50 μ m, was placed in the same volume (Fig. 3a). In this case, the four objects in the volume were multiplexed in both the lateral and axial dimensions, and encrypted and recorded as a light-field image in Fig. 3b. The same 3D deconvolution algorithm was utilized for decryption. In Fig. 3c-g, we show the successful recovery of all four volumetric objects. The information has mostly been recovered and the crosstalk between the multiplexed objects is minimal. By increasing the number of iterations of deconvolution and using proper filtering thresholds, we found that the reconstructed objects could be further refined. In order to quantitatively analyze the performance of the decryption method, normalized correlations between the initial objects of 'deer' , 'dog' and 'bird' , and their respective decrypted images were performed (Fig. 3h-j). The correlation peak value C in these cases were measured to be > 0.7-0.8, indicating a high similarity between the original and reconstructed data. In brief, the results demonstrate that both discrete and continuous spatially-multiplexed volumetric data can be reliably encrypted and decrypted using the light-field method. It also implies that high parallelization can be achieved using the light-field approach as the 2D wide-field image allows various types of data to be parallelized in all three dimensions and processed all at once through the algorithm using the same PSF key.

Digitization for Binary Storage and Read-out.
For data processing, storage and transmission, digitization allows the data to be operated on with high efficiency and minimal degradation. In this work, the pixel values of the light-field image can be normalized to 2 N − 1 and rounded to the closest integer values P in a N-level binary representation, expressed as = ⋅ + ⋅ + ⋅ + + ⋅ − −  P a a a a 2 2 2 2 i is the pixel-value composite coefficient. The digitization of the light-field image thus refers to the processing of light-field data into N binary (i.e. pixel value at 0 or 1, or black or white) images, where image i (i = 0 ··· N − 1) represents the binary composites of the pixel values of the light-field image at 2 i . In Fig. 4a, the pixel values of the 2D encrypted light-field image were converted into a N = 2 level representation (i.e. pixel values were first normalized to 2 N − 1 = 3, rounded to the closest integer values, and expressed in binary). Hence, two binary images (Fig. 4b) were formed containing the pixel-value composites at 2 0 = 1 and 2 1 = 2, respectively. For example, a normalized and rounded pixel value 3 can be written into ⋅ + ⋅ = a a a 2 2 ( 1 ) 0 0 1 1 0,1 , so the corresponding pixels in the two binary images will both be given the values of 1. As shown in Fig. 4c, despite the binarization, decryption using the 2-level digitized images largely reveals the original volumetric information. With an additional binary level (N = 3), the retrieval of the 3D data compares favorably to the reconstruction using the full light-field information as in Fig. 2c (Fig. 4d-f). Such a digitization procedure transforms a 2D image of full dynamic range into a discrete set of binary images, allowing a substantial reduction in data storage requirements without compromising the quality of retrieval. The conversion of the light-field image into binary images also allows the original data to be partially accessed by multiple users, suggesting another level of utility and security. Moreover, the practicality of digitizing these light-field images allows more convenient and versatile access to the volumetric data. For example, the digitized binary images can be scanned and processed in a similar way to a quick-response (QR) code by a smartphone or tablet device 41 . Digitization also provides a way for transmitting light-field information through digital systems, such as optical fibers. We anticipate more powerful and diverse ways for encrypting, storing and transmitting 3D volumetric data can be achieved with advanced data processing strategies for light-field images.

Discussion
In summary, we have developed a new light-field encryption method and demonstrated the encryption and decryption of high-content, microscopic 3D information based on a single 2D light-field image. The wave-optics based PSF key, consisting of the microlens array and the array of random phase masks, allows the retrieval of data with high resolution, accuracy and sensitivity. The method represents a new advance in the optical encryption of 3D image data in the microscopic regime. It should be mentioned that, instead of using phase masks, using amplitude masks can further strengthen data security, completely preventing any visual scrutiny (as some shape of the objects can still be discerned through individual lenslets in Figs 2 and 3). A similar light-field encryption system that employs random amplitude masks has been demonstrated in Supplementary Figures 1 and 2.
As a new encryption platform in the microscopic regime, we envision broad applications for the method, such as high-density data security, watermarking, 3D scanning, identity authentication, etc [42][43][44] . Given its high-resolution and volumetric features, the method will be especially useful in the processing of protected 3D biomedical data 45,46 , a field in which optical imaging is an essential tool for data collection but encryption techniques are still lacking. Our set-up and algorithm are readily compatible with non-binary data (Supplementary Figure 3) and bright-field or fluorescent imaging. It should also be noted that the axial dimension in the volumetric data can also be considered as a dimension for time, so that the method is readily adaptable to the encryption of dynamic 2D image data. Another worthwhile future development of the method would be to use different random phase/amplitude masks for each lenslet, a measure which would increase security by expanding the size of the key space. The method could also be enhanced to take advantage of other optical degrees of freedom, such as wavelength, fluorescent life-time, or polarization [47][48][49][50] , providing further applications of this volumetric multiplexing encryption method. Beyond image data, other types of information, either intrinsically 3D data or stacks of 2D data, can be digitally processed in a similar manner, with the principle of the PSF key being readily extended to other formats. Finally, optical methods have demonstrated highly compatible and complementary strengths to the fast-developing digital technologies 6,51 , which nowadays provide flexible and powerful means of encryption with fast speed and low cost. We anticipate that it may be highly advantageous in many applications to combine optical and digital encryption methods to further improve data-processing, storage and transmission.

Methods
Model of Light-field Propagation and Image Formation. When the 3D volume in the object domain is projected into the imaging space, the wavefront at the intermediate image plane, in the case of high numerical aperture microscopy, is predicted by the Debye theory as 52