Original Article

Citation: Light: Science & Applications (2015) 4, e346; doi:10.1038/lsa.2015.119
Published online 23 October 2015

Portable lensless wide-field microscopy imaging platform based on digital inline holography and multi-frame pixel super-resolution
Open

Antonio C Sobieranski1,2,3, Fatih Inci4, H Cumhur Tekin4, Mehmet Yuksekkaya1,2,5, Eros Comunello3,6, Daniel Cobra7, Aldo von Wangenheim3 and Utkan Demirci1,2,4

  1. 1Demirci Bio-Acoustic-MEMS in Medicine (BAMM) Laboratory, Division of Biomedical Engineering, Division of Renal Medicine, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
  2. 2Demirci Bio-Acoustic-MEMS in Medicine (BAMM) Laboratory, Division of Infectious Diseases, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
  3. 3INCoD – National Brazilian Institute for Digital Convergence/LAPIX, Image Processing and Computer Graphics Lab, Federal University of Santa Catarina, Brazil
  4. 4Demirci Bio-Acoustic-MEMS in Medicine (BAMM) Laboratory, Stanford University School of Medicine, Canary Center at Stanford for Cancer Early Detection, Palo Alto, CA, USA
  5. 5Present Address: Department of Biomedical Engineering Faculty of Engineering, Başkent University, Ankara, Turkey
  6. 64VisionLab, Master in Applied Computing, University of Itajaí Valley, Brazil
  7. 7CERTI Foundation, Florianopolis, Brazil

Correspondence: U Demirci, Email: utkan@stanford.edu

Received 25 December 2014; Revised 12 June 2015; Accepted 24 June 2015

Top

Abstract

In this paper, an irregular displacement-based lensless wide-field microscopy imaging platform is presented by combining digital in-line holography and computational pixel super-resolution using multi-frame processing. The samples are illuminated by a nearly coherent illumination system, where the hologram shadows are projected into a complementary metal-oxide semiconductor-based imaging sensor. To increase the resolution, a multi-frame pixel resolution approach is employed to produce a single holographic image from multiple frame observations of the scene, with small planar displacements. Displacements are resolved by a hybrid approach: (i) alignment of the LR images by a fast feature-based registration method, and (ii) fine adjustment of the sub-pixel information using a continuous optimization approach designed to find the global optimum solution. Numerical method for phase-retrieval is applied to decode the signal and reconstruct the morphological details of the analyzed sample. The presented approach was evaluated with various biological samples including sperm and platelets, whose dimensions are in the order of a few microns. The obtained results demonstrate a spatial resolution of 1.55 μm on a field-of-view of 30 mm2.

Keywords:

digital in-line holography; image registration; lensless imaging; point-of-care platform; pixel super-resolution

Top

Introduction

Microscale imaging in life sciences so far has been associated with conventional light microscopes using geometric optic1. The advent of the holography principle, in 1948 by Dennis Gabor2, allowed the development of electron microscopes with high-resolution (HR), opening a wide range of investigation for both microscale and nanoscale microscopy. Such optical instruments, however, may require advanced infrastructure, being mostly restricted to well-established institutes. Particularly, in resource-limited or in developing countries, inexpensive, reliable, and user-friendly imaging devices are critical for the diagnosis and treatment of a series of diseases3,4,5,6,7,8,9,10,11,12.

With recent advances in opto-electronic technologies, miniaturized lensless imaging devices for point-of-care (POC) platforms have been developed, where the shadow imaging principle is commonly utilized in portable devices. Shadow imaging is a simple technique, whose working principle is to illuminate the specimens and capture two-dimensional (2D) spatial signatures (shadows) with an electronic image detector placed underneath13,14. Interestingly, by using a coherent light-source (i.e., monochromatic wave), the spatial signatures are converted to pattern interferences of the analyzed sample, and holographic information is obtained according to the scalar diffraction theory. By utilizing this pattern, numerical methods for phase-retrieval can be used to reconstruct the morphological details of the sample at specific object-planes.

From a technological perspective, a portable lensless holographic platform presents some advantages compared to conventional optical microscopes. The first aspect is the simplicity and cost, where the entire system can be built in a few cubic centimeters, using off-the-shelf components such as pinholes, monochromatic light-emitting diodes (LEDs) and complementary metal-oxide semiconductor (CMOS) and charge-coupled device (CCD). Second, the platform works with no lens requirements, and the investigation of a specimen is performed directly over the imaging sensor surface13,15,16,17, which presents a wide field-of-view (FOV) for high-throughput applications. Third, a single hologram contains all the three-dimensional (3D) structural information of the specimen, and using a high-speed camera, four-dimensional (4D) in-situ inspection is also possible to be implemented. Another important aspect related to portable lensless platforms is the easy customization for hardware (e.g., microfluidic) and the integration with embedded software. With the power of the recently developed processor units, many image processing methods can be integrated to perform low-level and high-level procedures, such as decision-making tasks. For low-level applications at the image formation level, resolution can be also improved computationally during the acquisition step. Spatial resolution is particularly determined by the pixel-size of the imaging detector, and some physical limits can be overcome by computing a HR image with a larger density of pixels18,19. In the literature, a HR image can be obtained from a single or multiple frames. For a single frame, physical aspects of the sensor should be known in advance and a point-spread-function (PSF) is estimated to convolve the image and improve sharpness information20,21. When a coherent illumination system is used, however, as the image formation is linear in the complex field, the estimated PSF function can produce some cancellations or other undesirable non-linear effects22. For multiple frames, a higher-resolution image can be obtained from consecutive frames taken over time23,24,25. This improvement of resolution is possible since physical variations are introduced in time, such as pixel shifts or different flickering intensities of light achieving the imaging sensor cells.

In this paper, a portable lensless wide-field holographic platform based on irregular shifts of the light-source is presented. This approach combines digital inline holography (DIH) and multi-frame pixel resolution approaches to compute a single higher-resolution holographic image from a set of observations of the scene, obtained from small planar displacements of the light-source. A registration procedure is performed to align the multiple images onto the same planar domain, and then, a sub-pixel optimization approach is used to minimize a cost-function designed to increase the quality of the holographic information. Finally, a numerical phase-retrieval method is used to recover the morphological information of the corresponding sample at specific object-planes. The presented portable platform was evaluated considering static circular and non-circular human cells in the order of a few microns, such as sperm and platelets. The results also show a spatial resolution of 1.55 μm on a maximum FOV of 30 mm2.

Top

Materials and methods

Experimental setup

An experimental setup was developed to acquire holograms and produce shifts of the light-source. This setup and its components are demonstrated in Figure 1a–1d: (a) light-source, (b) spherical aperture and xy-stage built to displace the light-source, (c) surface to accommodate the sample, and (d) the imaging sensor. In Figure 1a, a light-source is used to illuminate the samples with a pinhole (b), where an ultraviolet LED of 385 nm, model M385L2 from ThorLabs was used. To produce the xy-stage demonstrated in Figure 1b, a CO2 laser cutter engraver model Versa Laser 2.3 was used. This stage has a chamber to scatter the light and two independent platforms on the top specifically designed to displace the light-source on x- and y-axis directions independently. This chamber was projected to image samples using cover glasses or microfluidic channels (Figure 1c), and signatures are recorded by a CMOS sensor (Figure 1d) and a xy-stage having dimensions of 10 × 10 cm2 placed 5 cm away from the sensor. On top of it, LED was placed in a cylindrical structure with 100 µm diameter pinhole located 15 cm away from the sensor. CMOS imaging sensor is fixed from the bottom side of the platform, where the distance between sample and electronic detector is as small as possible. The CMOS sensor used to capture the hologram shadows is an Aptina 10 megapixel monochromatic sensor with dimensions of 6.413 mm × 4.589 mm (30 mm2), spatial resolution of 3840 × 2748 pixels, frame rate of 3.2 frames per second, and 1.67 μm of pixel-size. 8-bit raw data can be directly accessed from the sensor via Software Development Kit, providing a practical manner to implement data acquisition, numerical diffraction and image analysis on the same portable platform.

Figure 1.
Figure 1 - Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author

Schematic representation of the experimental setup used to acquire holographic signatures. In (a) the light source is used with a pinhole placed over, as shown in (b). In (c) the object-plane corresponding to the sample to be inspected is illustrated. The detector-plane is shown in (d), where a CMOS-based imaging sensor is used. (e) General overview of the presented approach: set of LR images with small shifts of the light-source is registered and an optimization method minimizes an objective function. The obtained HR hologram is then processed by a phase-retrieval method into morphological information. Noise suppression can also be obtained from the multiple shifted images.

Full figure and legend (125K)

One of the advantages of our portable holographic platform for microscopy analysis is simplicity and low cost, since no lenses are required to image samples. The presented holographic platform can be built using inexpensive materials, such as UV light-source for illumination, pinholes, and the setup can be designed in a conventional laser cutter engraver to accommodate component parts. The most relevant component is the CMOS imaging-based detector, which meets the requirements (monochrome and reduced pixel-size) to compute pixel information, and costs 500 US dollars. The total cost of the presented portable platform using a single light-source is estimated in less than 600 US dollars with LED, pinhole and home-made PMMA parts.

Using the experimental setup described, a set of LR images is obtained from a displacement matrix of varied sifts of the light-source (7 × 7 different positions). To improve resolution, a two-step registration procedure is used to (i) align the LR images on the same planar domain by a feature-based method and (ii) optimize sub-pixel information by the minimization of an objective function. The registration method keeps the initial pre-alignment points from a feature-based registration method, while seeking for sharpness optimization in a pixel level. Thus, we obtain a HR image where the propagation of the holographic signal are improved by recover the missing information from the set of LR images, as illustrated in Figure 1e.

Feature-based image registration

Image registration is the process of computing an optimal transformation between two or more images and align them on the same domain. The pre-alignment procedure is a feature-based registration method performed as follows26:

  • The key-points are detected for the both reference and candidate images, and a feature-vector is calculated. The Speeded Up Robust Features (SURF) algorithm27, based on the sums of Haar wavelets and inspired on the Scale-Invariant Feature-Transform (SIFT) algorithm, is used to locate the relevant key-points26.

  • The feature-vectors of the reference and candidate images are matched by a neighborhood function. The Fast Approximate Nearest Neighbor Matcher was used to correlate key-points28.

  • To select only high-quality matches, the obtained results are validated by a clustering technique designed to remove outliers’ points. These outliers can be detected by a least-square fitting approach based on the geometrical distances and angular coefficients between reference and candidate images of the LR set26.

  • The resulting matrix-matching is computed for every single LR image using the Random Sample Consensus algorithm29.

Feature-based methods, on the other hand, are not properly designed for the sub-pixel alignment and interpolation once key-points represent an approximate matching of features. To optimize the alignment in a sub-pixel level, the homography matrix is used as an initial guess of the global optimum solution and an area-matching method is used to optimize alignment, as detailed in the next section.

Sub-pixel optimization using continuous ant colony optimization (ACO)

In this approach, sub-pixel optimization is performed by the ACO algorithm30. Differently from the traditional ACO algorithm, designed to be a metaheuristic for discrete combinatorial problems, the ACO is a category of algorithms for continuous optimization. As a typical formulation of combinatorial problems, continuous optimization is determined by a search-space S designed to operate over a set of decision variables and an objective function f: S right arrow 0+ to be minimized. S is a set of continuous variables Xi, i = 1, , n, and it has the constrained or unconstrained variables, i.e., vi D domain. A solution s* S is the best configuration of the continuous variables (i.e., for the sub-pixel registration problem) and it is considered a global optimum solution (the best minimization of an error metric) if and only if f(s*) f(s) for each possible f(s) S. The set of all global optimum solutions is presented by S* S, where at least one s* should be found to solve the continuous optimization problem. As we can see, this is an iterative process, where better solutions (with a minimum error) are kept to substitute worst ones in a minimization procedure.

To find s*, the minimization of an energy cost-function based on variational models is used for the holographic sub-pixel problem31,32. Two penalizer terms were used in our cost-function: a data term and a sharpness measure. The first term (i.e., data or fidelity term) penalizes deviations of a candidate HR image from the set of multiple LR images, expressing the assumption that the result should be close to the data during the minimization. The second term is a sharpness measure designed to indicate when a candidate solution is producing better holographic propagation. Consider a candidate solution containing the LR images transformed by a matrix-matching, associated with a HR image I. The energy E of this solution is defined as:

Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author

where L is the set of transformed LR observations, and m is the cardinality of L. The aforementioned equation is similar to the general formulation for super-resolution presented by the Ref. 33, but with the addition of a term specifically designed for holographic propagation. First term is the fidelity term measuring the quality of approximation between each LR image in L to I, being related to spatial locality, penalizing solutions whose correspondence to I is inappropriate. (i.e., large E). The second term I is a focus measure used to compute the relative focus degree of an image, and can be obtained from the sum-modified-Laplacian method (LAPM)34. Parameters α and β, like in many variational models formulated over the literature31, controls the relative importance for each penalizer term. As the LR set is obtained upon spatial displacements (shifts of the light-source), some diffraction patterns may change according to the position of the light-source, shape and height of object-plane, and holographic fringes may be slightly different for each LR image. Feature-based alignment and sub-pixel optimization computational procedures are also illustrated in Supplementary Fig. S1.

Diffraction calculation

Diffraction calculation is a computational numerical solution to the phase problem when a wavefront intercepts openings and obstacles. In Figure 2, the diffraction computed by the Angular Spectrum Method (ASM) applied to holograms of sperm sample is presented. From the first to third rows, the real (ℜ), imaginary (Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author) and composition signal are illustrated. Composition is obtained by computing a multi-dimensional signal from I. Using a composition signal, we visualize sperm heads and sperm tails with distinct color intensities, and then, allow automatic counting and image analysis. By varying z-specific object-planes are obtained, and distinct z-axis information can be visualized. Details in diffraction calculations were stated in Supplementary Information.

Figure 2.
Figure 2 - Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author

Numerical diffraction of a sperm sample hologram. First to third rows the real (ℜ), imaginary (Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author) and composition ( + Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author) parts of the diffraction are shown, respectively. Diffractive distances from the first to fourth columns are z = [30k, 45k, 60k, 70k] (k = x1000), according to Equation (S3) of Supplementary Information.

Full figure and legend (306K)

Biological sample preparation

Biological human samples were used to validate the presented experimental setup and algorithmic solution. Sperm refers to the male reproductive cells, with dimensions at the order of 3–5 μm35, and 55 µm long tail with a diameter of 0.5 μm36. Sperm cells are particularly interesting, presenting unique characteristics and morphology, so they can be easily identified using the presented approach even without microscopy confirmation. Sperm samples were in liquid nitrogen. For sampling, we thawed sperm at 37°C, and diluted in 1% bovine serum albumin (BSA) in human tubulin fluid media (1:20 dilution with the media). For the analysis, 10 µL was added on a glass slide, and mounted with an 8 × 8 mm cover slip. Sperm samples were incubated on the glass slides until they are metabolically dead, and thus, we evaluated sperm morphology in static mode.

Platelets, on the other hand, are small disc-shaped cell fragments with greatest diameter varying from 2 to 3 μm37, playing significant roles to repair and regenerate connective tissues. For evaluation of platelet morphology, platelet cells were captured from whole blood in microchannels. To produce microchips with nine-microchannels, poly(methyl methacrylate) (PMMA) (40.5 × 24.5 mm), double-sided adhesive film (DSA, 50 µm for microchannel height) and glass cover slide (40.5 × 24.5 mm) were used38. The inlets and outlets (0.65 mm in diameter, 26 mm apart) were cut on the PMMA layer using a VersaLASER (Universal Laser Systems Inc., Scottsdale, AZ, USA). DSA was cut to form the microchannels (6.6 × 2.4 × 0.05 mm) using the same procedure. This DSA layer was then attached to the PMMA layer. A glass slide was modified with oxygen plasma (100 mW, 1% oxygen) for 90 seconds in a PX-250 chamber (March instruments, Concord, MA, USA), followed by silanization step using 200 mM of 3-mercaptopropyl-trimethoxysilane (3-MPS, Sigma-Aldrich, St. Louis, MO, USA) dissolved in ethanol. After the silanization step, the glass slide was assembled with the PMMA–DSA. 2 mM of N-γ-maleimidobutyryl-oxysuccinimide ester (GMBS, Sigma-Aldrich, St. Louis, MO, USA) in ethanol was used as a cross linker of NeutrAvidin (Thermo Fisher Scientific Inc., Waltham, MA, USA) binding, followed by antibody immobilization (CD62p antibody biotin anti-human, Biolegends, San Diego, CA, USA) on NeutrAvidin-modified surface39. After surface chemistry steps were completed, whole blood which was diluted 1:40 in a buffer with 5% acid-citrate-dextrose, 2% Tyrode’s buffer, 3% Apyrase from potatoes, 5% BSA, 1% HEPES, in phosphate-buffered saline (PBS) was introduced into microchannels. After 20 minutes of incubation time PBS applied into microchannels to remove other blood cells and plasma and captured platelets are ready for observing. Further, we used a fluorescent-labeled antibody (Mouse Anti-human CD41: Alexa Fluor® 647 antibody, AbDSerotec, Raleigh, NC, USA) to visualize the captured platelets under microscope.

Samples were imaged using the presented approach, and microscopy analysis was also performed. To acquire the set of LR images, static samples were considered. To conduct our experiments, a microscopy analysis was performed using an inverted microscope Carl Zeiss Axio Observer-Z1. For sperm and platelet samples, microscope mosaic images with 10× were obtained. Bright-field images were acquired for both sperm and platelet samples for visual comparison. Additionally, fluorescent microscopy images were acquired for confirmation of platelets after incubating with Cy-5-labeled detection antibodies.

Top

Results and discussion

Resolution improvements for holographic signatures

A notable aspect to be considered in terms of qualitative and quantitative analysis is related to the quality of the holographic signatures obtained on the registered images. Low and high frequencies are fundamental in holography, and the number of cycles defines the level of details recovered by a numerical diffraction method. In Figure 3, the results are demonstrated using the sub-pixel registration method. First and second column correspond to LR and recovered HR holographic images, respectively. Holographic images were submitted to the Prewitt compass to make visible specific details of the reconstruction procedure, once they may not be perceptible for raw holographic images. Prewitt compass is a discrete differentiation operator used to compute the approximation of the gradient related to the image intensity, and it is better understood as either the gradient vector or the vector-norm for each coordinate in the input image. The results suggest two main aspects to be observed regarding resolution and noise suppression. Qualitative improvement of holographic frequency can be clearly observed by the number of recovered fringes and the quality of morphological information, as well as the effect of noise suppression for background areas.

Figure 3.
Figure 3 - Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author

LR and HR holograms of sperm samples. The prewitt compass is used for visualization purposes. At the left side, a single-shot (i.e., LR) holographic image is presented, and at the right side, the holograms are combined into a HR image obtained from 49 observations of the scene.

Full figure and legend (658K)

In Figure 4, a quantitative evaluation of the number of cycles for LR and HR holograms was performed. At the left and right side, the LR and HR holograms are presented, followed by the periodic analysis of holographic signal over a specific area (right side). The measurements are defined by the green and red lines over the LR and HR images, respectively. For the HR holograms, where the morphological information is improved and the high frequencies recovered, the number of gradient peaks is better when compared to those presented in a LR hologram, captured from a single frame. Generally, LR holograms present periodicity until the seventh or eighth cycle, although intensities are also well-described by the HR holograms. For a higher number of cycles, the correspondence between LR and HR is lost, suggesting better quantitative results for the multi-frame approach.

Figure 4.
Figure 4 - Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author

Quantitative evaluation of LR and reconstructed HR holograms. LR and HR holograms are shown at the left and right side, respectively. A measure of the intensities was performed over the red and green lines drawn on the LR and HR resolution hologram, respectively. At the right side, these intensities were plotted, showing a better propagation and signal discrimination for the HR holographic images.

Full figure and legend (336K)

In addition, noise may affect the quality of signal for the LR images, especially for high frequencies. Noise can be introduced by physical aspects, such as oscillatory differences of the light intensities captured by the sensor cells over time. In the Figures 3 and 4, some of the mentioned variations can be verified for the LR images, where several gradient peaks can be visualized along the background areas. Undesirable physical artifacts such as dust or scratches that may be introduced on the sensor surface or the top of the microfluidic or cover glasses can be minimized during the acquisition by the presented platform. These artifacts, which are not located at the same object-plane, are suppressed and redistributed on the HR holographic image. The artifact suppression is possible with no additional hardware or software computation thanks to the diffraction of light, which is differentiable for each object-plane when the point-source is changed. The displacement vector for such artifacts does not match with the vectors located at the same object-plane, and thus the matrix-matching ensures the spatial distribution of these false-positive particles.

Evaluation and microscopy analysis

Microscopy analysis was performed to compare the results of the portable platform. The amplitude signal of the ASM was used for visual comparison against the corresponding bright-field image of the microscope. Holograms of sperm cells presented in Figure 3 (first row holograms) were recovered by the ASM, as shown in Figure 5, where LR and HR results were presented for single shots and HR sub-pixel optimization, respectively. Amplitude, phase, and composition of the RGB signal were obtained. In HR images, improved resolution and noise suppression for the background areas was observed when compared to LR images. A higher level of high-frequency signal allows better level of details for numerical reconstruction. In Figure 6, the reconstruction of the HR holograms previously shown in Figure 3 is demonstrated with the corresponding bright-field microscopy image. A visual comparison demonstrates a high correlation of the obtained results and the bright-field microscopy image, even for small morphological structures, such as sperm tails.

Figure 5.
Figure 5 - Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author

Comparison of reconstructed LR (left side) and HR (right side) holograms, recovered from holograms shown in Figure 3 (first row holograms). From the first to third rows, the amplitude, phase, and multi-dimensional RGB composition signals are shown, respectively. Last row shows the normalized HR phase in radians, having values corresponding to (min. = −1.43, max. = 3.1416).

Full figure and legend (99K)

Figure 6.
Figure 6 - Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author

Comparison of the obtained holographic and bright-field microscopy analysis of sperms. Left side are the obtained results for decoded HR holographic images previously presented in Figure 4, and the corresponding bright-field microscopy images are shown at the right side for confirmation purposes.

Full figure and legend (321K)

Platelet cells (2–3 μm in size) were also imaged in the platform, as shown in Figure 7. The result of the numerical diffraction was shown in the first column, where the Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author-part is presented. Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author-part is particularly interesting especially when acquired holographic shadows are weak, providing better contrast responses. Microscopy confirmation was performed using the bright-field (middle), and Cy5 fluorescent microscopy images were also presented at right side of Figure 7. A high correlation can be also observed for both presented approach and microscopy analysis.

Figure 7.
Figure 7 - Unfortunately we are unable to provide accessible alternative text for this. If you require assistance to access this image, please contact help@nature.com or the author

High-resolution platelet holograms recovered by ASM. From left side to right: decoded platelet image from ASM, bright-field microscopy image, and fluorescent microscopy image, respectively. At the bottom side, zoomed areas are shown.

Full figure and legend (354K)

Top

Conclusions

Lensless imaging technologies present great impact on the development of portable POC platforms for resource-limited settings, and by associating them with computational pattern recognition and digital image-processing techniques, novel platform technologies are developed for real world applications. The presented portable lensless wide-field microscopy platform is one example for such applications, where both imaging sensors and computational image interpretation are integrated on the same platform for micron analysis. This platform is also validated with multiple imaging applications, such as the visualization of sperm morphology and the counting of platelets on-chip. Thus, we provide a new insight to broad applications for POC imaging applications in the future.

The principle of holographic imaging invented many decades ago has significantly advanced since the original digital in-line holographic experiment2,40. In the literature, there is a significant number of approaches developed upon the traditional holography principle, where a splitter is commonly used to separate object and reference beams41,42,43,44. Although traditional holography may presents some limitations regarding miniaturization into POC applications, many of their diffraction concepts can be brought to DIH. Due to its simplicity, DIH is preferable for POC applications, and it can be easily implemented into a miniaturized platform using inexpensive components.45,46,47. Many DIH approaches for POC applications are dedicated to improve holographic signatures by making use of high-speed frame-rate cameras in the order of thousand frames per second (fps)48,49,50. Small sub-pixel sweeping perspectives are then computed, and very slight differences of the sample in motion are recorded over time. Although the elevated acquisition cost for a higher-speed camera, this category of portable platforms can achieve very efficient sub-micron resolutions.

Using a different principle to increase resolution on a large FOV (24 mm2) of a conventional imaging sensor, an array of LEDs and a micro-controlled device was used to alternate the light point-source and shift holographic signatures on the sensor surface51,52. A HR holographic image is obtained from a set of 23 LR images, and an interactive phase-retrieval method recovers sub-micron scale visualization after a number of iterations is achieved. Previously a single light-source was used to capture sperm holograms to compute their movement within a rate of 2 frames per second, using a subtraction method between consecutive frames53. Similarly, the simulation of LR images captured from different orientations and positions was demonstrated in Ref. 42. Using an array of 25 LEDs connected to fibers, light is scattered to different locations to estimate a HR image. A lensless optical tomographic microscope using a partially coherent in-line holography was presented by Refs. 54 and 55, where a multi-angle lensless imaging platform and tomographic microscope was developed. In this approach the achieved spatial resolution is approximately 1 μm within a sample volume of 15 mm3. Also, a high-throughput lens-free 3D tracking of human sperms is presented in the Refs. 56 and 57, where two sequential light-sources of different wavelengths were positioned for vertical and oblique planes. Holograms in video are recorded and further decoded to track sperm trajectories in the medium. Recently, a platform was introduced to demonstrate a multi-angle observation model58. In the platform, multiple frames were taken from varied perpendicular and angular position, followed by the computation of a higher-resolution hologram. By increasing the distance between object and detector planes, magnification of holograms is obtained. Also, a tunable flexible hexagonal phase grating was used to improve resolution in Ref. 59. The numerical aperture is then increased, improving the spatial resolution for the imaging system.

In our approach, differences can be verified when compared to the aforementioned works in the literature. First, the multiple images are obtained under the illumination of a single and the same light-source and pinhole, arbitrarily displaced (i.e., 1–2 mm) over time by a xy-stager scheme. A single light-source makes the setup easier and simple to be implemented, and no variations provided by different wavelengths are introduced. Second, resolution for the holographic signal is improved and at the same time noise suppression is performed, as it can be verified in our experimental results section and in supporting reconstruction video. Third, in computational terms, image registration and optimization is performed by both feature-matching and area-based approaches, thus the system is unconstrained to relatively small steps of the light-source. The optimization of the sub-pixel information is performed by energy equations from variational models, and minimized by a bio-inspired framework. Based on this irregular displacement scheme and also computational registration and optimization, the presented platform is operated with no calibration requirements, since there is no need for pre-determined locations of point-source. Our method also tracks holographic shadows dynamically from arbitrary locations (produced by irregular shifts of the light-source), and combines the frames onto the same planar domain to compute a HR frame. The portable lensless wide-field microscopy based on digital in-line holography and multi-frame processing achieves a spatial resolution of 1.55 μm in a maximum FOV of 30 mm2.

From a computational perspective, multi-frame pixel super-resolution approaches can be greatly used to circumvent physical limits of hardware by recovering the missing holographic information from a set of observations of the scene. In this paper, a two-step registration method was used for both feature-based and area-based registration. The obtained results are closely comparable with those obtained with a conventional microscope using the bright-field.

Further improvements can be performed to increase the efficiency of the presented approach. A stable displacement micro-controlled scheme can reduce the need of the pre-alignment procedure, as well as distortions produced when large displacements are used. Sub-pixel information can be computed with no need of a pre-alignment step once the matrix of displacements for the initial guess solution is static. Artifacts provided by the twin-image problem also should be addressed to remove holographic interference after holograms are processed. On the other hand, intrinsic limitations can be verified related to the Gabor’s inline holography principle, such as twin-image problem and the density of the analyzed sample. Specifically for the confluence of samples, the reference wave gets distorted in the presence of high-density particles, producing self-imposed interferences patterns. Holographic patterns may be deformed due stretching and cross-section of shadows, produced by the incident illumination angle or partial interference in the presence of confluence of high-density objects60. Confluence problem can be solved in part by using a very tiny sample or tissue in the order of a few micrometers, where diffractive interference patterns can be captured on the detector plane.

Top

Competing Interests

Dr. U Demirci is a founder of, and has an equity interest in: (i) DxNow Inc., a company that is developing microfluidic and imaging technologies for point-of-care diagnostic solutions, and (ii) Koek Biotech, a company that is developing microfluidic IVF technologies for clinical solutions. U.D.’s interests were viewed and managed in accordance with the conflict of interest policies.

Top

References

  1. Schawlow AL, Townes CH. Infrared and optical masers. Phys Rev 1958; 112: 1940–1949. | Article | ISI | CAS |
  2. Gabor D. A new microscopic principle. Nature 1948; 161: 777–778. | Article | PubMed | ISI | CAS |
  3. Wang SQ, Xu F, Demirci U. Advances in developing HIV-1 viral load assays for resource-limited settings. Biotechnol Adv 2010; 28: 770–781. | Article | PubMed | CAS |
  4. Wang SQ, Inci F, de Libero G, Singhal A, Demirci U. Point-of-care assays for tuberculosis: role of nanotechnology/microfluidics. Biotechnol Adv 2013; 31: 438–449. | Article | PubMed | ISI |
  5. Tasoglu S, Gurkan UA, Wang SQ, Demirci U. Manipulating biological agents and cells in micro-scale volumes for applications in medicine. Chem Soc Rev 2013; 42: 5788–5808. | Article | PubMed | ISI | CAS |
  6. Inci F, Filippini C, Baday M, Ozen MO, Calamak S et al. Multitarget, quantitative nanoplasmonic electrical field-enhanced resonating device (NE2RD) for diagnostics. Proc Natl Acad Sci USA 2015; 112: E4354–E4363. | Article | PubMed |
  7. Shafiee H, Asghar W, Inci F, Yuksekkaya M, Jahangir M et al. Paper and flexible substrates as materials for biosensing platforms to detect multiple biotargets. Sci Rep 2015; 5: 8719. | Article | PubMed |
  8. Inci F, Tokel O, Wang S, Gurkan UA, Tasoglu S et al. Nanoplasmonic quantitative detection of intact viruses from unprocessed whole blood. ACS Nano 2013; 7: 4733–4745. | Article | PubMed | ISI | CAS |
  9. Lissandrello C, Inci F, Francom M, Paul M, Demirci U et al. Nanomechanical motion of Escherichia coli adhered to a surface. Appl Phys Lett 2014, 105: 113701. | Article | PubMed |
  10. Shafiee H, Wang S, Inci F, Toy M, Henrich TJ et al. Emerging technologies for point-of-care management of HIV infection. Annu Rev Med 2015; 66: 387–405. | Article | PubMed |
  11. Tokel O, Inci F, Demirci U. Advances in plasmonic technologies for point of care applications. Chem Rev 2014; 114: 5728–5752. | Article | PubMed | ISI | CAS |
  12. Tokel O, Yildiz UH, Inci F, Durmus NG, Ekiz OO et al. Portable microfluidic integrated plasmonic platform for pathogen detection. Sci Rep 2015; 5: 9152. | Article | PubMed |
  13. Ozcan A, Demirci U. Ultra wide-field lens-free monitoring of cells on-chip. Lab Chip 2008; 8: 98–106. | Article | PubMed | CAS |
  14. Moon SJ, Gurkan UA, Blander J, Fawzi WW, Aboud S et al. Enumeration of CD4+ T-cells using a portable microchip count platform in Tanzanian HIV-infected patients. PLoS One 2011; 6: e21409. | Article | PubMed | CAS |
  15. Seo S, Su TW, Tseng DK, Erlinger A, Ozcan A. Lensfree holographic imaging for on-chip cytometry and diagnostics. Lab Chip 2009; 9: 777–787. | Article | PubMed | ISI | CAS |
  16. Mudanyali O, Tseng D, Oh C, Isikman SO, Sencan I et al. Compact, light-weight and cost-effective microscope based on lensless incoherent holography for telemedicine applications. Lab Chip 2010; 10: 1417–1428. | Article | PubMed | ISI | CAS |
  17. Quan C, Chen W, Tay CJ. Numerical reconstruction in in-line digital holography by translation of CCD position and gradient operator method. Opt Commun 2011; 284: 2767–2770. | Article |
  18. Sheppard CJR. Fundamentals of superresolution. Micron 2007; 38: 165–169. | Article | PubMed |
  19. Zhang LP, Zhang HY, Shen HF, Li PX. A super-resolution reconstruction algorithm for surveillance images. Signal Process 2010; 90: 848–859. | Article |
  20. Glasner D, Bagon S, Irani M. Super-resolution from a single image. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. IEEE: Kyoto, 2009, pp. 349–356.
  21. Li JM, Gong WG, Li WH, Pan FY. Single-image super-resolution reconstruction based on global non-zero gradient penalty and non-local Laplacian sparse coding. Digital Signal Process 2014; 26: 101–112. | Article |
  22. van Alert S, van Dyck D, den Dekker AJ. Resolution of coherent and incoherent imaging systems reconsidered – classical criteria and a statistical alternative. Opt Express 2006; 14: 3830–3839. | Article | PubMed |
  23. Pickup LC. Machine learning in multi-frame image super-resolution. PhD thesis, University of Oxford, Oxford, 2008.
  24. Li XL, Hu YT, Gao XB, Tao DC, Ning BJ. A multi-frame image super-resolution method. Signal Process 2010; 90: 405–414. | Article |
  25. Ning BJ, Gao XB. Multi-frame image super-resolution reconstruction using sparse co-occurrence prior and sub-pixel registration. Neurocomputing 2013; 117: 128–137. | Article |
  26. Goshtasby AA. Image Registration: Principles, Tools and Methods. London: Springer; 2012.
  27. Bay H, Ess A, Tuytelaars T, van Gool L. Speeded-up robust features (SURF). Comp Vis Image Underst 2008; 110: 346–359. | Article |
  28. Muja M, Lowe DG. Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Pattern Anal Mach Intell 2014; 36: 2227–2240. | Article | PubMed |
  29. Fischler MA, Bolles RC. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 1981; 24: 381–395. | Article | ISI |
  30. Socha K, Dorigo M. Ant colony optimization for continuous domains. Eur J Oper Res 2008; 185: 1155–1173. | Article |
  31. Brox T. From pixels to regions: partial differential equations in image analysis. PhD thesis, Mathematical Image Analysis Group, Department of Mathematics and Computer Science, Saarland University, Saarbrücken, Germany, 2005.
  32. Mochizuki Y, Kameda Y, Imiya A, Sakai T, Imaizumi T. Variational method for super-resolution optical flow. Signal Process 2011; 91: 1535–1567. | Article |
  33. Mitzel D, Pock T, Schoenemann T, Cremers D. Video super resolution using duality based TV-L1 optical flow. Proceedings of the 31st DAGM Symposium on Pattern Recognition. Springer-Verlag: Berlin, Heidelberg, 2009, pp. 432–441.
  34. Nayar SK, Nakagawa Y. Shape from focus. IEEE Trans Pattern Anal Mach Intell 1994; 16: 824–831. | Article |
  35. El-Ghobashy AA, West CR. The human sperm head: a key for successful fertilization. J Androl 2003; 24: 232–238. | PubMed |
  36. Baltz JM, Katz DF, Cone RA. Mechanics of sperm-egg interaction at the zona pellucida. Biophys J 1988; 54: 643–654. | Article | PubMed |
  37. Reece JB, Urry LA, Cain ML, Wasserman SA, Minorsky PV et al. Campbell Biology. 9th edn. Benjamin Cummings; San Francisco. 2010.
  38. Wang SQ, Inci F, Chaunzwa TL, Ramanujam A, Vasudevan A et al. Portable microfluidic chip for detection of Escherichia coli in produce and blood. Int J Nanomedicine 2012; 7: 2591–2600. | PubMed | ISI | CAS |
  39. Wang SQ, Esfahani M, Gurkan UA, Inci F, Kuritzkes DR et al. Efficient on-chip isolation of HIV subtypes. Lab Chip 2012; 12: 1508–1515. | Article | PubMed | ISI | CAS |
  40. Gabor D. Microscopy by reconstructed wave fronts: II. Proc Phys Soc B 1951; 64: 449–469. | Article |
  41. Micó V, Zalevsky Z, García J. Edge processing by synthetic aperture superresolution in digital holographic microscopy. 3D Res 2011; 2: 1.
  42. Hussain A, Mudassar AA. Holography based super resolution. Opt Commun 2012; 285: 2303–2310. | Article |
  43. Hussain A, Martínez JL, Campos J. Holographic superresolution using spatial light modulator. J Eur Opt Soc – Rapid Publicat 2013; 8: 13007.
  44. Singh VR, Sui LS, Asundi A. Compact handheld digital holographic microscopy system development. Proc of SPIE 2010; 7522: 75224L.
  45. Shimobaba T, Taniguchi Y, Shiraki A, Masuda N, Ito T. Portable and low-cost digital holographic microscopy using web camera, point light source LED and open-source libraries. Biomedical Optics and 3-D Imaging, OSA Technical Digest, Optical Society of America, Miami, FL, USA, 28 April–2 May 2012.
  46. Lu Y, Liu Y, Lau TK. Simple, portable, and low-cost microscope based on off-axis digital holography using two spherical waves. Opt Lett 2014; 39: 4549–4552. | Article | PubMed |
  47. Garcia-Sucerquia J, Xu WB, Jericho SK, Klages P, Jericho MH et al. Digital in-line holographic microscopy. Appl Opt 2006; 45: 836–850. | Article | PubMed |
  48. Zheng GA, Lee SA, Antebi Y, Elowitz MB, Yang C. The ePetri dish, an on-chip cell imaging platform based on subpixel perspective sweeping microscopy (SPSM). Proc Natl Acad Sci USA 2011; 108: 16889–16894. | Article | PubMed | CAS |
  49. Yan M, Huang XW, Jia QX, Nadipalli R, Wang TX et al. (eds). High-speed CMOS image sensor for high-throughput lensless microfluidic imaging system. Proc of SPIE 2012, 2012; 8298: 829804.
  50. Huang X, Guo J, Wang X, Yan M, Kang Y et al. A contact-imaging based microfluidic cytometer with machine-learning for single-frame super-resolution processing. PLoS One 2014; 9: e104539. | Article | PubMed |
  51. Bishara W, Su TW, Coskun AF, Ozcan A. Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution. Opt Express 2010; 18: 11181–11191. | Article | PubMed | ISI |
  52. Bishara W, Sikora U, Mudanyali O, Su TW, Yaglidere O et al. Portable and cost-effective pixel super-resolution on-chip microscope for telemedicine applications. Conf Proc IEEE Eng Med Biol Soc 2011; 2011: 8207–8210. | PubMed |
  53. Su TW, Erlinger A, Tseng D, Ozcan A. Compact and light-weight automated semen analysis platform using lensfree on-chip microscopy. Anal Chem 2010; 82: 8307–8312. | Article | PubMed | CAS |
  54. Su TW, Isikman SO, Bishara W, Tseng D, Erlinger A et al. Multi-angle lensless digital holography for depth resolved imaging on a chip. Opt Express 2010; 18: 9690–9711. | Article | PubMed |
  55. Isikman SO, Bishara W, Mavandadi S, Yu FW, Feng S et al. Lens-free optical tomographic microscope with a large imaging volume on a chip. Proc Natl Acad Sci USA 2011; 108: 7296–7301. | Article | PubMed |
  56. Su TW, Xue L, Ozcan A. High-throughput lensfree 3D tracking of human sperms reveals rare statistics of helical trajectories. Proc Natl Acad Sci USA 2012; 109: 16018–16022. | Article | PubMed |
  57. Su TW, Choi I, Feng JW, Huang K, McLeod E et al. Sperm trajectories form chiral ribbons. Sci Rep 2013; 3: 1664. | PubMed | CAS |
  58. Luo W, Greenbaum A, Zhang YB, Ozcan A. Synthetic aperture-based on-chip microscopy. Light: Sci Appl 2015; 4: e261; doi: 10.1038/lsa.2015.34. | Article |
  59. Paturzo M, Merola F, Grilli S, de Nicola S, Finizio A et al. Super-resolution in digital holography by a two-dimensional dynamic phase grating. Opt Express 2008; 16: 17107–17118. | Article | PubMed | CAS |
  60. Greenbaum A, Luo W, Su TW, Göröcs Z, Xue L et al. Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy. Nat Methods 2012; 9: 889–895. | Article | PubMed | ISI | CAS |
Top

Acknowledgements

We would like to acknowledge National Institute of Health (NIH) NIH R01AI093282, NIH R01AI081534, NIH U54EB15408, NIH R21AI087107, and Brigham and Women’s Hospital – BRI Translatable Technologies and Care Innovation Grant. We would also like to acknowledge the support provided by the Brazilian National Council for Scientific and Technological Development (process 551967/2011-0). The authors would like to thank to Vasu Sah, Chiara Filippini, and Mert Keser, for their valuable efforts on samples and microscopy analysis.

Note: Accepted article preview online 29 June 2015

Note: Supplementary information for this article can be found on the Light: Science & Applications' website.

This work is licensed under a Creative Commons Attribution 4.0 Unported License. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.