96 Eyes: Parallel Fourier Ptychographic Microscopy for high-throughput screening

We report the implementation of a parallel microscopy system (96 Eyes) that is capable of simultaneous imaging of all wells on a 96-well plate. The optical system consists of 96 microscopy units, where each unit is made out of a four element objective, made through a molded injection process, and a low cost CMOS camera chip. By illuminating the sample with angle varying light and applying Fourier Ptychography, we can improve the effective brightfield imaging numerical apertuure of the objectives from 0.23 to 0.3, and extend the depth of field from ±5μm to ±15μm. The use of Fourier Ptychography additionally allows us to computationally correct the objectives’ aberrations out of the rendered images, and provides us with the ability to render phase images. The 96 Eyes acquires raw data at a rate of 0.7 frame per second (all wells) and the data are processed with 4 cores of graphical processing units (GPUs; GK210, Nvidia Tesla K80, USA). The system is also capable of fluorescence imaging (excitation = 465nm, emission = 510nm) at the native resolution of the objectives. We demonstrate the capability of this system by imaging S1P1-eGFP-Human bone osteosarcoma epithelial (U2OS) cells.

Introduction 1 e multi-well plate reader is extensively used in large for-2 mat cell culture experiments. A typical well plate reader is 3 designed to operate on either a 96-well or 384-well plate, and 4 is usually used to perform uorescence or absorbance mea- 5 surements on the contents of the wells. ese measurements 6 can be performed quickly -readers with a throughput rate 7 of 1 well plate per 10 seconds are commercially available 1, 2 . 8 However, by their nature, well plate readers can only provide 9 gross characterization of the samples. 10 To address this de cit, a number of imaging well plate 11 microscopy systems have been developed over the past two 12 decades [3][4][5] . ese systems are typically designed to use a sin- 13 gle microscope column to scan the entire well plate. e abil- 14 ity to collect microscopy images of the cell cultures provides 15 a wealth of information that simple well plate reader cannot 16 o er. With such a microscopy system, individual cells within 17 a culture can be examined for their individual morphology, 18 integrity, vitality, and their connections to neighboring cells. 19 e range of experiments can be further expanded with the 20 addition of uorescence imaging functionality. For example, 21 uorescence imaging provides the capability to track gene 22 expression in individual cells through the use of uorescence 23 biomarker methods 6 . Well plate imagers do come with a sig- 24 ni cant compromise. A single scanning microscope column 25 has a nite data throughput rate that is set by the camera 26 data rate and the mechanical scanning speed of the scanner 27 itself. As a reference point, the state-of-the-art commercial 28 well plate imager operating at an optical resolution of 1.2 µm 29 (corresponds to a typical 20× objective) can scan a complete 30 well plate in 8 min. Yet, in a per plate processing speed com-31 parison, the per plate processing time of a well plate imager 32 is ≈ 50 times longer than that of a non-imaging well plate 33 reader. 34 e paper reports on our work at addressing this per plate 35 throughput gap between a well plate reader and a well plate 36 imager through the use of parallel imaging. e idea of us-37 ing 96 objectives to simultaneously image all the wells on a 38 96 well plate may seem like an obvious and straightforward 39 way to boost a well plate imager throughout. However, a 40 more detailed analysis reveals several di cult engineering 41 challenges that need to be overcome. First, ensuring that 42 all the wells are simultaneously in focus during the imaging 43 process is a very tall order, as each multi-well plate has its 44 own unique warps. While the warp may be gradual, their 45 cumulative e ect across the wells can easily put a signi -46 cant fraction of the wells outside the depth of eld of the 47 objectives. Second, the physical size of the objectives for 48 parallel 96-well imaging is necessarily restricted due to their 49 closely packed geometry in the imaging system -each ob-50 jective cannot be larger than 6 mm in diameter. Designing 51 a scienti c-quality objective with this size constraint can 52 e FPM approach neatly solves the unique challenges 87 of high-throughput imaging on a multi-well plate format 88 described above, namely (i) defocus due to plate-to-plate over an e ective depth of eld of around 300 µm 7 , thereby 107 enabling all wells to simultaneously be within focus.

108
In addition to solving these challenges, the FPM approach 109 also brings the following advantages: (i) FPM's inherent 110 ability to improve resolution allows us to perform large eld 111 of view (FOV) imaging of cell cultures at improved resolution. 112 (ii) Computational refocusing 7 and aberration correction 9 113 can occur post data acquisition. is is a marked bene t as 114 users of conventional microscopy systems generally have no 115 recourse for image correction post acquisition. (iii) e FPM 116 image data contains phase information and we can render 117 phase images of the sample with ease.
is enables rapid, 118 live, and label-free imaging of cell cultures. 119 We have previously demonstrated that the strategy of us-120 ing FPM to implement parallel microscopy is feasible for 121 6 well plate imaging 10 . In this work, we tackle parallel imag-122 ing of 96-well plates, hence the name of our prototype -the 123 96 Eyes system. Di erent from our previous 6 well version 124 where we use research-grade objectives with near identical 125 aberrations characteristics, we custom-designed our own 126 objectives due to the high packing density required for 96 127 well simultaneous imaging. ese objectives exhibit signif-128 icant aberration variations due to surface defects from the 129 platic injection molding process, as well as the lens alignment 130 errors from the assembly process. As such, the completed 131 system relies much more signi cantly on the robustness of 132 the FPM technique in order to work. e 96 Eyes system also 133 requires signi cant engineering work on the data transfer 134 and processing as the data throughput rate is much higher. 135 During operation, the system prototype acquires and trans-136 fers the raw data at 340 MB/s. e system captures a batch 137 of 64 frames of all the wells on the well plate in 90 seconds, 138 equivalent to an imaging coverage of 3.6 mm 2 per second. 139 Equipped with a graphical processing unit (GPU) array to ac-140 celerate FPM rendering of phase images, we can achieve the 141 e ective plate-to-image pixel rate of 2 × 10 6 pixel/s. is 142 system can also support uorescence imaging at the native 143 resolution of the objectives.

144
In this paper, we will rst describe the design considera-145 tions that led to the current 96 Eyes system implementation. 146 Next, we will report on the speci cations of the objectives, 147 with our experimental ndings on the aberration variations 148 between the objectives. en, we will report on the imaging 149 process. e data acquisition process is complicated by the 150 variations in the well plate, i.e. warp and the menisci of the 151 uid within the wells. We will report on our ndings and 152 the strategies adopted to resolve the challenges. e data 153 processing pipeline and the computational hardware and 154 so ware implementations are reported in the Supplemen-155 tary Information. Last, we report our demonstrations of our 156 96 Eyes system to acquire images from xed mammalian cell 157 lines.

159
Design considerations 160 e system schematics of the 96 Eyes system is shown in 161 Figure 1. e FPM illumination is provided by an LED matrix 162 at the top of the system. e light transmits through the 163 target 96 well plate. e transmission through each well is 164 collected through an objective and projected onto a camera 165 sensor chip. e 96 objectives and camera chips are housed 166 on a customized 96-in-1 sensor board. e camera sensor 167 board is interfaced with four frame grabber boards, which 168 in turn connect to the workstation. A piezo-electric z-axis 169 stage is used to hold the well plate in place and to provide 170 z-axis translation as needed.  Fig. 1(b)]. is arrangement results in a self-repeating units 175 of compact microscopes in a 9 mm × 9 mm × 81 mm space. 176 To assemble individual CMOS sensors in such a tight layout, 177 we choose consumer-grade CMOS sensors for their smaller 178 footprint instead of the scienti c-grade sensors. e former 179 is also more cost-e ective (up to four-fold cheaper) consid-180 ering the large number (96) of sensors to be packed into the 181 system.

182
To maximize the imaging FOV without sacri cing the na-183 tive image resolution, the microscope objectives are custom-184 designed to provide a 4× magni cation, a working distance 185 of 4 mm, and a numerical aperture (NA) of 0.23. e NA of 186 0.23 is comparable to that of a standard 10X objective (e.g. 187 Newport M-10X). e unusual combination of a relatively 188 high NA and low magni cation is made possible in our de-189 sign by the use of a much shorter tube length (34 mm) by 190 design. Notably, a nite conjugate optical con guration is 191 chosen for its compactness and mechanical stability, result-192 ing in a xed object-to-image distance of 48 mm. Owing to 193 the large quantity of objectives in our system, we tolerate the 194 manufacture imperfection of the plastic-molded lenses for its 195 cost-e ectiveness over the polished individual glass lenses 196 or the lens array in single piece. As we will report in the 197 lens quality characterization below, the len-to-lens variation 198 as well as the intrinsic mis-alignment of the lens assemblies 199 will play a role in the variation of system aberrations. We 200 also demonstrate that the FPM method is able to retrieve and 201 correct these aberrations.  (Table 1), supporting a native space-206 bandwidth product of ≈ 687 (lateral) for the uorescence 207 imaging light path. For phase imaging, the resolution is 208 improved through the use of the FPM method.

209
FPM image acquisition involves illuminating the object 210 with a sequence of spatially distributed light sources. In 211 order to make sure all specimens on the 96-well plates re-212 ceive an identical set of illumination conditions, we incor-213

258
To further suppress stray light from an adjacent lens im-259 pinging on the image sensor, a wafer-shaped lens hood array 260 surrounds and shields the light path between the objectives 261 and the CMOS sensors [ Fig. 1(b)]. is light shielding can 262 be especially important in uorescence imaging where the 263 intensity of the emi ed light is much lower than that of the 264 excitation light. is is especially important in uorescence imaging mode 280 in native resolution determined by the corresponding objec-281 tives.

282
Our choice of nite-conjugate objectives also requires a 283 very speci c object-to-sensor distance. To evaluate the e ect 284 of distance deviation on the lens aberrations, we used manu-285 facturer provided simulation data of the spot diagram when 286 the distance shi s by (i) ±2 mm and (ii) ±5 mm at a wave-287 length of 530 nm [ Fig. 2(b)]. e object-to-objective-barrel 288 distance is allowed to adjust to ensure maximum resolution. 289 e data indicates that the spot size can stay within the disk 290 of 2.5 µm diameter when the sample-to-sensor distance stays 291 within ±2 mm of the ideal distance at 48 mm. is implies 292 Figure 2. Characterization of lens aberration in 96 Eyes system. (a) Illustration of the object-to-sensor distance for nite conjugate optics. e external threads of 350 µm pitch allows manual adjustment of lens-to-object distance. e inset shows the actual scale of the plastic-molded microscope objectives. (b) Spot diagram simulation of the lens at on-axis (0 µm) and o -axis position (800 µm), when the object-to-sensor distance deviates from design value (i.e. at 48 mm). In the simulation, the objective lens was allowed to move to optimize the focus. As revealed in the above study, natural geometrical warping 358 of the cell culture plates is signi cant enough that a conven-359 tional plate imager would have to compensate by mechanical 360 depth scanning and refocusing. Such a signi cant plate-to-361 plate variation of up to ±33 µm is more than 3 times the 362 native depth resolution (λ/NA 2 ≈ 10 µm at λ = 533 nm) 363 permi ed by the microscope objectives [Supp. Fig. 3]. To this 364 end, we characterize the robustness of FPM in restoring the 365 naturally out-of-focus images. Here, we intentionally and 366 precisely shi ed the sample plane of a Siemen Star phase 367 target followed by FPM acquisition [ Fig.4(a)]. e captured 368 data undergo FPM reconstruction with computational refo-369 cusing 12 . Speci cally, the wavefront error associated with 370 the focal shi was blindly estimated by minimizing the FPM 371 reconstruction residual [Eq. (S15) in Supplementary Informa-372 tion] of the refocused image with a linear search algorithm. 373 A er computational refocusing, the phase quality of the FPM 374 output is evaluated by measuring the lateral resolution. Refer 375 to Materials and methods for detailed experimental condition 376 and sample preparation. 377 e restored Siemen's star phase images are compared 378 against the raw intensity image captured from illumination 379 of a single LED [ Fig. 4(c)]. At ideal working distance from the 380 microscope objectives (i.e. δz = 0 µm), the lateral resolution 381 of the FPM restored image is ≤ 1.26 µm, below the detection 382 limit of the Siemen's star target. In contrast, the native lateral 383 resolution of the same pa ern is 2.05 µm, in agreement with 384 the theoretical resolution of the intensity image. When the 385 object is axially shi ed by δz = +30 µm, we observe a se-386 vere distortion of all spatial features below 3.5 µm. But with 387 the computational refocusing method, the restored phase 388 image is almost perfectly restored at resolution ≤ 1.26 µm. 389 Again, this is below the detection limit of the resolution tar-390 get. e phase image resolution only degrades marginally 391 at 1.45 µm at an axial shi δz = +50 µm. e experiments 392 were repeated over the range of −150 µm ≤ δz ≤ +150 µm 393 at 25 µm step size, which results in the lateral resolution 394 versus defocus distance plot [black trace in Fig. 4(e)]. e 395 restored lateral resolution of the phase image stays at around 396 1.26 µm for the defocus range of [−30 µm, +30 µm], then 397 ramps up to around 1.5 µm at δz = ±50 µm, and around 398 2.0 µm at δz = ±100 µm. 399 We performed a similar experiment with an adherent cell 400 line seeded and xed in a 96-well plate [ Fig. 4(c)]. Refer to Ma-401 terials and methods for the cell preparation protocol. e raw 402 6/14   (Table 1).
492 Figure 5 shows the resulting images as collected with the 493 96 Eyes. Figure 5 ing, the uorescence image is limited by the native DOF 517 imposed by the microscope objectives. Here, the plates 518 are scanned along z-axis with a step size of 25 µm (around 519 twice the native depth resolution (= 10 µm) permi ed by 520 the custom-designed microscope objective), and a range of 521 100 µm (around the 90% con dence well-depth range of the 522 UV-Star plate).

523
Discussions 524 We characterized the lens aberrations from identically-525 designed microscope objectives, which reveals a variation of 526 0.88 radian on astigmatism and 0.25 radian on spherical aber-527 ration. Such a tight lens aberration tolerance is required for 528 consistent measurement for multiple plates, which ultimately 529 improves the statistical strength of high-throughput biomed-530 ical studies. is is particularly important in uorescence 531 imaging because the image quality is directly tied to the na-532 tive image resolution dictated by the microscope objectives. 533 Although we previously demonstrated that the uorescence 534 image can be deblurred with the recovered aberration from 535 the phase image channel 15 , the method is not used in this 536 prototyping project as the image quality is su cient in the 537 96 Eyes system. e tight tolerance also improves the data 538 analysis of phase images recovered by FPM. Even though 539 they can be aberration-corrected computationally, a strong 540 manufacture variation of the objectives can cast doubts on 541 the data consistency of the images captured among di erent 542 lens. For instance, we have shown in the extended depth-of-543 focus study that the lateral resolution can be degraded by 544 a fraction even with accurate estimation of the aberration 545 wavefront. us, it cannot be ruled out that strong values in 546 higher order modes of aberrations can further degrade the 547 phase images. Here, a tight manufacturer tolerance of the 548 microscope objectives helps eliminate such doubts, and in 549 turn improves the statistical strength for high-throughput 550 studies with the 96 Eyes system. 551 e FPM-based aberration correction poses a unique 552 strength in determining the overall aberration of individual 553 objectives on the y. For instance, it uses the exact same 554 illumination condition and optical con guration as the phase 555 image acquisition of cell samples. No alternation of the light 556 path is required. More importantly, embedded aberration 557 recovery with FPM preserves the direction of the aberra-558 tion components (e.g. slopes of the tilted wavefront, major 559 axes of the astigmatisms) of the microscope objective at the 560 calibrated position in the 96 Eyes system. Such directional 561 information is essential in correcting aberration of the phase 562 images, which can be lost by merely rotating the lens barrel 563 -a necessary step in assembly and lens-to-sensor distance 564 calibration. Notably, the best alternative is to move the objec-565 tives from the 96 Eyes instrument to the specialized optical 566 pro ler for aberration analysis. e measured lens aberration 567 map may have served the purpose of lens tolerance study, 568 but has limited value in the aberration correction of phase 569 images. To this end, we are able to compensate for the dis-570  for photo-chemical analysis 16,17 , in which optical precision 589 is not a requirement. In spite of a number of recent a empts 590 to re-design the multi-well plate to address the above is-591 sues 18,19 , the 96-well plate format remains popular among 592 researchers and thus remains the gold standard in high-593 throughput screening applications 17 .

594
Notably, the plate-to-plate depth variation concern is 595 unique to our 96 Eyes system, as the lens positions (i.e. lens-596 to-sensor distance) are locked during image acquisition of 597 multiple plates. is is in contrast to conventional plate scan-598 ning microscopes, where either the objective or the plate can 599 be can be mechanically actuated to focus on individual wells, 600 albeit with a signi cant tradeo in speed. In our current 601 implementation of the 96 Eyes system, we simply pick the 602 best plate type and let the FPM phase retrieval algorithm 603 correct the aberration associated with the plate warping. In 604 our study, we have not looked into the connection of well 605 depth variation to various aberration modes in the imaging 606  We introduced the 96-in-1 parallel imaging system (96 Eyes) 663 for high throughout cell screening. By using 96 repeating 664 units of low-cost image sensors and custom designed objec-665 tives to simultaneously image all the wells on a 96-well plate, 666 the overall imaging throughput can be improved signi cantly. 667 Here we addressed and characterized the concern of lens-to-668 lens variations and deviations from the ideal sample-to-lens 669 distance; both issues can result in poor image quality. We 670 also characterized the culture plate warping by comparing 671 two plate types, and observed its e ect on the out-of-focus 672 aberrations in the imaging system. e e ect of liquid menis-673 cus on the image quality is much stronger than anticipated, 674 introducing additional image aberration. We heavily rely on 675 the FPM method to overcome the above challenges, utiliz-676 ing the embedded pupil recovery algorithm with adaptive 677 step size, as well as the meniscus-compensating ray trac-678 ing method. Equipped with the custom designed 96-in-1 679 CMOS array board, we successfully captured ptychographic 680 intensity images of a 96 well plate within 90 seconds, and 681 uorescence images within 30 seconds. e ptychographic 682 images are processed with GPU acceleration, feeding end-683 users with high resolution (1.2 µm), aberration-free phase 684 contrast images of 1.1 mm by 0.85 mm per condition at the 685 extended depth-of-focus of ±15 µm. 686 We also note the potential of a be er imaging robustness 687 through hardware improvement. For instance, 96 Eyes sys-688 tem can be upgraded with variable-focus micro-actuators or 689 liquid lenses to control the imaging focus independently, so 690 that the system can accept a wider range of 96-well plates 691 with a more relaxed plate warping constraint. In particu-692 lar, meniscus-free culture plates 26,27 can be adopted to the 693 96 Eyes system for a more predictable illumination angles, 694 thus reduces the errors in FPM phase restoration. On the 695 other hand, the optical sensitivity to uorescence can be in-696 creased with a be er powerline noise rejection and thermal 697 management of the 96-in-1 CMOS sensor array. A second 698 uorescence channel at longer wavelengths (i.e. between 699 560 nm and 700 nm) can also be included in the system with 700 a multi-bandpass emission lter and a second excitation light 701 source. However, since the plastic-molded microscope objec-702 tives were designed at a narrow range of wavelengths (i.e. 703 around 530 nm), further studies are required to evaluate the 704 aberrations at longer wavelengths. 705 We anticipate that the 96 Eyes system will help accelerate 706 biomedical and pharmaceutical research that utilizes high-707 throughput cell imaging and screening format. To measure the overall atness of COC microplates (Grenier 788 UV-Star Cat. No. 655801) compared to the polystyrene-based 789 microplates (Grenier Cell Star Cat. No. 655160), the Opera 790 Phenix Absorbance Microplate Reader was used to measure 791 the distance from the ange to the well bo om. Sixteen 792 (16) plates from two models of the 96-well plate, are seeded 793 with cells (described above) before the measurement. e 794 di erence between the measured distance and the minimum 795 distance of all measured plates are used to generate an overall 796 well depth pro le.

797
An FPM microscope, retro ed from a standard wide eld 798 microscope (Olympus Model XL-41) equipped with a 20X 799 objective (Olympus Plan N 20X; 0.4 NA) 7 , was used to mea-800 sure the atness within each well. e well was rst seeded 801 with polystyrene microspheres (described above) before pty-802 chographic imaging. Five (5) locations within the selected 803 24 wells of a 96-well plate are measured by the FPM tech-804 nique with local aberration recovery, which in turn was used 805 to compute the defocus distance from the focal plane. e 806 di erence between the maximum and minimum value was 807 calculated to determine the variance in atness within each 808 well. e variance of each well was than averaged to deter-809 mine overall well atness.