Time-of-Flight Microwave Camera

Microwaves can penetrate many obstructions that are opaque at visible wavelengths, however microwave imaging is challenging due to resolution limits associated with relatively small apertures and unrecoverable “stealth” regions due to the specularity of most objects at microwave frequencies. We demonstrate a multispectral time-of-flight microwave imaging system which overcomes these challenges with a large passive aperture to improve lateral resolution, multiple illumination points with a data fusion method to reduce stealth regions, and a frequency modulated continuous wave (FMCW) receiver to achieve depth resolution. The camera captures images with a resolution of 1.5 degrees, multispectral images across the X frequency band (8 GHz–12 GHz), and a time resolution of 200 ps (6 cm optical path in free space). Images are taken of objects in free space as well as behind drywall and plywood. This architecture allows “camera-like” behavior from a microwave imaging system and is practical for imaging everyday objects in the microwave spectrum.


Supplementary Materials
• Additional Related Work • External Database S1 • Figures S1-S11 • Table S1 Additional Related Work Please see Table 1 for a numerical performance comparison of this time-offlight microwave camera and other related work.
Phased Arrays and Synthetic Aperture Radar Work began on phased array radars in the 1960's as a method to dynamically control the beam direction of a radar antenna [1]. Phased-arrays have the advantage of a large field-of-view and fast scans, however they require many phase-coherent elements to form an image [1]. Phased array systems on aircraft early warning radars were composed of thousands of phase-controlled elements. Synthetic aperture systems form large radar apertures by scanning a single phasecoherent element across an aperture. This has been used for NASA's shuttle radar topography mission to achieve a spatial resolution of 5-10m by sampling an aperture of 50km at a density of λ/2 in order to avoid aliasing [2]. Our methodology does not sample the camera aperture with active elements, and instead captures reflected waves using a passive parabolic reflector and concentrates the active elements to a smaller focal plane. This allows for high resolution imaging with fewer measurements required to form an aliasing-free image (< 1/4 of the measurements), however at a limited field of view and depth of field.
Focal Plane Array Optical imaging techniques have also been applied to millimeter wave (mmW) and THz radio telescopes, where focal plane arrays (FPAs) are placed at the focal point of a parabolic reflector. A wideband 86 GHz-115 GHz FPA was developed for the 14 m radio telescope at the Fife College Radio Astronomy Observatory (FCRAO) consisting of 15 elements [3,4]. FPAs for 1.3 mm and 3 mm were also designed for the 12 m radio telescope at the National Radio Astronomy Observatory (NRAO) and the 14 m radio telescope at FCRAO [5]. A 160 element FPA bolometer for the 10 m radio telescope in the south pole, as presented in [6], was developed to operate at 90 GHz, 150 GHz, and 220 GHz. A high resolution map of cosmic background radiation was acquired by the Plank radio telescope satellite. The lower frequency FPAs for 30 GHz and 70 GHz and the optical systems were described in [7]. Additional FPAs at mmW bands were developed to measure cosmic background radiation polarization [8] and for 230 GHz radio astronomy applications [9]. At microwave frequencies, [10] proposed and modeled an FPA concept for 18 GHz-26.5 GHz using an array of Vivaldi antenna elements. These are radio telescopes where the instrument is focused at infinity and a limited number of pixels are used. Advanced focal plane array concepts have been developed for THz frequency ranges, such as the 1.6 THz FPA proof-of-concept discussed in [11]. To achieve mmW performance using 65 nm CMOS technology, [12] presents an interesting approach for a 200 GHz, 4-by-4 element FPA implemented using a regenerative receiver architecture. Other work with 230 GHz general purpose FPAs is ongoing [13].
Computational Computational approaches have been applied to microwave imaging for through-wall imaging, non-destructive testing, and human interaction. Adib et al. show in [14,15] that a set of multiple-input-multipleoutput (MIMO) Wi-Fi radios can be used to localize reflectors through walls and track the reflectors' position. Their work uses adaptive array processing to null the first flash from the wall. Charvat et al. [16], use a MIMO radar system to localize targets through walls and concrete, and an analog filter to reduce the flash from the wall. Our system surpasses these systems in being able to return a full X-Y image with 3D shape information to the user. Ghasr et al. demonstrated a microwave camera at 24 GHz with a PIN-diode based slot-array multiplexer [17]. Our method achieves images at better angular resolution with a fewer number of measurements due to the large passive-aperture in our method.
There has also been work in reducing the number of measurements required to perform imaging through compressed sensing. Spinoulas et al. introduce a lens and mask based sparse capture/reconstruction for passive millimeter wave imaging [18]. Chan et al. proved compressed sensing could be used in the terahertz regime [19]. We extend this work to active, coherent microwave imaging which means our architecture can address sparse measurements in both X-Y and in depth. Cull et al. demonstrate compressive millimeter wave holography by subsampling and reconstructing a Gabor hologram [20]. Hunt et al. show that a metamaterial aperture with a frequency dependent aperture function can be used to image metal targets, and demonstrate using a retroreflective corner cube [21]. They further show that the number of measurements can be reduced through sparse reconstruction. While their work addresses many of the complications for consumer radar imaging systems, our work does not trade-off angular sampling for frequency, and enables 3D radar imaging with spatially-independent spectral information, allowing the user to arbitrarily break down an image by its spectral information at no loss of X-Y resolution. This has also been accomplished using dynamic metamaterial apertures [22]. Our architecture is able to incorporate multi-spectral capabilities while still providing depth, 3D shape, and a reduced number of sensor elements.

Focal-length versus Frequency
The focal point (spatially) for an ideal reflector will not change with frequency, however the antenna gain will increase as the wavelength decreases (see Figure S1). Conducting parabolic reflectors (not lenses) at these wavelengths/bandwidth show negligible dispersion characteristics. In measuring our setup, we achieved close to the resolution that we expected. For those reasons, we assume that our setup approximates a reflective parabolic mirror. Furthermore, changes in antenna gain due to frequency are equalized during our calibration procedure.

Angular Resolution of the System
We note that the diffraction-limited resolution of our system should be where D is the aperture of the system (1.22 meters). The angular resolution thus depends on wavelength.
In order to measure the PSF of the system, we used 1/4 in radius spheres placed at a distance of 2.06 m in front of our camera, thus each sphere covered 0.35 degrees (much below the resolution of our system). Thus each sphere can be considered a point source for the PSF. The first row of spheres were spaced apart so that the camera viewed them as separated by 3.5 degrees. Figure S2 shows the target setup (with point reflectors) on the left. On the right, you can see a reconstruction which is corrected for the camera projection, with the correct positions overlaid on top in red circles. Note that this is a horizontal view of 3D data (X,Y).
By capturing the intensity along the X dimension at various frequencies we can show the width of the PSF. Here we show the width of the PSF at a wavelength of 2.56 cm to be: 1.512 degrees. Aberrations in our parabolic reflector and positional error in measurements can lead to a wider PSF than expected.
At higher frequencies, this PSF is smaller, however the image SNR increases due to the bandwidth limitations of our electronics. A higher center frequency will lead to a tighter PSF. In the revision, we address this by stating that our system can achieve a theoretical resolution of 0.5 degrees, given a perfect reflector and sufficient system bandwidth.

Materials and Methods
The experimental setup is shown in Fig. S3. Akin to a flash for an optical camera, this work uses an antenna (ANT1 in Fig. S4) to illuminate the scene. This microwave camera uses an ultra-wideband (UWB) antenna emitting a continuous wave (CW) of which the frequency is linearly swept from 7.835 GHz to 12.817 GHz, for a bandwidth, BW , of 4.982 GHz, over a chirp width, T p , of 10 ms. This is represented by where f osc is the start frequency of the linear-ramp modulated voltage controlled oscillator (VCO) and c r is equal to the radar chirp rate BW/T p . Scattered microwaves are collected and focused by a lens-equivalent dish of radius r. Typically a parabolic reflector focuses objects in the far field to the focal point f = r/2. In this paper the target scene is in the near-field of the dish and is focused to a point between r and r/2 (Fig. S5). Using a parabolic reflector is less expensive and less difficult to mount than using an equivalently sized dielectric lens.
An X band (WR-90) waveguide probe (ANT2) is mounted to an X-Y translation stage and is used to sample the fields over the image plane. Scattered and focused signals are collected by ANT2 and are represented by (without loss of generality, ignoring amplitude coefficients) Roundtrip time t delay is defined as time taken to reach and scatter off of the target, return to and be focused by the dish, and be collected by the waveguide probe (ANT2).
The probe signal is fed into the receive port of a UWB frequencymodulated continuous-wave (FMCW) receiver [23]. The received delayed FMCW signal from ANT1 is then amplified by a low-noise amplifier (LNA1) then fed into a frequency mixer (MXR1). The illumination chirp is also fed into MXR1 which multiplies the sampled chirp by the reflected chirp, This product is amplified by the video amplifier and then digitized. This analog signal is digitized at a rate of 200 kHz.
The high frequency term of this cosine multiplication is rejected by the low-pass filter within the video amplifier resulting in When there is only one target present V (t) is a sinusoidal wave with frequency proportional to t delay ; if multiple targets are present then V (t) is a superposition of numerous sinusoids, each with frequencies proportional to the round-trip delay to its corresponding reflector. A cube of data (angle, angle, time) is acquired using an X-Y translation stage where a 41 pixel by 41 pixel image is sampled. At each position 2000 analog samples are acquired and synchronized to the chirp time, T p . This results in the signal V (x n , y n , t), where x n and y n are the horizontal and vertical positions, respectively, of the waveguide probe, ANT2, in the 2D detector sampling plane.
To process and generate a microwave image, the time-average intensity of the entire signal V (t) is computed for each pixel x n and y n , resulting in the image s image (x n , y n ).

Calibration, Time, and Color
Calibration In order to calibrate the system's frequency response, we acquired data with a baseline target, an aluminum sphere placed in front of the camera, and performed background subtraction. We compared the measurement with the sphere with the Mie series solution for an ideal sphere of the same size [24]. The system phase and frequency response is calibrated by dividing the Mie data cube S M ie by the calibration image S cal : To apply calibration, each sample over V (t) in the data cube is multiplied by the calibration factor S cf , resulting in the calibrated data cube S calibrated x n , y n , V (t) .
Time Domain To convert to the range domain after calibration is applied to V (t), we observe the magnitude of the Fourier transform (DFT) at each pixel, resulting in S x n , y n , t delay . We note that the time/frequency resolution of the system is limited by the bandwidth of the chirp.

Multi-Spectral
The spatial frequency domain data cube is divided up evenly into three sub-bands over V (t) of 666 samples each. These bands are color coded as red, green, and blue with mean center frequencies of red (8.66 GHz), green (10.32 GHz), and blue (11.97 GHz) respectively. To do this, s x n , y n , V (t) becomes s red x n , y n , V red (t) , s green x n , y n , V green (t) , and s blue x n , y n , V blue (t) . A DFT is applied to each sub-band in order to provide the range domain response of each color. The imagery is displayed in both range and color. Note that since FMCW processing uses system bandwidth to determine depth, each spectral division reduces the range domain resolution of the system. The reduction in range domain resolution is linearly related to the number of sub-bands desired in the final image. An additional calibration step is taken when imaging in multi-spectral mode, white balance. White balance is achieved by imaging a relatively large (i.e. compared to a wavelength) sphere and scaling all color bands to the same amplitude as the center of the image. White balance is similarly applied to all multi-spectral imagery thereafter. The color image of a 76.2 mm diameter sphere in Fig. S6 shows the response of the broadband reflection of a sphere after white balance correction.

Design Space
The camera described in this paper enables us to view the world at microwave frequencies by combining optical design with time-resolved microwave techniques. The current implementation over samples the diffractionlimited image.
The Fresnel number for our proposed system at the largest wavelength is roughly F ≈ 20. Since F 1, Fresnel diffraction was used to model wave propagation of the imaging system. Geometric optics takes advantage of the small wavelengths of the relatively short wavelengths of visible light to make ray approximations for light propagation. However, at microwave and millimeter wavelengths the system design is more likely to run into the diffraction limit, thus it is important to consider pixel spacing, aperture size, and depth of field.
We follow the derivation of [25] and begin by deriving an expression of the wave field reflected off the scene measured at the image plane using the Fresnel diffraction integral and considering a point source at ξ , η (see Fig. S7) where P (x, y) is the 2D aperture function in the lens plane. The imaging equation 1/f = 1/d o + 1/d i makes the quadratic terms A and B go to zero. By introducing a magnification term M = −d i /d o and the change of baseŝ ξ = M ξ andη = M η, the wave at the image plane can be expressed by the following convolution: where:ĥ (a, b) = 1/|M | P (x, y)e −j2π(ax+bŷ)dxŷ .
Thus the point spread function (PSF) can be expressed as When P (x, y) is a disk the size of the reflector, the solution for the PSF is the sombrero function resulting in an Airy disc. The Raleigh resolution limit of the system is defined by the first zero of the Airy disc which is where D is the diameter of the parabolic reflector. The Airy disc defines the diffraction limited pixel pitch of the system and also its depth of field (DOF). Objects closer than the object plane will be focused to a point behind the image plane and thus will cast a circle of confusion on the image plane (Fig. S8). Similarly objects farther than the object plane will cast a circle of confusion. If the circle of confusion is smaller than the pixel pitch of the system, all scattering points within the depth of field will be in focus. Having a large depth of field is desirable for depth-imaging using the microwave camera system; however, a shallow depth of field has the advantage of allowing the system to focus-through clutter. In this derivation, the pixel pitch is placed at the diffraction limit of the system.
It is important to note that many approximations are made when using a parabolic reflector for imaging. The expression derived above through Fresnel-diffraction relies on the paraxial approximation in order to analytically solve the diffraction integral. In order to develop a more precise solution, we simulated Fresnel-Kirchhoff diffraction of the system to determine the focusing properties of the system at microwave wavelengths. An example of these simulations is shown in Fig. S7 and S9. While a parabolic reflector approximates a perfect lens, it suffers from off-axis aberrations such as comma for all points off-axis. Furthermore, while the focal point of a parabolic reflector is well-understood for point sources in the far-field, the focal length of the reflector deviates from the lens-law as the point source comes closer to the parabola.
When imaging a scene using a lens a practical concern is the depth of field of the system since scene reflectors out of the depth of field will have larger PSFs. Here we define the frequency dependent depth of field of the system. The depth of field as defined by the lens-law, PSF, and wavelength dependent circle of confusion, is expressed below.
The near focus of the system is determined using the lens law and assuming that pixels are spaced at the diffraction limit. This allows us to calculate a circle of confusion and determine the near and far focus of the system for a given focal length and wavelength.
Here f is the focal length of the system, I is the distance between the aperture and the sensor, D is diameter of the parabolic reflector (Fig S8).
O f ar = f ( 1.22λ D 2 + 1) 1.22λ This is the expression of the far focus, found by calculating the object distance which would cause a circle of confusion on the sensor of equivalent size to the diffraction limited pixel pitch. Subtracting these two equations results in a depth of field of the system: Fig. S10 shows that for any given wavelength λ, the DOF is inversely proportional to the diameter increases. Intuitively, a larger aperture leads to a shallower depth of field. There is a point, however, where the DOF becomes infinite defined by f = 1/(1 + 1.22λI/d 2 ). At this point the DOF is long and the system is able to focus on many depths within the scene. At this point though, both the distance between the scene and the camera, and the pixel pitch (size of the receiving array) becomes large.
In order to achieve the maximum DOF for a diffraction limited system at a given pixel-pitch, one should select the largest focal length possible for the reflecting optics; however, this comes at the cost of increasing the object distance.
Knowing the DOF and out-of-field PSF is important for future applications involving model-based reconstruction through deconvolution such as that shown in [26,27]. In order to theoretically determine the effect of focal-displacement on PSF, we show the ray-optics theoretical curve, a simulated wave-optics theoretical curve, and an experimentally measured curve in Fig. S11. This information is important to understand imaging performance outside of the system's DOF.
The performance of the camera and a comparison to lens-based and lensless cameras are shown in Table S1.

Additional Results
Results in addition to those discussed in the main paper are presented here for completeness.
Grid Fly Through A frame by frame "fly-through" of a grid of spheres is shown in Fig. S12. The grid is shown in Fig. S13. This demonstrates the capability of the camera to separate a set of 15 reflectors into five different depth planes. This also demonstrates that despite shadowing, the reflectors are still visible. Furthermore it can be seen that reflectors in the back-row are closer to the optical axis (center pixel) than the front row. This is due to the camera projection.

Movie S1
Movie S1 shows the system and time-of-flight capabilities.

External Database S1
Please find code and data available at http://scripts.mit.edu/MicrowavePackage.zip/ Figure S1: The focal point of the dish does not vary with frequency. In A, we show a 1D slice of Fresnel-diffraction simulations of an ideal source being focused onto a focal plane after reflecting off of a discretized reflector at 3 different wavelengths (2cm, 6cm, and 12cm). The deviation between the focal points is minimal. In B, we show the magnitude of the wave across 2 spatial dimensions. In C, we show the phase of the wave across 2 spatial dimensions.      Figure S10: Selecting system geometry parameters such as dish diameter and focal length affect the system depth of field. The DOF for the camera design space is shown in (A). In order to increase depth of field, either the diameter or the focal length of the reflector can be increased. Increasing the diameter of the dish also increases the minimum pixel pitch (C), thus increasing the size of the focal plane array. By increasing the focal length of the dish, the object distance from the camera must be increased (B). The red lines here signify the operating characteristics of our camera. Figure S11: The PSF of Out of Focus (OOF) reflectors is larger than in-focus reflectors. Here the theoretical-ray, theoretical-wave, and experimentallymeasured PSF of out of focus elements is shown. The experimentallymeasured curve was found by placing reflectors displaced from the focal point and measuring their full width half maximum. Note that the rayoptics perspective approaches an infinitesimally small focus point, while the wave-optics curve is diffraction limited. The theoretical ray-optics curve is shifted to correct for the near-field aberration caused by the parabolic reflector so that the focal point of the ray-optics perspective from the lens equation matches the wave-optics focal point. Figure S12: A fly-through of the grid of 0.5 inch diameter spheres, where although of equal spacing throughout the scene, the three spheres appear to be getting closer and closer to each other as time progresses (A-E). This is due to the camera projection. This is intuitive if one were to contemplate how railroad tracks converge to a vanishing point as they travel into the distance as viewed by the eye.  Table S1: Common metrics for evaluating a camera are shown above for the microwave camera presented along with values or equations for both lens-based and lenless cameras. Here it is demonstrated how a lens/reflectorbased imaging system has advantages over a lensless system since it is possible to achieve a large aperture and a reduced element count while avoiding aliasing and ghosts in images [28,25]. NA implies the metric is notapplicable. NL implies the metric was not listed in the publication. N is the number of measurements, D is the diameter of the aperture, d o is the object distance, d i is the imaging distance, BW is the bandwidth of the system, and λ is the wavelength of transmission. Ranges are specified for papers wherein multiple sparse configurations were used.