How did the In-Situ Storage Image Sensor camera come into existence?

Takeharu Etoh from the School of Science and Engineering, Kinki University:

Images of the dynamics of a water–glycerin net captured by the Shimadzu Hypervision HPV-1 high-speed camera operating at 5,000 fps. Images courtesy of Takeharu Etoh.

Commercially available high-speed digital video cameras typically feature partial and parallel readout. In 1991, I developed the world's first high-speed video camera with 16 parallel readout ports. It had a speed of 4,500–40,500 frames per second (fps) and was marketed by Kodak in 1993 as the Kodak Ektapro HS4540. It became a de facto standard for high-speed video cameras at the time. These cameras were used extensively in the automobile industry to study combustion processes, and improve the efficiency of fuel consumption. But users soon complained that it was too slow. The speed was limited by the time taken to read out the signals from the photodiodes on the sensor to the external memory device. Our solution was much simpler and involved positioning a linear CCD memory adjacent to the photodiode to create what's known as a local-memory image sensor. We called this sensor the In-Situ Storage Image Sensor (ISIS), and it is capable of one million fps. In 2001, we developed an ultrahigh-speed video camera based on the sensor, in collaboration with the Japanese company Shimadzu. The camera is now marketed as the Hypervision HPV-1 system and is commercially available. It can store up to 100 monochrome frames, each with a resolution of 312 × 260 pixels, and it can operate at up to 1,000,000 fps.

What are the main features of the ISIS and how can it be improved?

The ISIS-V2 is a monochromatic sensor with 312 × 260 pixels. A colour version with 420 × 720 pixels, the ISIS-V4, has been developed by NHK, Japan's public broadcasting company. The ISIS is being used by scientists and engineers for applications such as studying shock waves on the surfaces of liquids and laser ablation of soft gels. In ultrahigh-speed imaging, the camera's sensitivity to light is an important factor in its overall performance. In 2003, we started development of a photon-counting camera, the PC-ISIS, with the aim of achieving both ultrahigh sensitivity as well as ultrahigh speed. We expect to be able to reach 100 million fps and sensitivity of less than 10 photons. This will be achieved by combining the ISIS structure with three technologies for the ultrahigh sensitivity: backside illumination of the image sensor; cooling; and charge-carrier multiplication. The PC-ISIS has a pixel size of 43.2 × 43.2 μm2, and is equipped with 120 in situ CCD memory elements (each 3.0 × 3.6 μm2), which enable 120 consecutive images to be captured. The design has 480 × 360 pixels and the total photoreception area is 20.73 × 15.55 mm2. It is sensitive to wavelengths in the UV and visible, in the range 350–650 nm. There are also demands for higher spatial resolution, which will require the size of the ISIS to be increased.

What are the applications of such cameras?

The first real important application was the imaging of the droplets generated in the high-performance ink-jet printers that are now sold in large quantities around the world. Another recent application we are looking at is the 'bio-nanoscope'. This is a project funded by the Japan Science and Technology Agency (JST), and is being carried out by myself and Atsushi Miyawaki of RIKEN, with industrial support from Olympus and NHK Engineering Service. We are looking at biological applications of ultrafast video cameras, such as the movement of calcium inside cells. It's a very challenging application area. One feature of this work is that it is extremely difficult to control the movement of living cells and the problem is deciding when to start or stop imaging. With an explosion it is easy to trigger the explosion and synchronize the imaging. In biology it is difficult to predict when the interesting phenomena will occur. Other biomedical applications we are considering are 'cell surgery', where we want to monitor laser slicing of cells. I expect that even more applications will follow in the future. People are often surprised to hear that I am not an electronics engineer and that my background is in fluid dynamics. I initially thought of producing a high-speed video camera to study the flow of water, but I did not expect it to become my life work.

Adarsh Sandhu is a professor at the Tokyo Institute of Technology.

Supplementary material related to this article, including a series of movies of ultrafast events, is available here .