Tetsuya Hayashida

6.2 133Mpixel 60fps CMOS image sensor with 32-column shared high-speed column-parallel SAR ADCs

To realize next-generation highly realistic sensation broadcasting systems, the research and development of 8K ultrahigh-definition television (UHDTV) systems have been promoted. To realize 8K video cameras, 33Mpixel sensors [1-2] and a full-resolution camera system that uses three 33Mpixel sensors [3] have been reported. However, the weight of the camera with three sensors is over 40kg because the camera requires a large-format color-separation prism. To reduce the size of the camera, single-chip imaging is a promising approach, and a compact single-chip 8K camera that weighs only 2kg has already been developed using a color 33Mpixel CMOS image sensor [4]. However, a conventional single-chip camera has a lower image quality than a full-resolution camera because the total pixel count of the single-sensor camera is only one-third of that of the three-sensor camera, and pixel interpolation is required to configure a full-resolution image. In this paper, a 133Mpixel sensor that can be operated at 60fps to realize a full-resolution 8K single-chip camera is described. To achieve both high speed and suitable ADC resolution, 32-column multiplexing analog readout circuitry and 14b high-speed redundant successive approximation register (SAR) ADCs [5] are adopted. As a result, a full-size image with a data rate of 128.71Gb/s at 60fps has been captured.

A 16 Mfps 165kpixel backside-illuminated CCD

In 2002, we reported a CCD image sensor with 260×312 pixels capable of capturing 103 consecutive images at 1,000,000 frames per second (1Mfps) [1]. We named the sensor “ISIS-V2”, for In-situStorage Image Sensor Version 2. 103 memory elements are attached to every pixel; generated image signals were instantly and continuously stored in the in-situstorage without being read out of the sensor. The ultimate high-speed recording was enabled by this parallel recording at all pixels. In 2006, the color version, ISIS-V4, was reported [2]. In 2009, we developed ISIS-V12, a backside-illuminated image sensor mounting the ISIS structure and the CCM, charge-carrier multiplication, on the front side [3]. The CCM is a CCD-specific efficient signal-amplification device. CCM, combined with the BSI structure and cooling, achieved very high sensitivity. The ISIS-V12 was a test sensor intended to prove the technical feasibility of the structure. The maximum frame rate was 250kfps for a charge-handling capacity of Qmax=10,000e− and 1Mfps for a reduced Qmax. The pixel count was 489×400 pixels. For backside-illuminated (BSI) image sensors, metal wires can be placed on the front surface to increase the frame rate without reducing fill factor or violating uniformity of the pixel configuration. It has been proved by simulations that 100Mfps is achievable by introducing innovative technologies including a special wiring method [4]. We now report on ISIS-V16, developed by incorporating technologies to increase the frame rate with those to achieve very high sensitivity, which was confirmed by evaluation of ISIS-V12. The performance specification of ISIS-V16 is summarized in Fig. 23.4.1.

Back-side-illuminated image sensor with burst capturing speed of 5.2 Tpixel per second

We have developed a back-side-illuminated image sensor with a burst capturing speed of 5.2 Tpixels per second. Its sensitivity was 252 V/lux·s (12.7 times that of a front-side-illuminated image sensor) in an evaluation. Sensitivity of a camera system was 2,000 lux F90. The increased sensitivity resulted from optical and time aperture ratios of 100% and also by increasing from a higher optical utilization ratio. The ultrahigh-speed shooting resulted from the use of in-situ storage image sensor. Reducing the wiring resistance and dividing the image area into eight blocks increased the maximum frame rate to 16.7 million frames per second. The total pixel count was 760 horizontally and 411 vertically. The product of the pixel count and maximum frame rate is often used as a figure of merit for high-speed imaging devices, and in this case, 312,360 multiplied by 16.7 million yields 5.2 Tpixels per second. The burst capturing speed is thus 5.2 Tpixels per second, which is the highest speed achieved in high-speed imaging devices to date.

A 252-

We developed a 312-kpixel back-side-illuminated ultrahigh-speed charge-coupled device (CCD) that has a sensitivity of 252 V/lux · s and is capable of operating at 16.7 Mfps. The potential profile of the pixel was designed by using a 3-D semiconductor device simulator. The high sensitivity results from the unit having fill factor and time aperture ratios of 100% and a high optical utilization ratio. Its sensitivity is 12.7 times that of a front-side-illuminated image sensor. Ultrahigh-speed shooting was enabled by an in situ storage image sensor. By reducing the wiring resistance and dividing the image area into eight blocks, a maximum frame rate of 16.7 Mfps was attained. The total pixel count is 760 horizontally and 411 vertically. The burst capturing speed is thus 5.2 Tpixel/s, making it the fastest imaging device to date.

{\rm V/lux}{\cdot}{\rm s}

We are developing an ultrahigh-speed, high-sensitivity broadcast camera that is capable of capturing clear, smooth slow-motion videos even where lighting is limited, such as at professional baseball games played at night. In earlier work, we developed an ultrahigh-speed broadcast color camera1) using three 80,000-pixel ultrahigh-speed, highsensitivity CCDs2). This camera had about ten times the sensitivity of standard high-speed cameras, and enabled an entirely new style of presentation for sports broadcasts and science programs. Most notably, increasing the pixel count is crucially important for applying ultrahigh-speed, high-sensitivity CCDs to HDTV broadcasting. This paper provides a summary of our experimental development aimed at improving the resolution of CCD even further: a new ultrahigh-speed high-sensitivity CCD that increases the pixel count four-fold to 300,000 pixels.

, 16.7-Million-Frames-Per-Second 312-kpixel Back-Side-Illuminated Ultrahigh-Speed Charge-Coupled Device

The authors have been developing ultrahigh-speed, high-sensitivity broadcast cameras that are capable of capturing clear, smooth, slow-motion video even in conditions with limited lighting, such as at professional baseball games played at night. In 2003, the first broadcast color camera using three 80,000-pixel ultrahigh-speed, high-sensitivity charge-coupled devices (CCDs) was developed. This camera is capable of ultrahigh-speed video recording at up to 1,000,000 frames/sec, with about ten times the sensitivity of standard high-speed cameras. It has enabled an entirely new style of presentation for sports broadcasts and science programs. The authors continue research to improve the cameras resolution. This paper discusses the development of the first ever ultrahigh-speed high-sensitivity CCD with 300,000 pixels—a four-fold increase over the previous version, as well as the development of a single-chip portable color camera mounted with this CCD.

Development of a 300,000-pixel ultrahigh-speed, high-sensitivity CCD

We developed an ultrahigh-speed, high-sensitivity, color camera that captures moving images of phenomena too fast to be perceived by the human eye. The camera operates well even under restricted lighting conditions. It incorporates a special CCD device that is capable of ultrahigh-speed shots while retaining its high sensitivity. Its ultrahigh-speed shooting capability is made possible by directly connecting CCD storages, which record video images, to photodiodes of individual pixels. Its large photodiode area together with the low-noise characteristic of the CCD contributes to its high sensitivity. The camera can clearly capture events even under poor light conditions, such as during a baseball game at night. Our camera can record the very moment the bat hits the ball.

An Ultrahigh-Speed, High-Sensitivity, Portable CCD Color Camera

We developed a 300,000-pixel ultrahigh-speed CCD with a maximum frame rate of 2,000,000 frames per second. The shooting speed of the CCD was possible by directly connecting CCD memories, which record video images, to the photodiodes of individual pixels. The simultaneous parallel recording operation of all pixels results in the ultimate frame rate. We analyzed a voltage wave pattern in the equivalent circuit model of the ultrahigh-speed CCD by using a SPICE simulator to estimate the maximum frame rate. The pixel area was consisted of 410 and 720 pixels in the vertical and horizontal and divided into 8 blocks for parallel driving. An equivalent circuit of one block was constructed from an RC circuit with 410 × 90 pixels. The voltage wave pattern at the final stage of an equivalent circuit was calculated when a square wave pulse was input. Results showed that the square wave pulse became blunt when the driving speed was increased. After estimation, we designed the layout of the new ultrahigh-speed CCD V6 and fabricated the device. Results of an image capturing experiment indicated a saturation signal level of 100% that was maintained up to 300,000 frames per second. A saturation signal level of 50% was observed in 1,000,000 frames per second and of 13% in 2,000,000 frames per second. We showed that the maximum frame rate is dependent on a drop of the saturation signal level resulting from the driving voltage wave pattern becoming blunt.

Color video camera capable of 1,000,000 fps with triple ultrahigh-speed image sensors

An image sensor for an ultra-high-speed video camera was developed. The maximum frame rate, the pixel count and the number of consecutive frames are 1,000,000 fps, 720 x 410 (= 295,200) pixels, and 144 frames. A micro lens array will be attached on the chip, which increases the fill factor to about 50%. In addition to the ultra-high-speed image capturing operation to store image signals in the in-situ storage area adjacent to each pixel, standard parallel readout operation at 1,000 fps for full frame readout is also introduced with sixteen readout taps, for which the image signals are transferred to and stored in a storage device with a large capacity equipped outside the sensor. The aspect ratio of the frame is about 16 : 9, which is equal to that of the HDTV format. Therefore, a video camera with four sensors of the ISIS-V4, which are arranged to form the Bayer’s color filter array, realizes an ultra-high-speed video camera of a semi-HDTV format.

Development of ultrahigh-speed CCD with maximum frame rate of 2 million frames per second

An ultrahigh-speed charge-coupled device (CCD) with an increased dynamic range at a frame rate above 200 kiloframes per second (kfps) was developed. The dynamic range of a CCD operating at extremely high speeds is reduced as a result of rounding of a sharp voltage waveform inside the device. The amount of rounding was estimated by using an equivalent circuit model of one kind of electrodes in a four-phase CCD memory. The simulation showed that the calculated voltage at a quarter period and the measured saturation signal level have similar dependence on the frame rate. To suppress the drop in voltage at a quarter period, the active pixels and the driving circuit were divided, and the resistance of the pixel wiring was reduced. A new ultrahigh-speed CCD, whose active pixels are divided into eight separately driven blocks and that employs dual wirings to each electrode of the four-phase CCD memory, was designed and fabricated. A driving evaluation experiment showed that the ultrahigh-speed CCD had a dynamic range of 48.6 dB at 1 000 000 fps. This range is equivalent to 8-bit digital and is 2.5 times higher than that of a previous ultrahigh-speed CCD.