Cliff Young

In-Datacenter Performance Analysis of a Tensor Processing Unit

Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC-called a Tensor Processing Unit (TPU)-deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPUs deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X–30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X–80X higher. Moreover, using the GPUs GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.

D-T gamma-to-neutron branching ratio determined from inertial confinement fusion plasmasa)

A new deuterium-tritium (D-T) fusion gamma-to-neutron branching ratio [3H(d,γ)5He/3H(d,n)4He] value of (4.2u2009±u20092.0)u2009×u200910−5 was recently reported by this group [Y. Kim et al. Phys. Rev. C (submitted)]. This measurement, conducted at the OMEGA laser facility located at the University of Rochester, was made for the first time using inertial confinement fusion (ICF) plasmas. Neutron-induced backgrounds are significantly reduced in these experiments as compared to traditional beam-target accelerator-based experiments due to the short pulse nature of ICF implosions and the use of gas Cherenkov γ-ray detectors with fast temporal responses and inherent energy thresholds. It is expected that this ICF-based measurement will help resolve the large and long-standing inconsistencies in previously reported accelerator-based values, which vary by a factor of approximately 30. The reported value at ICF conditions was determined by averaging the results of two methods: (1) a direct measurement of ICF D-T γ-ray and neutron ...

Measurement of areal density in the ablators of inertial-confinement-fusion capsules via detection of ablator (n, n′γ) gamma-ray emission

We report the first gamma-ray-based measurements of the areal density of ablators in inertial-confinement-fusion capsule implosions. The measurements, made at the OMEGA laser [T. R. Boehly et al., Opt. Commun. 133, 495 (1997)], used observations of gamma rays arising from inelastic scattering of 14.1-MeV deuterium-tritium (DT) neutrons on 12C nuclei in the compressed plastic ablators. The emission of 12C(n,n′γ) gamma rays from the capsules is detected using the Gamma Reaction History instrument [H. W. Herrmann et al., J. Phys.: Conf. Ser. 244, 032047 (2010)] operating at OMEGA. From the ratio of a capsules 12C(n,n′γ) emission to the emission from the same processes in an in situ reference graphite “puck” of known mass and geometry [N. M. Hoffman et al., in IFSA 2011 proceedings (submitted)], we determine the time-averaged areal density of 12C in the capsules compressed ablator. Measured values of total ablator areal density for thirteen imploded capsules, in the range 23u2009±u200910 to 58u2009±u200914u2009mg/cm2, are comp...

A New Golden Age in Computer Architecture: Empowering the Machine-Learning Revolution

The end of Moores law and Dennard scaling has led to the end of rapid improvement in general-purpose program performance. Machine learning (ML), and in particular deep learning, is an attractive alternative for architects to explore. It has recently revolutionized vision, speech, language understanding, and many other fields, and it promises to help with the grand challenges facing our society. The computation at its core is low-precision linear algebra. Thus, ML is both broad enough to apply to many domains and narrow enough to benefit from domain-specific architectures, such as Googles Tensor Processing Unit (TPU). Moreover, the growth in demand for ML computing exceeds Moores law at its peak, just as it is fading. Hence, ML experts and computer architects must work together to design the computing systems required to deliver on the potential of ML. This article offers motivation, suggestions, and warnings to computer architects on how to best contribute to the ML revolution.

A domain-specific architecture for deep neural networks

Tensor processing units improve performance per watt of neural networks in Google datacenters by roughly 50x.