China extended its lead in numbers of supercomputers, but the U.S. made gains in overall performance in the rankings released Monday.

The latest Top 500 list comes as China and the U.S. are running neck-in-neck in a race to deliver before 2022 exascale-class machine, 10 times more powerful than today’s largest systems.

The latest TOP500 list saw five US Department of Energy (DOE) supercomputers in the top 10 positions, with the first two captured by Summit at Oak Ridge National Laboratory (ORNL) and Sierra at Lawrence Livermore National Laboratory (LLNL).

Summit widened its lead as the number one system, improving its High Performance Linpack (HPL) score from 122.3 to 143.5 petaflops since its debut on the previous list in June 2018. Sierra also added to its HPL result from six months ago, going from 71.6 to 94.6 petaflops, enough to bump it from the number three position to number two. Both are IBM-built supercomputers, powered by Power9 CPUs and NVIDIA V100 GPUs.

Sierra’s ascendance pushed China’s Sunway TaihuLight supercomputer, installed at the National Supercomputing Center in Wuxi, into third place. Prior to last June, it had held the top position on the TOP500 list for two years with its HPL performance of 93.0 petaflops. TaihuLight was developed by China’s National Research Center of Parallel Computer Engineering & Technology (NRCPC).

Tianhe-2A (Milky Way-2A), deployed at the National Supercomputer Center in Guangzho, China, is now in the number four position with a Linpack score of 61.4 petaflops. It was upgraded earlier this year by China’s National University of Defense Technology (NUDT), replacing the older Intel Xeon Phi accelerators with the proprietary Matrix-2000 chips.

At number five is Piz Daint, a Cray XC50 system installed at the Swiss National Supercomputing Centre (CSCS) in Lugano, Switzerland. At 21.2 petaflops, it maintains its standing as the most powerful system in Europe. It is powered by a combination of Intel Xeon processors and NVIDIA Tesla P100 GPUs.

Trinity a Cray XC40 system operated by Los Alamos National Laboratory and Sandia National Laboratories improved its performance to 20.2 petaflops, enough to move it up one position to the number six spot. It uses Intel Xeon Phi processors, the only top ten system to do so.

The AI Bridging Cloud Infrastructure (ABCI) installed in Japan at the National Institute of Advanced Industrial Science and Technology (AIST) is listed at number seven with a Linpack mark of 19.9 petaflops. The Fujitsu-built system is powered by Intel Xeon Gold processors, along with NVIDIA Tesla V100 GPUs.

Germany provided a new top ten entry with SuperMUC-NG, a Lenovo-built supercomputer installed at the Leibniz Supercomputing Centre (Leibniz-Rechenzentrum) in Garching, near Munich. With more than 311,040 Intel Xeon cores and an HPL performance of 19.5 petaflops, it captured the number eight position.

Titan, a Cray XK7 installed at the DOE’s Oak Ridge National Laboratory, and previously the most powerful supercomputer in the US, is now the number nine system. It achieved 17.6 petaflops using NVIDIA K20x GPU accelerators.

Sequoia, an IBM BlueGene/Q supercomputer installed at DOE’s Lawrence Livermore National Laboratory, is the 10th-ranked TOP500 system. It was first delivered in 2011, achieving 17.2 petaflops on HPL.

Exascale systems

Both China and the U.S. have ramped up programs to deliver as many as three exascale systems each before 2022. The U.S. Department of Energy announced in April it is spending $1.8 billion on hardware alone for its program that includes separate spending on software.

Three U.S. government labs will house its exasacle machines — Argonne, Livermore and Oak Ridge. China is expected to give a green light to three projects competing to deliver exascale systems as early as 2020.

One system is expected to use a China-designed CPU called the FT-2000 based on 64-bit ARM cores with vector extensions. China is designing its own chip, at least in part because the U.S. is putting restrictions on Intel supplying government-led projects its high-end Xeon chips that are widely used in supercomputers.

In recent years, China has been increasing the amount of silicon it designs for its high-end systems, including its own interconnect chips and accelerators.

For example, the Tianhe-2A, No. 4 on the latest list at 61.4 petaflops, uses a homegrown Matrix-2000+ accelerator, replacing an Intel Xeon Phi it used previously.

The Matrix-2000+ packs 128 cores in a 16nm device that delivers 2.048 teraflops/second at 2 GHz. A Matrix-3000 chip is expected to power one of China’s three exascale systems.

Other TOP500 highlights

The share of TOP500 installations in China continues to rise, with the country now claiming 227 systems (45 percent of the total). The number of supercomputers that call the US home continues to decline, reaching an all-time low of 109 (22 percent of the total). However, systems in the US are, on average, more powerful, resulting in an aggregate system performance of 38 percent, compared to 31 percent for China.

Japan is third in system share, with 31 systems, followed by the UK, with 21, France with 18, Germany with 17, and Ireland with 12. All other countries have fewer than ten systems on the list.

There are now 429 supercomputers on the TOP500 list that deliver over one petaflop on the Linpack benchmark, up from 272 systems six months ago. The entry point to the list is now 874.8 teraflops, with the entry point for the top 100 systems at 1.97 petaflops.

The top ten systems manufacturers (by number of systems) are Lenovo (140), Inspur (84), Sugon (57), Cray (49), HPE (46), Bull (22), Fujitsu (15), Huawei (14), Dell EMC (13), and IBM (12).

Intel continues to provide the processors for the largest number of TOP500 systems, with a dominating 95.2 percent share. There are just three systems on the latest list powered by AMD CPUs.

For the first time, an ARM-powered supercomputer has made it into the TOP500 rankings. The new system, known as Astra, is an HPE-built supercomputer deployed at Sandia National Laboratories. It’s powered by 125,328 Cavium ThunderX2 cores and has achieved an HPL Linpack score of 1.5 petaflops. It enters the list at number 205.

A total of 137 systems on the list are using accelerators/coprocessors, up from 110 six months ago. NVIDIA remains, by far, the dominant supplier, with 64 of these systems using P100 GPUs, 46 using V100 GPUs, and 12 systems using Kepler GPUs (K80, K40, and K20x).

The most popular TOP500 interconnect technologies are Gigabit Ethernet (252 systems), InfiniBand (135 systems), custom interconnects (64 systems), and Intel Omni-Path (43 systems).

The aggregate performance of the entire list has increased to 1.42 exaflops, up from 1.21 exaflops six months ago.

Green500 results

The most energy-efficient system on the Green500 list is once again the Shoubu system B, a ZettaScaler-2.2 supercomputer installed at the Advanced Center for Computing and Communication, RIKEN, Japan. It was remeasured and achieved 17.6 gigaflops/watt during its 1.06 petaflops Linpack performance run. It occupies position 376 on the TOP500 list.

In second position on the Green500 is the DGX SaturnV Volta system, an NVIDIA system installed at NVIDIA, USA. It achieved 15.1 gigaflops/watt and is in position 375 on the TOP500 list.

At number three is the Summit system at the Oak Ridge National Laboratory. It achieved 14.7 gigaflops/watt and is listed at number one in the TOP500.

At number four is the AI Bridging Cloud Infrastructure (ABCI) system installed at the National Institute of Advanced Industrial Science and Technology (AIST). It achieved 14.4 gigaflops/watt and is listed at number seven in the TOP500.

Another supercomputer from Japan, TSUBAME 3.0, captured the number five spot on the Green500. The Tokyo Tech-based system delivered a power efficiency of 13.7 gigaflops/watt.

A surprise entry at number six, is Sugon’s HKVDPSystem cluster, installed at an anonymous internet company in China. Its 13.0 gigaflops/watt was achieved with Intel Xeon processors and 10G Ethernet technology.

Sierra, the AIST AI Cloud, the MareNostrum P9 CTE, and Sugon’s Advanced Computing System (PreE) captured the number seven through ten positions on the Green500, respectively. The PreE system is powered by the Dhyana CPU, a Hygon-licensed implementation of AMD’s EPYC processor.

HPCG Results

Some of the highest-ranked supercomputers on TOP500 list were also top performers on the High-Performance Conjugate Gradient (HPCG) benchmark. In fact, the top two Linpack performers, Summit and Sierra, captured the first two positions on the HPCG benchmark. Summit achieved 2.93 HPCG-petaflops and Sierra, 1.80 HPCG-petaflops.

They are followed by the previous HPCG leader, Fujitsu’s K computer, which delivered 0.60 HPCG-petaflops; it is ranked 18th in the TOP500. Trinity grabbed the number four HPCG position, with 0.55 HPCG-petaflops, followed by the ABCI system at 0.51 HPCG-petaflops. Those two systems are ranked 6th and 7th, respectively in the TOP500. Piz Daint, Sunway Taihulight, Nurion, Oakforest-PACS, and Cori round out the top ten.