HPC Resources

Summit

The 200-petaflop system called Summit, is available to users through INCITE 2023 program. The Summit system has very powerful, large memory nodes that are most effective for those applications that effectively use the GPUs. The machine also has node-local, non-volatile memory that can be used to increase I/O bandwidth or provide expanded local storage for applications. The peak node performance is 42 teraflops per node with 512 gigabytes of DDR4 memory, 96 gigabytes of HBM2 memory, and 1,600 gigabytes of non-volatile memory.

Summit uses IBM’s Spectrum Scale™ file system.  It will have 250 PB of capacity and bandwidth of up to 2.5 TB/second.  In addition, each node has a 1.6 TB of non-volatile memory that provides high-speed local storage as well as serving as a burst buffer in front of the file system.  All OLCF users will have access to the HPSS data archive, the Rhea and Eos pre- and post-processing clusters, and the EVEREST high-resolution visualization facility. All of these resources are available through high-performance networks including ESnet’s recently upgraded 100 gigabit per second links.

For more information about any of the OLCF resources, please visit https://www.olcf.ornl.gov/olcf-resources/

Frontier (Available in 2023)

The Frontier system will be based on Cray’s new Shasta architecture and Slingshot interconnect with high-performance AMD EPYC CPU and Radeon Instinct GPU technology. The new accelerator-centric compute blades will support a 4:1 GPU-to-CPU ratio connected with high-speed links and coherent memory between them within the node. The peak performance is expected to be over 1.5 exaslops (EF).

For more information about any of the OLCF resources, please visit https://www.olcf.ornl.gov/olcf-resources/

Theta

Theta, the Argonne Leadership Computing Facility’s Cray XC40 supercomputer, is equipped with 281,088 cores, 70 terabytes of high-bandwidth MCDRAM, 843 terabytes of DDR4 memory, 562 terabytes on SSDs, and has a peak performance of 11.69 petaflops.

Theta’s 4,392 compute nodes have an Intel “Knights Landing” Xeon Phi processor containing 64 cores, each with four hardware threads, and 16 gigabytes of high-bandwidth MCDRAM, 192 gigabytes of DDR4 memory, and a 128 gigabyte SSD. Theta uses Cray’s high performance Aries network in a Dragonfly topography. The Xeon Phi supports improved vectorization using AVX-512 SIMD instructions.

Theta users have access to Grand and Eagle, ALCF’s file storage systems with 650 GB/s bandwidth. These 100-Petabytes Luster file systems, supported by two ClusterStor E1000 storage systems, represent a substantial upgrade over their predecessors. The Grand files system will be primarily used for output generated from computational runs on ALCF computing systems and the Eagle file system will be primarily used for data sharing with the research community across high-performance computing facilities.

For more information about any of the ALCF resources, please visit http://www.alcf.anl.gov/computing-resources.

Polaris

Developed in collaboration with Hewlett Packard Enterprise (HPE), the 44-petaflop system called Polaris, is available to users through the INCITE 2023 program. Polaris is a hybrid CPU/GPU leading-edge testbed system that will give scientists and application developers a platform to test and optimize codes for Aurora, Argonne’s upcoming Intel-HPE exascale supercomputer.

The Polaris software environment is equipped with the HPE Cray programming environment, PE Performance Cluster Manager (HPCM) system software, and the ability to test programming models, such as OpenMP and SYCL, that will be available on Aurora and the next generation of DOE’s high-performance computing systems. Polaris users will also benefit from NVIDIA’s HPC software development kit, a suite of compilers, libraries, and tools for GPU code development.

For more information about any of the ALCF resources, please visit http://www.alcf.anl.gov/computing-resources.

Aurora

Aurora, Argonne National Laboratory’s first exascale computer is due to arrive in 2022. Scientist will use the new machine to pursue science and engineering breakthroughs beyond what is possible with today’s supercomputers. Aurora, designed in collaboration with Intel and HPE, will support machine learning and data science workloads alongside traditional modeling and simulation workloads.

Aurora will feature several technological innovations and will be built on a future generation of Intel Xeon® Scalable processor accelerated by Intel’s Xe compute architecture. HPE Slingshot™ fabric and Shasta™ platform will form the backbone of the system. Aurora will also include a revolutionary I/O system – the Distributed Asynchronous Object Store (DAOS) – to support new types of workloads.

Public information for Aurora is listed below.

The expected software stack includes the Cray Shasta software stacks, Intel software, and data and learning frameworks. Supported programming models include MPI, Intel OneAPI, OpenMP, SYCL/DPC++., Kokkos, Raja, and others.

For the 2023 INCITE proposal submission, PIs are encouraged to express interest in Aurora for the 2023 cycle. Aurora allocations will be requested in equivalent ‘Theta node-hours’. For planning purposes for this cycle, we conservatively expect nearly 1.78 billion Theta-equivalent node-hours to be allocated on Aurora per year and that the average INCITE project will be awarded approximately 40-50 million Theta-equivalent node-hours in 2024. Awarded three-year projects will be required to revisit their Theta-equivalent node-hour request for Aurora and the computational readiness for Aurora will be re-evaluated in future renewals.

The most recent public information on Aurora can be found at

For more information about any of the ALCF resources, please visit https://www.alcf.anl.gov/alcf-resources