2021 INCITE Call for Proposals

The Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program promotes transformational advances in science and technology for compute and/or data-intensive large-scale research projects such as scientific modeling, simulation, and data analytics and artificial intelligence (AI) campaigns.  INCITE projects are awarded large allocations of computer time and supporting resources at the Argonne and Oak Ridge Leadership Computing Facility (LCF) centers, operated by the US Department of Energy (DOE) Office of Science.  INCITE seeks research challenges requiring capability computing from diverse areas: production simulations, compute intensive machine or deep learning applications, or large-scale data analysis that use a large fraction of the LCF systems and cannot be performed on less capable resources.  INCITE supports high impact simulation, data and AI approaches which require the unique architectural infrastructure, high performance storage and networking capabilities available at the LCF centers.

INCITE is currently soliciting research proposals for awards on the LCF high-performance computing (HPC) machines for calendar year (CY) 2021.  LCF HPC machines available in CY 2021 include Summit, the 200-petaflop IBM AC922 machine; and Theta, the 12-petaflop Cray XC40 machine; Aurora and Frontier will be available in CY 2023.  Up to sixty percent of the allocable computing time will be made available on each of these machines to the INCITE program.  Proposals may be for up to three years.

Overview

INCITE proposals should be high-impact research campaigns and may arise from a broad array of science, engineering, and computer science domains. INCITE proposals should demonstrate the need for LCF capability computing resources stemming from compute-intensive, data-intensive needs, and/or large-scale data storage and processing requirements. Individuals and/or teams of researchers from academia, national laboratories, industry and federal agencies are eligible to apply, and awards of one, two, or three years may be granted. Multiyear awards must submit annual renewal requests, which will undergo peer review.

If you answer “yes” to any of the following questions, INCITE may be right for you:

  • Is your science campaign outpacing the computing resources available to you?
  • Does the volume (and/or velocity or variety) of your data processing require LCF computing and storage resources?
  • Do you have a long-term vision for your research campaign and see it operating over a period of years rather than months?
  • Can your workflow or AI application needs allow you to effectively use an INCITE-sized award?

Campaigns chosen by the INCITE program typically cannot be performed anywhere else and require extremely large HPC systems, large awards of computing time, very large memory, mixed/reduced precision accelerator hardware, or other unique LCF architectural infrastructure to succeed.

Eligibility

The INCITE program is open to US- and non-US-based researchers and research organizations needing large allocations of computer time, support resources, and data storage to pursue transformational advances in science and engineering. INCITE considers requests regardless of funding source.

The intent of INCITE is to support large-scale, compute and/or data intensive projects that would not be possible or productive without capability computing or other aspects of the LCFs architectures and/or infrastructure associated with such resources. In addition to scientific modeling and simulation campaigns, INCITE welcomes large-scale data analytics and AI applications. High impact data and learning proposals (e.g., data-intensive computing, HPC enabled data mining, machine learning and deep learning) with a focus on incorporating data-driven discovery for breakthroughs in science and engineering are encouraged.

Recognizing the diversity of data analytic and AI applications, INCITE encourages crosscutting proposals that bring together scientific modeling, simulation, data analysis, and/or learning in an integrated research campaign. Refer to the Frequently Asked Question “What constitutes a “Data and/or AI Application research project?”  Requests smaller than 20 percent of the average award on any particular resource and that do not make a strong case for the need for leadership computing resources may be more appropriate for another LCF User Program such as the Director’s Discretionary (DD) or ASCR Leadership Computing Challenge (ALCC) programs.

The INCITE program encourages community proposals that reflect scientific consensus on, and participation in, pursuing a particular investigative strategy for addressing a grand challenge–scale science problem. This consensus should be in large part reflected in the proposal’s management plan, which describes how the problem will be tackled and the interrelationship of the activities of each of the proposal team investigators.

The INCITE program does not encourage umbrella proposals, which is defined to be a collection of individual proposals aimed at various aspects of a particular science problem but with no clearly articulated interrelationship of the activities of each of the proposal team investigators and/or that do not exploit the unique capabilities of the leadership facilities.

Resources

Awards of computing time are made on the leadership-class systems at Oak Ridge National Laboratory (ORNL) and Argonne National Laboratory (ANL).  Up to sixty percent of time on the leadership-class systems (Summit at ORNL and Theta at ANL in CY2021; additionally Aurora at ANL and Frontier at ORNL will be available in CY2023) is allocated through INCITE.

Researchers may request access to more than one LCF center and to more than one resource at an LCF.  The full description of the resources available under this call includes important details that all applicants should review prior to proposal submittal.

One of the unique features of the INCITE program is that at the LCF center where an allocation is awarded, the project will be assigned a staff member to provide scientific and technical support. This individual will have a background in the project’s domain or in a related scientific field and will bring to the project extensive experience using the LCF computational resources.

Trends

Last year’s call for proposals resulted in 47 projects (30 new and 17 renewals) awarded an INCITE allocation in CY 2020. The acceptance rate for new proposals was 31 percent. Representative awards include the following:

  • Astrophysics: “Extreme-scale Simulation of Supernovae and Magnetars from Realistic Progenitors” (1,000,000 Theta node-hours)
  • Computer Science: “Exascale Artificial Intelligence to Accelerate Scientific Discovery” (1,000,000 Theta node-hours)
  • Engineering: “High fidelity study of impact of complex flows on high pressure turbines” (590,000 Summit node-hours)
  • Fluid Dynamics: “High-speed turbulence with shocks over non-adiabatic and flexible walls” (2,000,000 Theta node-hours)
  • Materials Science: “Large-scale first principles calculations of dislocation core energetics in Al and Mg” (360,000 Summit node-hours)
  • Plasma Physics: “High-fidelity gyrokinetic simulation of tokamak and ITER edge physics” (970,000 Summit node-hours and 1,500,000 Theta node-hours)

Fact sheets summarizing INCITE projects awarded since 2010 can be found in the Awardees page.

Selection

Awards are made based on the quality and impact of the research and the suitability of the proposed campaign for the requested resource(s). INCITE employs a two-phase review process. Proposals are peer-reviewed by an international panel of experts for scientific and technical merit and potential impact. This assessment is the primary basis for identifying the top-ranked proposals for potential awards. The computational readiness of the proposal is also assessed to determine how effectively the project will use the requested system. Multiyear awards are subject to annual review.

The INCITE Awards Committee, composed of the INCITE manager and LCF management teams, will make selections based on the rankings by the peer review panel. Potential for impact is the predominant determinant for awards. The readiness ratings are used to determine the capability of the project to effectively use the selected system and are based on proficiency shown through benchmarking data and/or proposed development plans. Proposals are selected that answer a high-impact, key scientific/technical question and/or point to a new area of research. Successful proposal teams demonstrate a very clear understanding of extreme-scale computing and can optimally use these resources to accomplish the stated scientific/technical goals.

Preparing a proposal

Detailed instructions on preparing a proposal can be found on the INCITE website. Further, INCITE and the LCF centers offer webinars to advise potential authors on best practices for INCITE proposal writing. Dates will be announced via the website and by email for registered users.

Authors are encouraged to attend the webinars and to apply for LCF center DD time. The DD program allows proposers to conduct porting, tuning, and scaling and to gather benchmarking data for inclusion in INCITE proposals. DD requests may be submitted any time during the year. DD applications are available at:

Also, the list of questions used by reviewers to assess INCITE proposals is available and can be used to guide the development of an INCITE application. For more details about these aids to proposal writing, see the INCITE Instructions.

User agreements

The use of HPC resources under the INCITE program is conditional upon acceptance of the relevant user agreement provided by the center at which an allocation is awarded. Upon receipt of an INCITE award, each participating institution on the awarded proposal must accept or already have signed the user agreement; otherwise, access by the institution’s researcher(s) to the LCF system will be delayed until the agreement is signed. The ANL and ORNL user agreements are accessible online. The proposer may contact the INCITE manager at INCITE@doeleadershipcomputing.org for further details. The proposer’s and any participating institution’s authorized agents (typically contract specialists or legal representatives who have the authority to sign the user agreement on behalf of the institution) should be prepared to review and sign the agreement once an award of time is granted. Inability of the proposer’s institution to sign the agreement in a timely fashion may put the award at risk.

Proprietary use

Proprietary use (e.g., using the resources to generate data or results that a user or project wishes to designate as proprietary) of the HPC resources is allowed and will include full cost recovery through a proprietary user agreement. The use of proprietary or export-controlled software applications or input data does not by itself constitute proprietary use of the facility. A single and standardized peer-review selection process and project reporting requirement, as described within the INCITE Overview and Policies, will be implemented for both proprietary and nonproprietary user proposals. Given the complexities associated with proprietary research, individuals considering submittal of a proposal for proprietary research must contact the INCITE manager at INCITE@doeleadershipcomputing.org before the call for proposals closes to discuss the policy on proprietary work.

Note: Proprietary information may NOT be included in the proposal submitted via the INCITE proposal website.

Acknowledgements and Reporting Requirements

All recipients of resources allocated through the INCITE program are expected to acknowledge the center and the program in publications resulting from their award. Refer to the Frequently Asked Question “What are my responsibilities if I receive an award?” for the standard acknowledgements and other reporting requirements. The principal investigator of each awarded project will be expected to provide periodic reports (e.g., quarterly and end-of-year) and respond to an annual survey.

Security

The LCF centers have implemented cybersecurity programs at a moderate baseline with compensatory controls in accordance with National Institute of Standards and Technology Special Publication 800-53. Project-specific security requirements can also be discussed and potentially addressed; the proposer should contact the center directly for details. Risk is currently mitigated using available enterprise infrastructure and network-based tools to provide perimeter protection and vulnerability resolution. System- and application-level security controls are implemented in accordance with industry and federal best practices. All nontemporary user data reside on centralized file and archival storage systems that are regularly backed up. These data are protected and segmented from other users through standard access controls. 

Data sensitivity and protection

The LCF centers cannot guarantee zero risks to information stored on their systems. Requests for processing sensitive (e.g., proprietary or export-controlled) data must therefore be clearly identified in the proposal. Information sensitivity concerns apply to the intent of the research, the storing and compiling of source code, the loading and execution of application software, the input data for the application software, the output data generated by the application software, and data resulting from the analysis of output data. If, for example, the application software has an Export Control Classification Number (ECCN), prospective users should state this fact in the proposal and provide the ECCN. Prospective users should work with the center to identify appropriate levels of data protection. Greater levels of protection not typically offered may be provided by the principal investigator’s funding source.

Deadlines

Electronic applications for new proposals will be accepted starting mid-April 2020, through 8:00 pm EDT on Friday, June 19, 2020. All proposals must follow these instructions. INCITE reserves the right to decline consideration of proposals not compliant with these instructions. Hardcopies will not be accepted for review.

Awards are expected to be announced in November 2020. Access to a computing system, granted over the 2021 calendar year, will be established upon completion of the appropriate agreements and in accordance with center-specific procedures. Contact us for questions about the application procedure.