2023 INCITE Call for Proposals
The Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program promotes transformational advances in science and technology for compute and/or data-intensive large-scale research projects such as scientific modeling, simulation, and data analytics and Artificial Intelligence (AI) campaigns. INCITE projects are awarded large allocations of computer time and supporting resources at the Argonne and Oak Ridge Leadership Computing Facility (LCF) centers, operated by the US Department of Energy (DOE) Office of Science. INCITE seeks research challenges requiring capability computing from diverse areas: production simulations, compute intensive machine or deep learning applications, or large scale data analysis that use a large fraction of the LCF systems and cannot be performed on less capable resources. INCITE supports high impact simulation, data and AI approaches which require the unique architectural infrastructure, high performance storage and networking capabilities available at the LCF centers.
INCITE is currently soliciting proposals of research for awards on LCF HPC machines for calendar year (CY) 2023. LCF HPC machines include the exascale systems Frontier and Aurora, the 44-petaflop HPE Apollo Gen-10+ system Polaris, Summit, the 200-petaflop IBM AC922 machine, and Theta, the 12-petaflop Cray XC40 machine. Up to sixty percent of the allocatable computing time will be made available on each of these machines for CY 2023. Proposals may make requests for the coming exascale systems Aurora (ALCF) and Frontier (OLCF). Proposals may request periods of performance from one to three years.
INCITE proposals should be high-impact research campaigns and may arise from a broad array of science, engineering, and computer science domains. INCITE proposals should demonstrate the need for LCF capability computing resources stemming from compute-intensive, data-intensive needs, and/or large scale data storage and processing requirements. Individuals and/or teams of researchers from academia, national laboratories, industry and federal agencies are eligible to apply, and awards of one, two, or three years may be granted.
If you answer “yes” to any of the following questions, INCITE may be right for you:
- Is your science campaign outpacing the computing resources available to you?
- Does the volume (and/or velocity or variety) of your data processing require LCF computing and storage resources?
- Do you have a long-term vision for your research campaign and see it operating over a period of years rather than months?
- Can your workflow or AI application needs allow you to effectively use an INCITE-sized award?
Campaigns chosen by the INCITE program typically cannot be performed anywhere else and require extremely large high-performance computing (HPC) systems, large awards of time, very large memory, mixed/reduced precision accelerator hardware, or other unique LCF architectural infrastructure to succeed.
The INCITE program is open to US- and non-US-based researchers and research organizations needing large allocations of computer time, supporting resources, and data storage to pursue transformational advances in science and engineering. INCITE considers requests regardless of funding source.
The intent of INCITE is to support large-scale, compute and/or data intensive projects that would not be possible or productive without capability computing or other aspects of the LCFs architectures and/or infrastructure associated with such resources. In addition to scientific modeling and simulation campaigns, INCITE welcomes large-scale data analytics and AI applications. High impact data and learning proposals (e.g., data-intensive computing, HPC enabled data mining, machine learning and deep learning) with a focus on incorporating data-driven discovery for breakthroughs in science and engineering are encouraged.
Recognizing the diversity of data analytic and AI applications, INCITE encourages crosscutting proposals that bring together scientific modeling, simulation, data analysis, and/or learning in an integrated research campaign. Refer to the Frequently Asked Question “What constitutes a Data and/or AI Application research project?” Requests smaller than 20% of the average award on any particular resource and that do not make a strong case for the need for leadership computing resources may be more appropriate for another LCF User Program such as the Director’s Discretionary or ASCR Leadership Computing Challenge (ALCC) programs.
The program encourages community proposals that reflect scientific consensus on, and participation in pursuing, a particular investigative strategy for addressing a grand-challenge-scale science problem. This consensus should be in large part reflected in the proposal’s management plan, which describes how the problem will be tackled and the interrelationship of the activities of each of the proposal team investigators.
The program does not encourage umbrella proposals, which is defined to be a collection of individual proposals aimed at various aspects of a particular science problem but with no clearly articulated interrelationship of the activities of each of the proposal team investigators and/or that do not exploit the unique capabilities of the leadership facilities.
Early Career Track
Starting in the INCITE 2022 year, INCITE is committing 10% of allocatable time to an Early Career Track in INCITE. The goal of the early career track is to encourage the next generation of high-performance computing researchers. Researchers within 10 years from earning their PhD (PhD on or after December 31st 2012) and who have not been a previous INCITE PI may choose to apply. Projects will go through the regular INCITE Computational Readiness and Peer Review process, but the INCITE Management Committee will consider meritorious projects in the Early Career Track separately.
Awards of time are made on the leadership-class systems at Oak Ridge National Laboratory (ORNL) and Argonne National Laboratory (ANL). Up to sixty percent of time on the leadership-class systems (Summit at ORNL and Theta and Polaris at ANL in CY2022; Aurora and Frontier in CY2023) is allocated through INCITE.
Researchers may request access to more than one LCF center and to more than one resource at an LCF. A full description of the resources available under this call can be found at the system descriptions and includes important details that all applicants should review prior to proposal submittal.
One of the unique features of the INCITE program is that at the LCF center where an allocation is awarded, the project will be assigned a staff member to provide scientific and technical support. This individual will have a background in the project’s domain or in a related scientific field. The bring to the project extensive experience using the LCF computational resources.
Last year’s call for proposals resulted in 52 projects (37 new and 15 renewals) awarded an INCITE allocation in CY 2022. The acceptance rate for new proposals was 34 percent. Representative awards include the following:
- Astrophysics (2,000,000 Theta node-hours) “Electron Kinetic Plasma Physics of Black Hole Accretion Flows”
- Earth Sciences (620,000 Summit node-hours) “Energy Exascale Earth System Model”
- Engineering (800,000 Theta node-hours) “Automatic Building Energy Modeling (AutoBEM)”
- Biology (690,000 Summit node-hours) “Deep learning for accurate and cost-effective imputation of genotypes at whole genome level”
- Materials Science (1,500,000 Theta node-hours, 200,00 Polaris node hours) “UltrafastMat: Ultrafast Control of Functional Materials”
- Plasma Physics (790,000 Summit node-hours and 1,000,000 Theta node-hours) “High-fidelity gyrokinetic simulation of tokamak and ITER edge physics”
Awards are made based on the quality and impact of the research and the suitability of the proposed campaign for the requested resource(s). INCITE employs a two-phase review process. Proposals are peer-reviewed by an international panel of experts for scientific and technical merit and potential impact. This assessment is the primary basis for identifying the top-ranked proposals for potential awards. The computational readiness of the proposal is also assessed to determine how effectively the project will use the requested system. Multiyear awards are subject to annual review.
The INCITE Awards Committee, composed of the INCITE manager and LCF management teams, will make selections based on the rankings by the peer-review panel. Potential impact is the predominant determinant for awards. The readiness ratings are used to determine the capability of the project to effectively use the selected system and are based on proficiency shown through bench-marking data and/or proposed development plans. Proposals are selected that answer a high-impact, key scientific/technical question and/or point to a new area of research. Successful proposal teams demonstrate a very clear understanding of extreme-scale computing and can optimally use these resources to accomplish the stated scientific/technical goals.
Preparing a proposal
Detailed instructions on preparing a proposal can be found on the INCITE website. Further, INCITE and the LCF centers offer webinars to advise potential authors on best practices for INCITE proposal writing. Dates will be announced via the website and by email for registered users.
Authors are encouraged to attend the webinars and to apply for LCF center Director’s Discretionary (DD) time. The DD allows proposers to conduct porting, tuning and scaling and to gather benchmarking data for inclusion in INCITE proposals. DD requests may be submitted any time during the CY. Director’s Discretion applications are available at:
- ALCF: https://www.alcf.anl.gov/science/directors-discretionary-allocation-program
- OLCF: https://my.olcf.ornl.gov/project-application-new
Also, the list of questions used by reviewers to assess INCITE proposals is available and can be used to guide the development of an INCITE application. For more details about these aids to proposal writing, see the INCITE Instructions.
The use of HPC resources under the INCITE program is conditional upon acceptance of the relevant user agreement provided by the center at which an allocation is awarded. Upon receipt of an INCITE award, each participating institution on the awarded proposal must accept or already have signed the user agreement; otherwise, access by the institution’s researcher(s) to the LCF system will be delayed until the agreement is signed. The ANL and ORNL user agreements are accessible online. The proposer may contact the INCITE manager at INCITE@doeleadershipcomputing.org for further details. The proposer’s and any participating institution’s authorized agents (typically contract specialists or legal representatives who have the authority to sign the user agreement on behalf of the institution) should be prepared to review and sign the agreement once an award of time is granted. Inability of the proposer’s institution to sign the agreement in a timely fashion may put the award at risk.
Proprietary use (e.g., using the resources to generate data or results that they wish to designate as proprietary) of the HPC resources is allowed and will include full cost recovery through a proprietary user agreement. The use of proprietary or export-controlled software applications or input data does not by itself constitute proprietary use of the facility. A single and standardized peer-review selection process and project reporting requirement, as described within the INCITE Overview and Policies, will be implemented for both proprietary and nonproprietary user proposals. Given the complexities associated with proprietary research, individuals considering submittal of a proposal for proprietary research must contact the INCITE manager at INCITE@doeleadershipcomputing.org before the call for proposals closes to discuss the policy on proprietary work. Note: Proprietary information may NOT be included in the proposal submitted via the INCITE proposal website.
Acknowledgements and Reporting Requirements
All recipients of resources allocated through the INCITE program are expected to acknowledge the center and the program in publications resulting from their award. Refer to the Frequently Asked Question “What are my responsibilities if I receive an award?” for the standard acknowledgements and other reporting requirements. The principal investigator of each awarded project will be expected to provide periodic reports (e.g., quarterly and end-of-year) and respond to an annual survey.
The LCF centers have implemented cybersecurity programs at a moderate base-line with compensatory controls in accordance with the National Institute of Standards and Technology Special Publication 800-53. Project-specific security requirements can also be discussed and potentially addressed; the proposer should contact the center directly for details. Risk is currently mitigated using available enterprise infrastructure and network-based tools to provide perimeter protection and vulnerability resolution. System and application level security controls are implemented in accordance with industry and federal best practices. All non-temporary user data resides on centralized file and archival storage systems that are regularly backed up. These data are protected and segmented from other users through standard access controls.
Data sensitivity and protection
The LCF centers cannot guarantee zero risks to information stored on their systems. Requests for processing sensitive (e.g., proprietary or export-controlled) data must therefore be clearly identified in the proposal. Information sensitivity concerns apply to the intent of the research, the storing and compiling of source code, the loading and execution of application software, the input data for the application software, the output data generated by the application software, and data resulting from the analysis of output data. If, for example, the application software has an Export Control Classification Number (ECCN), prospective users should state this fact in the proposal and provide the ECCN. Prospective users should work with the center to identify appropriate levels of data protection. Greater levels of protection not typically offered may be provided by the principal investigator’s funding source. Generally, the LCFs expect the awardees to work with the center’s resources to facilitate the awarded project’s data management plans including retention, sharing, curation, and transfer of data assets.
Electronic applications for new proposals will be accepted starting Monday April 11, 2022, thru 8:00 pm EDT on Friday, June 17, 2022. All proposals must follow these instructions. INCITE reserves the right to decline consideration of proposals not compliant with these instructions. Hardcopies will not be accepted for review.
Awards are expected to be announced in November 2022. Access to a computing system, granted over the 2023 calendar year, will be established upon completion of the appropriate agreements and in accordance with center-specific procedures. Contact us for questions about the application procedure.