About BEACON
BEACON is an open collaboration that coordinates, integrates, and advances the work and science of the critical assessment community. Its key objectives are to increase the collective impact of critical assessment approaches across scientific domains, advancing rigour and reproducibility in research, and to create a comprehensive and adaptable benchmarking framework to support the development and evaluation of AI approaches.
Anchored by a think-tank on benchmarking and critical assessment, designed to promote methodological coherence and thought leadership across domains, BEACON provides an open platform to conduct challenges, engage benchmarking and participant communities in consortium activities, and provide easy access to data and methods.
The problem: Science’s replication crisis
Scientific progress is the engine of human advancement, yet it is facing a critical bottleneck. Rigour and reproducibility are essential for progress, but a comprehensive framework for validating research is lacking. This replication crisis means that billions of dollars and decades of research, particularly in critical areas like cancer, cardiovascular and Alzheimer’s disease, are wasted building on a foundation of uncertain, unreliable, and irreproducible results.
The explosive growth of Artificial Intelligence is accelerating this crisis. AI can generate hypotheses and predictions at a pace that has completely outstripped traditional peer review and our ability to test these hypotheses experimentally. Without a new operating system for validation, we cannot trust the outputs of these powerful new tools, and the promise of AI-driven medicine will stall.
The proven solution: Critical Assessment
The solution is not to slow down science, but to build a better system for validating it. This system is open, community-driven Critical Assessment, a method where independent groups compete in challenges to solve problems using shared data, with their results judged against objective, gold-standard benchmarks.
This model has a spectacular track record. The CASP challenge, now in its 17th instance, nurtured the protein folding community and was a key contributor to the development of the Nobel Prize winning AlphaFold algorithm. Other Critical Assessment initiatives like DREAM, OpenADMET and CACHE have organized more than 100 challenges and successfully accelerated algorithm development for systems biology, translational medicine and drug discovery. DREAM in particular has tackled a raft of important problems such as breast cancer diagnosis through mammography images, reverse engineering gene regulatory networks in biology, drug sensitivity, and predicting the smell of molecules.
The initiative: BEACON (Benchmarking, Evaluation and Assessment Consortium)
Until now, these powerful assessment efforts have been siloed. BEACON unites the world’s most successful and credible Critical Assessment organizations — the founders of CASP, DREAM, Sage Bionetworks, CACHE/Conscience, and OpenADMET into a single, coordinated and open consortium.
BEACON’s mission is to become the unbiased, trusted arbiter of rigour for the entire scientific ecosystem, ensuring that innovation is accompanied by accountability.
Strategic scope
BEACON will operate through two horizontal support platforms and six initial vertical focus areas to tackle some of science’s biggest challenges:
Horizontal support platforms:
- The Think Tank: A center for methodological innovation to promote thought leadership and the development of new validation methods.
- The Benchmarking Platform: A robust technology infrastructure to host challenges, engage the community, and provide easy access to data and methods.
Initial vertical focus areas:
- Structural Biology of Cell Machinery: Moving beyond single proteins to map the entire “virtual structural biology” of the cell
- Ligand-Protein Interactions: Radically accelerating drug discovery by benchmarking methods that find pharmacological modulators for human proteins, in partnership with Target2035
- Scientific Literature Assessment: Deploying structured, rule-based evaluation to overcome the deficiencies of the conventional peer-review process
- Disease Mechanisms: Creating AI-enabled solutions to validate new disease mechanism findings, starting with Alzheimer’s
- Virtual Cell Biology: Serving as the unbiased evaluator for the ambitious global effort to build “virtual cells,” ensuring their fidelity and trustworthiness
- Frontier “n-of-1” Science: Developing new frameworks to benchmark candidate discoveries for single-patient scenarios, focusing on cures and disease mechanisms where no “ground truth” exists
The team and the opportunity
BEACON is structured as a Limited Partnership managed by Conscience. It is led by the founders of scientific benchmarking efforts, with a Governance Committee that includes:
John Moult (Co-Founder and Chair, CASP; Co-founder, CAGI)
Gustavo Stolovitzky (Co-Founder and Chair, DREAM Challenges)
Pablo Meyer (Vice Chair, DREAM Challenges)
Julio Saez-Rodriguez (Head of Research, EMBL-EBI and Director, DREAM Challenges)
Patrick Walters (Chief Scientist, OpenADMET)
Aled Edwards (CEO, Structural Genomics Consortium and CSO, Conscience, Founder CACHE)
Luca Foschini (President, Sage Bionetworks)
Peng Fu (CEO, Conscience)
Become a founding Limited Partner
We are seeking visionary philanthropists and organizations to join BEACON as Limited Partners (LPs). This is a unique, high-leverage opportunity to contribute to repairing the foundational infrastructure of science itself. As an LP, you will have a seat on the Governance Committee, providing oversight and helping to direct the future of scientific rigour. This investment ensures that the coming wave of AI-driven discovery is built on a foundation of truth, accelerating progress across all human endeavors. If you are interested in becoming an LP, please contact us.
