IBBIS is pleased to announce the publication of “Strengthening nucleic acid biosecurity screening against generative protein design tools“, led by Microsoft and co-authored with a consortium from across the synthesis screening field. The paper shows that screening tools, including the Common Mechanism, can be made resilient to AI protein designs, while also setting a precedent for responsible red-teaming of synthesis screening. The study is described in a Science news article and has received coverage in Nature News, The Financial Times, The Washington Post, MIT Technology Review, BBC World Service, NPR, and elsewhere.

DNA synthesis screening has been highlighted over and over as a key safeguard as AI-Bio capabilities advance, but there has also been a fear that AI tools could be used to redesign harmful proteins while evading detection by screening tools. While preparing for a biosecurity workshop in October 2022, Microsoft researchers decided to test this by generating thousands of AI-rewritten versions of the ricin toxin. They found that screening did not consistently catch these proteins, and started a confidential collaboration to make screening tools more AI-resilient.

Many of us in the field expected people to be trying these sorts of techniques, and were concerned these attempts would be shared publicly, before any affected systems could be patched. So, when we heard this team had put together a well-constructed test and were following good cybersecurity practices in probing whether there was an issue, it was a really good sign that we could address these issues safely.
― Dr. Nicole Wheeler, UK ARIA Programme Manager and lead developer of Common Mechanism v1.0

The study involved a collaboration between Microsoft researchers, screening tool developers, and leading synthesis providers Twist and IDT. Microsoft researchers used open-source protein design software to generate 76,089 variants of 72 proteins of concern, primarily well-known toxins. The screening tool developers and synthesis providers screened the variants, then updated their screening tools to more effectively flag variants. The findings were not disclosed publicly until protective measures were widely distributed.

As we prepared to share this work, we faced a challenge balancing open, reproducible science and safeguarding against biorisks. “Vulnerability reporting at the intersection of software and biology is fairly uncharted territory, so figuring out what norms we want to set is a really important process,” said Dr. Wheeler. We spoke to stakeholders including government agencies, international biosecurity organizations, and policy experts, and got very different recommendations ― some said we should publish our results as openly as possible, others said that the information hazards were so great that we should only do closed briefings with trusted government officials.

We’ve settled on a tiered system for access in which some data and code will only be available after third-party review. IBBIS and Microsoft spent many hours considering information hazards, dividing access into tiers ― which prompted us to move some data into the paper ― and developing a transparent review process. When IBBIS receives an access request, it will be reviewed independently by two IBBIS staff members through a process inspired by best practices for customer screening in DNA synthesis. Microsoft has also created an endowment to support the costs of IBBIS’s review work and computational hosting so that this data will be available in perpetuity.

I’m particularly excited about this because I’ve seen managed access identified as a potential intervention for biological design tools by many biosecurity experts. This is, as far as we know, the first time a leading scientific article has formally endorsed a tiered-access approach to responsibly manage information hazards.
― Tessa Alexanian, IBBIS Technical Lead

This approach is an experiment, and we are eager to evolve our managed access policy in response to our first access requests, as well as changes in the biosecurity landscape. We hope this framework offers a way forward for researchers and journals to balance openness and security, and IBBIS is already speaking with other researchers about using the same framework for managing access to data from their work.

This publication represents a milestone for AI-resilient synthesis screening. The same workshop that launched this study led to the Responsible AI x Biodesign Commitments, currently signed by over 180 AI developers, which include a commitment to “support the development of new strategies to improve DNA synthesis screening, with the aim of better detecting hazardous biomolecules before they can be manufactured”. This is exactly what happened in our study, and we hope to collaborate on similar future efforts.

Strengthening DNA synthesis screening must go beyond AI resilience, however. The most advanced screening tools now implement robust methods to detect AI-designed proteins of concern, but early results from the Global DNA Synthesis Map indicate that screening remains voluntary, inconsistent, and globally fragmented; it is easy to find companies or intermediaries that do not screen order sequences. As the paper states, “Institutional and community strategies will also be important to pursue… National governments have several critical roles to play, including incentivizing adherence to screening best practices”. IBBIS is working to increase the global share of synthesis orders for which sequences and customers are screened by engaging with policymakers, developing new standards, and creating free, open-source screening tools.

Full paper citation: Bruce J. Wittmann, Tessa Alexanian, Craig Bartling, Jacob Beal, Adam Clore, James Diggans, Kevin Flyangolts, Bryan T. Gemler, Tom Mitchell, Steven T. Murphy, Nicole E. Wheeler, and Eric Horvitz. Strengthening nucleic acid biosecurity screening against generative protein design tools. Science, 390, 82-87 (2025).DOI:10.1126/science.adu8578

Feature image of generative protein design Towards Breakthroughs in Protein Structure Calculation and Design by Argonne National Laboratory on Flickr, CC BY-NC-SA 2.0.