Testing v0.1 of the Common Mechanism
The Common Mechanism is a free, open-source, globally-available tool for synthesis screening. Our automated software for screening sequences of nucleic acids (including DNA and RNA) was shown to sensitively flag sequences of concern without raising unnecessary alarms.
This page describes the results of testing done before the release of version 0.1 of the Common Mechanism sequence screening package. Before the launch of the software in May 2024, our testing showed:
- Compliance with industry best practices when tested against real order customer streams by beta testers such as Twist Bioscience
- <2% false positive flags of real synthetic biology designs
- Performance on par with industry screening tools in a standard test set
- Resilience to AI-generated challenges
Detailed Performance Information
Compliance with best industry practices
Twist Bioscience ran a series of real customer orders through the Common Mechanism, finding that the tool would allow users to screen their orders in compliance with industry best practices.
A number of other synthesis providers, including benchtop manufacturers, academic biofoundries, and smaller synthesis companies were engaged to install and test the tool during the beta testing process.
The Common Mechanism is designed to meet and exceed current screening guidance, such as the IGSC Harmonized Screening Protocol and U.S. Framework for Nucleic Acid Synthesis Screening. IBBIS is a member of the International Gene Synthesis Consortium and an active participant in efforts to standardize best practices for synthesis screening, such as through ongoing engagements with EBRC and NIST.
<2% false positive flags of real synthetic biology designs
The Common Mechanism was run against the iGEM Registry of Standard Biological Parts, a public database capturing over 20,000 biological parts used to build synthetic biology devices and systems. Teams participating in the international iGEM Competition submit the biological parts designed for their projects to the Registry every year.
In total, less than 2% of the parts were flagged (422/22809). Not all of these represent false positives; in fact, the search highlighted several sequences that we have reported to iGEM HQ as likely requiring safety flags.
Results of the Common Mechanism run against the iGEM Registry. 124 parts were flagged because of homology to a regulated toxin (“biorisk” in the Sankey diagram above), 9 due to homology to a regulated virus, and 315 for homology to non-viral regulated pathogens.
Performance on par with industry screening tools
IBBIS is among the screening tools providers participating in the development of a standard test set for benchmarking synthesis screening performance.
As reported in Progress and Prospects for a Nucleic Acid Screening Test Set, the Common Mechanism showed strong alignment with other screening tools: 100% agreement for Orbivirus sequences, 99.5% agreement for bacterial sequences from Coccidioides and 92.1% agreement for fungal sequences from Francisella tularensis. Other tools showed similar levels of agreement and disagreement.
The Common Mechanism is currently being tested against sequences from every pathogen on the U.S. Select Agent List and Australia Group Common Control Group List, paired with sequences from non-listed close relatives. Results for this testing will be reported once consensus flags have been established among the working group of RTX BBN, Aclid, and Battelle.
Resilience to AI-generated challenges
The Common Mechanism has been tested against various obfuscation challenges, including functional reformulation using AI protein design tools, camouflaging sequences with “decoy” non-hazard sequences that could be removed using standard laboratory methods, and splitting hazards for later reassembly.
A preprint addressing resilience to synthetic homologs created with protein design tools is now available on biorxiv: Toward AI-Resilient Screening of Nucleic Acid Synthesis Orders: Process, Results, and Recommendations.