Announcing a draft AI policy
Overview
Generative artificial intelligence tools (“AI”)[1] are spurring a great deal of debate in the scientific and open source software communities. This document announces an initial policy regarding the use of AI for the purposes of software development within the rachis ecosystem.
As a starting point, we are adopting the SciPy project’s AI Policy[2], with some adaptations for our community. Because adoption of AI-assisted coding is moving quickly, we feel that it’s important to have a transparent policy in place while we begin the process of getting feedback from our community. The rachis AI Policy can be found here.
Our current use of AI in software development
Within the rachis development team[3], we have begun exploring and using various LLMs in our development work; both to understand the current landscape of these tools, the opportunities and risks associated with their use, and to determine if and how these tools can help us to better serve our user communities.
Currently, we are exploring low-risk applications such as:
- Developing utilities for simplifying workflows, both for users and for developers.
- “Side projects” where errors are low consequence, and thus are good testing ground.
Of those of us who have started using these tools, there is a consensus that AI-assisted coding tools have dramatically improved over the last 6-12 months, in both their capabilities and accuracy. We think that these tools, particularly coding agents, may provide a mechanism for composing new code and/or changes to existing code at a faster pace than human development alone providing the potential for us to address more issues and feature requests in each development cycle. And that code produced by the most recent agentic systems such as OpenAI Codex and Claude Code, while often highly workable, still requires careful human software engineer guidance and scrutiny to ensure correct behavior and accuracy of results.
Drawing from Matplotlib’s policy regarding the use of generative AI[4], we think it’s clear that these tools are “evolving rapidly and can be helpful”. Despite the use of new tools however, our underlying priorities remain the same: delivering research software that is trustworthy, accessible, and transparent in support of cutting edge, reproducible biological data science.
How AI impacts our code review process
tl;dr: it doesn’t right now. Maybe it will in the future, but we don’t know where this is going yet.
We treat code we are developing the same, regardless of the methods or tools that were used to create it. As you may or may not know, before we merge code into the main branch of our repositories, it undergoes a thorough code review by at least one additional developer who did not write that code. When a code’s developer is satisfied with their new code, they submit a pull request for review by another developer (the code’s reviewer). By handing it off for review, the developer implicitly asserts that they fully understand what the code is doing, and that they are confident that there is sufficient test coverage and test quality for new features/methods. The reviewer reviews the code and ensures that they also fully understand how the code works and that the tests are sufficient. For most code additions, this is an iterative process where the reviewer provides feedback and change requests, the developer addresses those, and eventually the code is merged. This ensures that at least two humans (the code’s developer and the code’s reviewer) see every line of code that is being merged. We don’t pass any code off for use until those two humans are satisfied with it. For complex changes, there are sometimes multiple reviewers involved.
Our developers can use AI coding tools in accordance with our AI Policy, but that doesn’t change the process. The developer must still understand the code before passing it off for review, and the review works in the same way. Importantly, the developer and the reviewer are both still human: the AI agent might help the developer, but the AI agent doesn’t become the developer.
Humans drive the decision making, and humans take responsibility for the code we are producing and distributing. At its most basic level, we are simply utilizing a new tool to assess whether it can help us better serve our user community, but we still take the same amount of time to understand and improve upon our code, and ultimately still retain full responsibility for it.
All of that said, code review is probably the biggest bottleneck in moving new functionality out to users. We plan to monitor how AI usage for code review evolves (e.g., as of this writing, new models are finding long-standing security vulnerabilities in code that human developers missed for years, suggesting that some form of AI-augmented code review may be helpful in the future). If we start considering changes, we’ll have that discussion in the open.
AI policy in distribution-associated plugins versus stand-alone plugins
Since Day 1, a goal of our community has been to support decentralized development by enabling anyone to write plugins to QIIME 2 (and now also to MOSHPIT and our other distributions). There are plugins that are components of the rachis distributions, and there are plugins that are not associated with any distribution, which the rachis developers or others create as “stand-alone” plugins, meaning they are installed independently of the distributions.
The emergence of AI coding tools requires us to navigate a world where it is now very easy for someone to generate a high volume of code (e.g., a full plugin, or even suite of plugins) in a very short span of time without fully understanding what that code is doing. We can set requirements for our distribution-associated plugins, and we can make recommendations for stand-alone plugins (and in some cases help users assess whether those recommendations are being adopted).
Distribution-associated plugins
We plan to adhere to the rachis AI Policy for distribution-associated plugins. This is an expectation of core developers as well as outside contributors. As indicated above, we’ve been experimenting with these tools while maintaining the principles that we believe lead to our high quality software, and up until now different developers have disclosed this in different ways. With the introduction of an official policy, we plan to standardize on disclosure approaches, and use those going forward (and backporting these as it makes sense). Plugins associated with any rachis distribution distributed by the rachis development team must adhere to these standards; and moving forward adherence with this policy will be one criterion for considering which plugins can be included in these distributions.
Stand-alone plugins
Stand-alone plugins are not necessarily developed by members of the rachis development team, and as a result the rachis development team can't take responsibility for these plugins. The developer of the plugin is responsible for their plugin, and they should provide information (e.g., in their README.md or documentation) that help you understand who they are, what they are providing, and how it has been validated. We will look for that information, as well as AI disclosures, when we review plugins for inclusion in the Library, but you as the user should look for this information as well.
We plan to start integrating some new functionality in the QIIME 2 Library to help prospective users assess plugins they are considering using. This may include indications of code coverage and quality, presence or absence of documentation, and references to AI disclosure statements.
We additionally reserve the right to reject contributions to QIIME 2 Library for any reason, including if they don't appear to align with our AI Policy.
Going forward
As described above, we consider the rachis AI Policy to be a starting point for additional discussion and policy decisions. Our ultimate goal is balancing management of risks of these technologies with realization of new opportunities for advancing microbiome science for broad societal benefits. We expect to start some public discussion on this topic on the forum in the coming months. Thanks, as always, for your interest in QIIME 2 and our growing ecosystem of data science tools!
For the purpose of this document, we adopt the definition of AI presented in the SciPy AI Policy: “AI" herein refers to generative AI tools like large language models (LLMs) that can generate, edit, and review software code, create and manipulate images, or generate human-like communication. ↩︎
In accordance with SciPy’s BSD-3-Clause License. ↩︎
The
rachisdevelopment team is defined here as the software developers and contributors within the Caporaso and Bokulich Labs. ↩︎See Matplotlib’s contribution guidelines regarding the use of generative AI. ↩︎