The mia ecosystem of packages in R/Bioconductor is an extensive framework for modern microbiome data science based on the TreeSummarizedExperiment (TreeSE) data container. In parallel, alternative frameworks with similar objectives have also been developed by others, including phyloseq, QIIME 2 and Mothur. While most frameworks support most routine tasks for microbiome analysis, they might differ in terms of performance and efficiency. In this repository, we provide an extensive benchmark between TreeSE/mia, QIIME 2, phyloseq and speedyseq.
Comparisons include five routine operations: melting, agglomeration, assay transformation, alpha and beta diversity estimation. Each operation is applied to random subsets of samples and features from the Metalog database. Performance is measured in terms of execution time (s) and allocated memory (MB) over ten replicates using the bench package.
Benchmark:
Sample composition:
- Create Apptainer with build.sh
- Create feature/sample subsets with preprocess.sh
- Run benchmark with array.sh
- Visualise results with plot.R and composition.R
Currently, it is required to manually adjust some parameters between the steps. Technical details are further provided in the corresponding scripts.
Each operation was run on a single node of the CSC Puhti supercomputer cluster with 4 CPUs and 16 GB RAM. However, larger resources are required to preprocess the Metalog dataset into feature/sample subsets, especially for QIIME 2. While the original benchmark was parallelised using SLURM array jobs, it is possible to perform single operations locally with single.sh.
The current benchmark was conceived after several iterations in a continuously developing framework, using the most extensive microbiome data resource to date.
The initial version of the benchmark, which compared only mia and phyloseq based on multiple smaller datasets of variable size, is available in the legacy branch of this repository, and detailed information is provided in the related README.
This work is part of mia. The scripts and results in this repository are openly accessible under an Artistic License 2.0.
- Bolyen, Evan, et al. "Reproducible, interactive, scalable and extensible microbiome data science using QIIME 2." Nature biotechnology 37.8 (2019): 852-857.
- Borman, Tuomas, et al. "Orchestrating Microbiome Analysis with Bioconductor." bioRxiv (2025): 2025-10.
- Kuhn, Michael, et al. "Metalog: curated and harmonised contextual data for global metagenomics samples." Nucleic Acids Research 54.D1 (2026): D826-D834.
- McMurdie, Paul J., and Susan Holmes. "phyloseq: an R package for reproducible interactive analysis and graphics of microbiome census data." PloS one 8.4 (2013): e61217.