⚓ Eurybia monitors model drift over time and securizes model deployment with data validation

A curated list of awesome open source tools and commercial products for monitoring data quality, monitoring model performance, and profiling data 🚀

In this repository, we will present techniques to detect covariate drift, and demonstrate how to incorporate your own custom drift detection algorithms and visualizations with SageMaker model monitor.

These are my notes of the Udacity Nanodegree Machine Learning DevOps Engineer.

資料科學的日常研究議題

Simulation, testing and comparison of state of the art Unsupervised Concept Drift Detectors used in a batch Machine Learning scenario.

Detect behavioural drift between LLM versions before you upgrade. Compare model responses, classify regressions, and generate migration reports with validated prompt patches.

In this project, we illustrate how the Kolmogorov Smirnov (KS) statistical test works, and why it is commonly used in Machine Learning (ML), Deep Learning (DL) and Artificial Intelligence (AI).

Learn how to handle model drift and perform test-based model monitoring

A reproducible benchmark for whether commercial LLMs silently drift over time. Deterministic graders, balanced tasks, every raw response in git.

Hey LLM, you okay? — pyramid-ordered LLM testing CLI for CI/CD. One YAML for every layer, LLM-as-a-judge gates, and A/B triage that tells prompt regressions from model drift.

Time‑aware NBA forecasting pipeline (R² 0.94 points) with rolling CV, leakage guards, and automated retraining; includes backtesting reports and model card.

ModelPulse helps maintain model reliability and performance by providing early warning signals for these issues, allowing teams to address them before they impact users significantly.

From model.fit() to models that survive production — a phased climb through ML fundamentals and the MLOps layer: versioning, serving, drift, scale.

The Taravangian Test for AI: catch silent model degradation before your users do. SPC-based reasoning-quality monitoring for Claude, GPT, Gemini, and Grok.

Decision DNA is an AI governance and monitoring platform designed to supervise machine learning models used in credit risk decision systems. The system helps detect model drift, operational risks, and security threats, while maintaining transparent and auditable AI decision pipelines.

Production-style ML monitoring template on the Wine Quality (red) dataset: Evidently (data/target/prediction drift, data quality) + adversarial validation, PSI/JS effect sizes, SHAP/PDP, slice analysis, and an Alert Policy with actions

"Past performance of machine learning model is no guarantee of future results." We call it "model drift" or "model decay". This repository will introduce various methods for detecting model drift.

Diagnostic test suite for measuring whether AI models preserve the named Origin boundary inside AI Foundations / Origin | Continuum.

Machine learning monitoring project focused on detecting data drift and model performance degradation in production-like data using Python, scikit-learn and Evidently AI