Skip to content

DEV_COMMANDS.md

Latest commit

 

History

History
60 lines (44 loc) · 1.33 KB

DEV_COMMANDS.md

File metadata and controls

60 lines (44 loc) · 1.33 KB
Raw

MagicAI developer campaign commands

This document covers long evaluation runs. Evaluation artifacts do not train models, modify weights, or promote cases automatically.

Recommended campaign layout

CAMPAIGN="oracle-exhaustive"
OUTPUT="quality-results/${CAMPAIGN}"
LOG="logs/${CAMPAIGN}.log"
PIDFILE="logs/${CAMPAIGN}.pid"
mkdir -p "$OUTPUT" logs

Background execution

nohup env PYTHONPATH=. python -u -m tests.quality.oracle_exhaustive_test \
  --workers 4 \
  --shard-size 250 \
  --output-dir "$OUTPUT" \
  > "$LOG" 2>&1 &
echo $! | tee "$PIDFILE"

Monitor

tail -f "$LOG"
PID=$(cat "$PIDFILE")
kill -0 "$PID" 2>/dev/null && echo running || echo stopped

Resume

PYTHONPATH=. python -u -m tests.quality.oracle_exhaustive_test \
  --workers 4 \
  --shard-size 250 \
  --output-dir "$OUTPUT" \
  --resume

Package results

tar -czf "${CAMPAIGN}.tar.gz" "$OUTPUT" "$LOG"
sha256sum "${CAMPAIGN}.tar.gz" > "${CAMPAIGN}.tar.gz.sha256"

Worker policy

Use bounded parallelism. Start with four workers and reduce the value if the machine becomes memory-bound. Do not launch every shard simultaneously.

Interpretation

A harness PASS is not factual authority. Review premise validity, semantic contracts, evidence sufficiency, routing origin, and LLM-call counts.