Add a live coop test harness (autonomous two-client testing)

[NonPolynomialTim]

Add a live coop test harness (autonomous two-client testing)

Why

There's currently no automated way to test coop end-to-end: every change to the netcode is validated by hand, two people clicking through a session. This PR adds a live test harness, an in-game command server plus Python drivers that spawn real game instances, drive them through menus/saves/lobby/geoscape/battle, and assert that host and client stay in sync. It runs unattended, needs no external mod and no pre-existing save, and builds on a stock checkout.

What the harness is

An in-game command server (src/CoopMod/TestServer.{h,cpp}, active only when the OXC_TEST_PORT env var is set, so zero footprint in normal play) that a Python driver talks to over newline-delimited JSON on 127.0.0.1:<port>. It lets a script read game state and drive a real running client, and lets those interactions be baked into automated functional tests across two live coop instances.

flowchart LR
  subgraph DRV["Python driver · tools/coop_test/"]
    direction TB
    GC["GameClient<br/>cmd() / ok() / wait_for()"]
    T["test_*.py<br/>baked cmds + asserts"]
  end
  subgraph HOST["HOST · OpenXcom.exe · OXC_TEST_PORT=47801"]
    direction TB
    IO["Socket IO thread (SDL_net)<br/>never touches game state"]
    Q["_inbox / _outbox<br/>(mutex)"]
    P["pump()<br/>main thread, per frame"]
    EX["execute()<br/>~45 handlers"]
    G["Real game objects<br/>Game · SavedGame<br/>State · BattlescapeGame"]
    IO --> Q --> P --> EX --> G
    G -.->|reply| Q
  end
  subgraph CLI["CLIENT · OpenXcom.exe · 47802"]
    direction TB
    G2["same internals<br/>IO → pump →<br/>execute → game"]
  end
  GC -->|"JSON + newline"| IO
  GC -->|"JSON + newline"| G2
  G <-->|"real coop sync<br/>connectionTCP :47900"| G2

Loading

Key property: the socket thread does I/O only; every command executes on the main thread via the per-frame pump(), so all state access is race-free. Handlers invoke the real State methods (Profile::buttonOK, BuildNewBaseState::placeAt, UnitWalkBState, …) rather than faking SDL input, so a command exercises the exact path a human click would. The two game instances sync to each other over the real connectionTCP coop link, which is the actual product under test; the tests assert host-vs-client agreement to catch replication desyncs.

What's in this PR

• src/CoopMod/TestServer.{cpp,h}: the in-game command server (~45 command handlers covering session/introspection, lobby bootstrap, geoscape play, battlescape combat, transfer/UI).
• tools/coop_test/: the Python drivers plus harness.py (the GameClient socket wrapper and hermetic make_user_dir).
• Minimal wiring in existing files: a per-frame TestServer::pump() call in Game::run(), a read-only Game::getStates() accessor, CMake / vcxproj(.filters) registration, and a few public-ised State entry points so the harness can drive real handlers instead of synthesizing input.
• .agents/docs/coop-test-harness.md (architecture and command catalog).

All of it is gated behind OXC_TEST_PORT; with the var unset the server never starts and the added surface is inert.

Builds cleanly on a stock checkout

A few handlers drive coop features that aren't in this repo yet (soldier ownership transfer, a multi-rendezvous server browser, a TX-queue drop counter). Rather than #ifdef the feature code or stub things out by hand, the harness detects each feature at compile time with C++17 __has_include. Detection lives entirely in TestServer.cpp; nothing is added to any other file:

Optional feature (not in this repo)	Compile-time detector
soldier ownership transfer	__has_include("TransferSoldierMenu.h" && "TransferNoticeState.h")
multi-rendezvous server browser	__has_include("../Interface/DisableableComboBox.h")
TX-queue drop counter	OXC_HARNESS_HAS_TX_DROP_COUNTER (harness macro, default 0), see note

Note: that counter is a namespace-scope global with no dedicated header, so __has_include can't see it and SFINAE can't probe it (referencing an undeclared name is a hard error, not a substitution failure), so it uses a harness-owned macro that is off by default.

When a feature is absent (as here), its handlers compile out and answer "<feature> not built"; the harness builds and runs. If such a feature later lands in this repo, __has_include re-fires at build time and the matching handlers switch on automatically, with no manual toggles and no feature flags in feature code.

Runs on any machine (hermetic)

make_user_dir() writes a fresh minimal options.cfg that pins the stock xcom1 master (no external mods), with intro/audio/mouse-capture off and a small 640×400 window (so it doesn't grab focus while running). OpenXcom defaults every unspecified key and resolves data (UFO/TFTD/standard/common) from the exe's own dir. No local config is read; no pre-existing save is required. The tests bootstrap a brand-new campaign each run.

Verification

• Full serial MSBuild Release|x64 returns 0 Error(s); OpenXcom.exe links with no unresolved externals (confirms the gated feature symbols don't leak when compiled out).
• boot_check on a fresh generated config brings up active mods [xcom1 v8.6.0] only, display 640×400, data loaded, 0 errors.
• Full suite run from hermetic dirs; every test spawned instances and connected on both ports (isolation confirmed):

Test	Result on a stock checkout
boot_check	pass (single-instance smoke)
test_geoscape_sync	runs fully; currently reports a geoscape host/client divergence, a real finding surfaced by the harness, worth triage
test_transfer_fresh / test_bug_fixes / test_transfer_rollback	report soldier-transfer feature not built (feature not in this repo)
test_server_browser	reports server-browser feature not built
test_txq_flood	TX-drop counter compiled out

The four feature-gated results are expected on this repo; those tests light up when their feature is present. boot_check passes, and test_geoscape_sync already earns its keep by flagging a sync divergence.

How to run

# build once (serial; the project can exhaust the compiler heap with /m)
MSBuild src/OpenXcom.2010.sln /p:Configuration=Release /p:Platform=x64

# any test: spawns real windowed instances and tears them down
python tools/coop_test/test_geoscape_sync.py
python tools/coop_test/boot_check.py

Notes / limitations

• The Python driver is Windows-only for now (SDL_net plus subprocess/STARTUPINFO window placement); the in-game server itself is portable.
• Entirely opt-in: no behavior change unless OXC_TEST_PORT is set.
• Feature-gated tests are included so they're ready when their features land; they no-op cleanly until then.

[NonPolynomialTim]

@xcomcoopdev I've been developing this and using it to create automated bug repros, as well as to let Claude run through scenarios autonomously. It has been extremely handy and has saved me a ton of time, and I figured I'd open a PR here in case you wanted to have it available here too.

I also don't know if you saw my offer to set up automated builds here (because the PR was already closed Idk if it sends notifications or not): #20 (comment), but I also plan on building out a much wider and more robust set of tests to be able to catch regressions in my changes and I'd be happy to wire that into the automated builds as well to prevent regressions. An automated test suite like this would also help when syncing with the OXCE official repo to make sure that the merge didn't break anything, so you'd be able to sync more often and with less stress

[NonPolynomialTim]

@xcomcoopdev thank you for push access! Do you have any guidance on what sorts of PRs you'd prefer I didn't merge without you looking at it first? I'm thinking large, architectural changes like the soldier transfer PR (bc it touches the save architecture) and like the test harness PR I would leave until you had a chance to look at them, but bugfixes I would merge myself?

[xcomcoopdev]

Hi, I looked through those 4 pull requests, so you can approve them now. The "main" branch is protected, so I’m not completely sure what permissions you have there, but you can try. You should be able to create your own testing branch for nightly builds.

[xcomcoopdev]

Or I can just go ahead and approve those 4 pull requests now... 😅

[xcomcoopdev]

There are some merge conflicts in this pull request. Should I resolve them, or do you want to handle them?

[NonPolynomialTim]

I can handle those!

[xcomcoopdev]

@xcomcoopdev thank you for push access! Do you have any guidance on what sorts of PRs you'd prefer I didn't merge without you looking at it first? I'm thinking large, architectural changes like the soldier transfer PR (bc it touches the save architecture) and like the test harness PR I would leave until you had a chance to look at them, but bugfixes I would merge myself?

Hi, in my opinion you can merge PRs yourself as long as the changes are documented and the implementation works.

I’ll try to review your code changes weekly, and I’ll check and test everything before publishing anything to ModDB and/or mod.io.