Stop reporting an ordinary disconnect as a send error - #977
Open
M-r-A wants to merge 1 commit into
Open
Conversation
LiteNetConnection.SendRaw refused to send unless the LiteNetLib peer was Connected, and logged an error otherwise. The peer's transport state changes on LiteNetLib's own thread, and ConnectionBase.Send has already approved the send against the protocol state by the time SendRaw runs, so the two can disagree and no check here can be race-free. LiteNetLib discards sends to a peer that has gone away, silently and without throwing, so the guard suppressed nothing the library would not have suppressed anyway. ServerTest routes ServerLog.Error into Assert.Fail, so that log line failed the suite whenever a client disconnected while the server was still ticking: 7 runs in 1000 idle and 7 in 100 under CPU load, against zero in 1100 runs afterwards. Two tests in the same class also asserted nothing. One waited for Players.Count == 0 before its client had connected, the other for Players.Count == 1 when it had already seeded that player itself; both returned on their first poll and passed whether or not the join happened. They now watch the client arrive and then leave. Making those waits real exposed a further problem: SetImplementation is additive and rejects duplicate handlers, which suits registering each state once at start-up, but the test listener registers per connection with a per-test type, so two tests using different state classes threw and took the test host down. ClearImplementation lets a caller replace a registration; production still registers once and never calls it. The shared wait helper's budget goes from 2000 ms, which was never derived and is thin on slower CI runners, to 5000 ms. The longest wait measured was 757 ms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ServerTest.Testfails intermittently — about 1 run in 140 idle, and 1 in 14when the machine is under CPU load — always with the same message:
Why it fails
Two independent state machines guard the same send:
ConnectionBase.SendDisconnectedPlayerManager.SetDisconnectedLiteNetConnection.SendRawConnectedThe server only learns a peer has gone when
OnPeerDisconnectedis delivered,and that happens at exactly one point per tick, inside
netManagers.ForEach(m => m.Tick()). The transport state changes wheneverLiteNetLib decides. A change landing just after that point leaves the rest of
the tick believing the peer is live — and
TickNetbroadcastsServerTimeControlPacketto every playing player unconditionally, every tick.So
Sendapproves the send,SendRawfinds the transport already gone, andlogs
ServerLog.Error.ServerTestroutes that intoAssert.Fail, which iswhat turns an ordinary client disconnect into a red build.
No teardown ordering can close this, because the state changes on a thread the
server does not control.
Why the guard is removed rather than corrected
NetPeer.Sendon a peer that has gone away returns normally, throws nothing,and logs nothing — LiteNetLib already discards it. I verified this directly
against the referenced version rather than assuming it.
So the guard suppressed nothing the library would not have suppressed anyway,
and existed only to report an unavoidable race as though it were a programming
error.
SendRawbecomes the unconditional send; a seven-line method becomestwo.
What else is in this change
The flake sits in a test class that had two further problems, and fixing them
turned out to be a prerequisite rather than a nicety.
Two of the four tests asserted nothing. One waited for
Players.Count == 0before its client had connected; the other forPlayers.Count == 1when it had already seeded that player itself. Bothconditions were true on the first poll, so the tests passed whether or not the
join they are named for ever happened — a broken loading state would still have
reported green. They now watch the client arrive and then leave, and the
keep-alive test asserts the join point it was blocked on has completed.
Making those waits real exposed a registry problem.
MpConnectionState.SetImplementationis additive and rejects a duplicatehandler for the same packet, which suits production registering each state once
at start-up.
TestNetListenerregisters per connection with a per-test type, soonce two tests genuinely connect with different state classes it threw
Packet ClientJoining:Server_KeepAlive already has a handlerand took the wholetest host down. This had stayed hidden precisely because those two tests
returned before their client finished connecting.
The polling waits are now a shared helper that reports what it observed
rather than bare
Assert.Fail("Timeout"), and emits one measurement line perwait on success as well as failure. That is what made the diagnosis possible:
the failures turned out to be confined to a single poll count, which is what
identified this as an ordering race rather than a slow machine.
Its budget also goes from 2000 ms to 5000 ms. The 2000 was never derived; the
longest wait measured over 1000 runs was 757 ms, so the margin is thinner than
it looks on a fast machine. A budget costs nothing when the condition holds,
because the wait returns on the poll that satisfies it.
Verification
1100 runs, zero failures, zero timeouts.
Worth more than the failure count: every failure had occurred at one specific
poll count, and that bucket is still reached — 38 of 1000 runs — and now never
fails. A green run would also be produced by a test that stopped reaching the
window, so this distinguishes a fixed race from a test that stopped looking.
Notes for review
MpConnectionState.ClearImplementation, which exists so a caller can replacea registration rather than only add one. Production still registers each state
once at start-up and never calls it. I would rather add the seam than have the
test suite depend on registration order, but it is the one line here that is
arguably test-driven, and I am happy to take a different approach if you'd
prefer.
ServerLog.error = Assert.Failis deliberately kept. Silencing it duringteardown was considered and rejected: it would suppress every genuine
server-side error the suite can catch, in order to hide one misclassified
line.
was not exercised by the runs above, so it is the one behavioural gap I would
flag.
dotnet teston a loadedrunner is the quickest way to confirm it independently.