test(generator): pin that generating a spec twice gives the same bytes - #123
Merged
Conversation
Generated clients are committed and reviewed, so output that shifts between runs shows up as churn in a diff nobody made. Nothing held the property, and the usual way to lose it is a map iterated somewhere on the way out. Generating the corpus spec twice and comparing the bytes holds it. The five real specs I have are deterministic today, so this adds no fix. The corpus gains a second named error shape and one with no name of its own, which the guard needed: with a single error type in the package, iterating the map that holds them is stable however it is written, and the test could not fail. With two, disabling the ordering makes it fail on errors.go, which is what a guard has to do to be worth having.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated clients are committed and reviewed, so output that shifts between runs shows up as churn in a diff nobody made. Nothing held that property, and the usual way to lose it is a map iterated on the way out.
TestGenerationIsDeterministicgenerates the corpus spec twice and compares the bytes.The corpus needed enriching for the guard to mean anything
My first version passed while I was actively sabotaging the generator: I made
uniqueErrorTypesround-trip through a map, and the test still passed, because the corpus had exactly one error type and iterating a one-entry map is stable however it is written.The corpus now declares a second named error shape and one with no name of its own, and the same sabotage fails the test:
Both new error shapes also exercise paths worth having in the corpus: a second wrapper in one package, and the operation-derived naming from #61 for a body with no type name.
Current state
Generation is already deterministic: ACTIVATE, GitHub, Stripe, Mealie, and the corpus each produce byte-identical output across repeated runs, so this adds no fix. It stops the property from being lost quietly.
gofmt,golangci-lint,go vet ./..., andgo test ./...pass.