[AMORO-4139] Delete resource and optimizer atomically when releasing optimizer - #4310
[AMORO-4139] Delete resource and optimizer atomically when releasing optimizer#4310seoyoniee wants to merge 1 commit into
Conversation
johntomcat7408-cmyk
left a comment
There was a problem hiding this comment.
Thanks for working on this. I traced the transaction flow through PersistentBase and NestedSqlSession: the outer doAsTransaction keeps one thread-local SQL session, so the nested deleteResource/getAs/unregisterOptimizer calls do not commit independently; this is a sound way to prevent future partial deletion. I do think this PR should add regression coverage before merge: (1) the success path deletes the resource and every optimizer row for the resource ID, and (2) a failure during optimizer deletion rolls back the preceding resource deletion, proving the intended atomicity rather than only the final happy-path state. There is also one scope point to clarify. The existing inconsistent state described in #4139 is not recovered by this patch: OptimizerController still dereferences resource.getProperties() immediately after getResource(resourceId), so a missing resource row fails before deleteOptimizerAndResource is reached. If container startup-only properties make terminating the underlying process impossible, could the PR define and test a graceful non-NPE behavior for that state (and document any cleanup limitation), or explicitly agree with maintainers that recovery will be a separate follow-up? Finally, the group argument is still unused; since the API is already being renamed, a resourceId-only signature would be clearer unless group is intentionally retained for future validation.
Why are the changes needed?
Close #4139
OptimizerController#releaseOptimizerdeleted the resource record and the optimizer records through two independent transactions:Both records describe the same optimizer, so deleting them in separate transaction can leave the resource gone while its optimizer rows survive - a state the release API can no longer recover from.
If the first call committed and the second one failed, the resource row was deleted while the matching optimizer rows survived. That state cannot be recovered through the API:
Brief change log
OptimizerManager: rename deleteOptimizer(String, String) todeleteOptimizerAndResource(String, String), since the operation now owns both
records. OptimizerController#releaseOptimizer was its only caller;
DefaultOptimizingService#deleteOptimizer is a separate method and is left untouched.
DefaultOptimizerManager#deleteOptimizerAndResource: run the resource deletion andthe optimizer deletion inside a single doAsTransaction(...). The existing deletion
logic is unchanged, only wrapped. Because NestedSqlSession keeps one SQL session per
thread, the nested doAs commits become no-ops and the outermost transaction commits
or rolls back both deletions together.
OptimizerController#releaseOptimizer: replace the two calls with the singletransactional one.
How was this patch tested?
Should I add a
TestDefaultOptimizerManagerto this PR?Add some test cases that check the changes thoroughly including negative and positive cases if possible
Add screenshots for manual tests if appropriate
Run test locally before making a pull request
Documentation