Seek the primary key in the SQL Server upsert instead of scanning the table - #867
Open
vharseko wants to merge 1 commit into
Open
Seek the primary key in the SQL Server upsert instead of scanning the table#867vharseko wants to merge 1 commit into
vharseko wants to merge 1 commit into
Conversation
… table The MSSQL driver binds setString parameters as NVARCHAR. Under the server's SQL_Latin1_General_CP1_CI_AS collation, comparing the char(128) h column against an NVARCHAR value converts the column rather than the value, so no statement could seek the primary key: every read, delete and upsert scanned the whole table. The upsert runs that scan under WITH (HOLDLOCK, UPDLOCK), which range-locks the entire table instead of the single key being written - the lock footprint behind the intermittent "Transaction (Process ID N) was deadlocked on lock resources" failures of MsSqlTestCase#test_issue_496_2. Casting the parameter back to char(128) keeps the comparison seekable. Verified against mssql/server:2019-CU30 with driver 13.4.0: the MERGE plan goes from Clustered Index Scan (no seek predicate, CONVERT_IMPLICIT on the column) to Clustered Index Seek, and eight threads writing distinct keys in a 2000-row table finish in 3.7 s instead of 20.1 s. Other drivers keep the plain "?" placeholder, so their SQL is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
MsSqlTestCase#test_issue_496_2fails intermittently on CI with SQL Server error 1205, most recently in run 31627184500 — the only failing job of that build, 1 failure out of 31844 tests:The upsert already carries
WITH (HOLDLOCK, UPDLOCK)from an earlier deadlock fix, so this is a residual deadlock rather than a missing hint.Root cause
The MSSQL driver binds
setStringparameters as NVARCHAR. Under the server's defaultSQL_Latin1_General_CP1_CI_AScollation, comparing thechar(128)hcolumn against an NVARCHAR value converts the column rather than the value, so the primary key cannot be sought — everyread,deleteand upsert scans the whole table.That scan is what makes the deadlock possible: the upsert runs it under
HOLDLOCK, which range-locks the entire table instead of the single key being written.Fix
Cast the parameter back to
char(128)where thehcolumn is compared, so the comparison stays seekable. Other drivers keep the plain?placeholder — their SQL is byte-for-byte unchanged.Verification
Against
mcr.microsoft.com/mssql/server:2019-CU30-ubuntu-20.04with mssql-jdbc 13.4.0 (same image and driver as CI), replaying the storage access pattern ofupdate():@P0 nvarchar(4000)@P0 nvarchar(4000)MERGEplanCONVERT_IMPLICITon the columnmvn -pl opendj-server-legacy -Pprecommit verify -Dit.test=MsSqlTestCase->Tests run: 37, Failures: 0, Errors: 0, BUILD SUCCESS.Note: the 1205 deadlock itself could not be reproduced locally — 0 occurrences both before and after the change. It is flaky on CI as well, passing on 4 of the 5 JDK jobs of that same build. The effect on the deadlock is therefore inferred from the measured reduction in lock footprint, from a whole-table range lock down to a single key, rather than directly observed.
A complementary hardening — retrying a transaction on error 1205, which SQL Server documents as transient ("Rerun the transaction") — is deliberately left out of this PR.