Skip to content

Seek the primary key in the SQL Server upsert instead of scanning the table - #867

Open
vharseko wants to merge 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:mssql-upsert-index-seek
Open

Seek the primary key in the SQL Server upsert instead of scanning the table#867
vharseko wants to merge 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:mssql-upsert-index-seek

Conversation

@vharseko

Copy link
Copy Markdown
Member

Problem

MsSqlTestCase#test_issue_496_2 fails intermittently on CI with SQL Server error 1205, most recently in run 31627184500 — the only failing job of that build, 1 failure out of 31844 tests:

Transaction (Process ID 57) was deadlocked on lock resources with another process
and has been chosen as the deadlock victim. Rerun the transaction.
  at JDBCStorage$WriteableTransactionTransactionImpl.upsert(JDBCStorage.java:411)
  at JDBCStorage$WriteableTransactionTransactionImpl.put(JDBCStorage.java:377)

The upsert already carries WITH (HOLDLOCK, UPDLOCK) from an earlier deadlock fix, so this is a residual deadlock rather than a missing hint.

Root cause

The MSSQL driver binds setString parameters as NVARCHAR. Under the server's default SQL_Latin1_General_CP1_CI_AS collation, comparing the char(128) h column against an NVARCHAR value converts the column rather than the value, so the primary key cannot be sought — every read, delete and upsert scans the whole table.

That scan is what makes the deadlock possible: the upsert runs it under HOLDLOCK, which range-locks the entire table instead of the single key being written.

Fix

Cast the parameter back to char(128) where the h column is compared, so the comparison stays seekable. Other drivers keep the plain ? placeholder — their SQL is byte-for-byte unchanged.

Verification

Against mcr.microsoft.com/mssql/server:2019-CU30-ubuntu-20.04 with mssql-jdbc 13.4.0 (same image and driver as CI), replaying the storage access pattern of update():

before after
declared parameter @P0 nvarchar(4000) @P0 nvarchar(4000)
MERGE plan Clustered Index Scan, no seek predicate, CONVERT_IMPLICIT on the column Clustered Index Seek, seek predicate present
8 threads x 200 iterations on distinct keys, 2000-row table 20 124 ms 3 726 ms

mvn -pl opendj-server-legacy -Pprecommit verify -Dit.test=MsSqlTestCase -> Tests run: 37, Failures: 0, Errors: 0, BUILD SUCCESS.

Note: the 1205 deadlock itself could not be reproduced locally — 0 occurrences both before and after the change. It is flaky on CI as well, passing on 4 of the 5 JDK jobs of that same build. The effect on the deadlock is therefore inferred from the measured reduction in lock footprint, from a whole-table range lock down to a single key, rather than directly observed.

A complementary hardening — retrying a transaction on error 1205, which SQL Server documents as transient ("Rerun the transaction") — is deliberately left out of this PR.

… table

The MSSQL driver binds setString parameters as NVARCHAR. Under the server's
SQL_Latin1_General_CP1_CI_AS collation, comparing the char(128) h column against
an NVARCHAR value converts the column rather than the value, so no statement could
seek the primary key: every read, delete and upsert scanned the whole table. The
upsert runs that scan under WITH (HOLDLOCK, UPDLOCK), which range-locks the entire
table instead of the single key being written - the lock footprint behind the
intermittent "Transaction (Process ID N) was deadlocked on lock resources" failures
of MsSqlTestCase#test_issue_496_2.

Casting the parameter back to char(128) keeps the comparison seekable. Verified
against mssql/server:2019-CU30 with driver 13.4.0: the MERGE plan goes from
Clustered Index Scan (no seek predicate, CONVERT_IMPLICIT on the column) to
Clustered Index Seek, and eight threads writing distinct keys in a 2000-row table
finish in 3.7 s instead of 20.1 s.

Other drivers keep the plain "?" placeholder, so their SQL is unchanged.
@vharseko vharseko added bug jdbc java Pull requests that update java code performance Performance / concurrency / lock-contention work concurrency Thread-safety / race-condition bugs labels Aug 14, 2026
@vharseko
vharseko requested a review from maximthomas August 14, 2026 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug concurrency Thread-safety / race-condition bugs java Pull requests that update java code jdbc performance Performance / concurrency / lock-contention work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant