Implements Jakarta Persistence 3.2

[cristof]

Hi! This work is an effort to implement JPA 3.2. I've started it a long time ago. Richard Zowalla picked it up and, with AI help (Claude), implemented the missing features, including some JPA <3.0 that weren't implemented.
I've tested it against default database, h2, mariadb(lts) e postgresql (18).
Please, check it against your favorite DB so we may fix some edge cases. It would be great if you can run TCK to be sure of the implementations.

[solomax]

Hello all,

little status update:

all in-memory and dockerized DBs listed in pom.xml are currently green for me except for

Oracle
HerbDB

I'm trying to fix Oracle :)

[solomax]

I'll start another thread :)

I would like to propose this script https://github.com/apache/openjpa/blob/9da3e207072a3b5cf553e8a7f17cd41f609839bc/scripts/complex_test.sh
As drop-in replacement for https://github.com/apache/openjpa/blob/9da3e207072a3b5cf553e8a7f17cd41f609839bc/scripts/run-build-matrix.sh

IMO it is better because:

• it uses docker configs inlined into pom.xml
• it is more flexible: any available versions can be tested

WDYT?

[rzo1]

Ok for me.

[solomax]

@ilgrosso for whatever reason this KubernetesTCPRemoteCommitProviderTest.addresses:165 expected:<2> but was:<0> test is unstable ...

Could you please take a look at it?

[ilgrosso]

@ilgrosso for whatever reason this KubernetesTCPRemoteCommitProviderTest.addresses:165 expected:<2> but was:<0> test is unstable ...

@solomax I took a quick look but could not figure out how that test might be unstable; actually, I was able to track down a single failure by looking at GitHub workflow runs.

Since it is a mocked test, I can only suppose that there could be some dependency upgrade in this PR which is interfering somehow.

[solomax]

Another update: HerbDB seems to be broken in master so it should be fixed separately :)

[solomax]

Another update: HerbDB seems to be broken in master so it should be fixed separately :)

@eolivelli maybe you would like to fix HerbDB support? ;)

[rmannibucau]

is this by spec? how do I know my delete was a noop as a caller now?

[rzo1]

Good catch - I think as written this is too broad.

This was added for two TCK tests (persistMX1Test2, cascadeAllMX1Test2) where a parent and child are removed in the same tx and the FK has a DB-level ON DELETE CASCADE. OpenJPA flushes the parent DELETE first, the DB cascades and removes the child row, and then OpenJPA's own DELETE of the child returns count == 0. Before this change that 0 always produced a spurious OptimisticException, because RowManagerImpl.getRow() sets a non-null failed object on essentially every managed row; so the old code couldn't tell "benign already-gone" from a real conflict either.

But you're right on the spec point (just checked again): count == 0 on a DELETE also legitimately means an optimistic-lock failure. For a @Version entity the SQL is DELETE .. WHERE id = ? AND version = ?, and 0 rows there is a concurrent update/delete that the spec says must surface as OptimisticLockException.

As currently written we'd swallow that, and the caller loses the only signal we had - exactly your "how do I know it was a no-op" concern. The TCK entities here are non-versioned, which is why it slipped through.

We could gate/check for an entity having no version field; tolerate count == 0 on DELETE only when there's no @Version (so there's no optimistic signal to honor), and otherwise keep the existing OptimisticException path for flushAndUpdate, flushSingleRow, and checkUpdateCount case 0.

[rmannibucau]

wonder if these null cases shouldn't throw, means enhancement is broken or setup (agent) is broken no?

[rzo1]

I would already have thrown ClassCastException at the (PersistenceCapable) o cast on line 465 if there is broken enhancement, no?

[rmannibucau]

same, to review if we don't loop a case there

[solomax]

I need help on this one as well :( (I'm totally lost in the code :(( )

this simple JPQL is transformed to following SQL: SELECT ROUND(CAST((CAST(SUM(t0.age) AS NUMERIC) / ?) AS DOUBLE), CAST(? AS DOUBLE)) FROM CompUser t0

at hsqldb

Which leads to

 Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: incompatible data type in operation in statement [SELECT ROUND(CAST((CAST(SUM(t0.age) AS NUMERIC) / ?) AS DOUBLE), CAST(? AS DOUBLE)) FROM CompUser t0] {SELECT ROUND(CAST((CAST(SUM(t0.age) AS NUMERIC) / ?) AS DOUBLE), CAST(? AS DOUBLE)) FROM CompUser t0} [code=-5563, state=42563]

I believe this is because second argument of ROUND should be INT (not DOUBLE)

how it can be fixed? :)

[rzo1]

Had a look at this one. I debugged it by replaying the generated SQL against a plain in-memory HSQLDB (jdbc:hsqldb:mem:) from a small standalone main() and comparing the error codes / results of different cast variations:

ROUND(..., CAST(? AS DOUBLE))   -> -5563 incompatible data type in operation
ROUND(..., CAST(? AS INTEGER))  -> works
ROUND(..., ?)                   -> works

So your guess is exactly right: HSQLDB insists on an integer scale for ROUND.

Where the CAST(? AS DOUBLE) comes from: HSQLDictionary sets requiresCastForMathFunctions = true, and DBDictionary.mathFunction() then promotes both arguments to the common numeric type (Filters.promote(Double, Integer) → double) — even though the kernel (Math constructor) had already typed the scale argument as Integer.

There is a second problem hiding behind it though: with only the scale cast fixed, the query returns 21.0 instead of the expected 21.857. The inner CAST(SUM(t0.age) AS NUMERIC) is the culprit — in HSQLDB, NUMERIC without precision/scale means scale 0, and the bare ? divisor infers its type from the other operand, so 7.0 is truncated to 7 and the division is evaluated with scale 0:

CAST(SUM(age) AS NUMERIC) / ?          -> 21
CAST(SUM(age) AS NUMERIC(40,20)) / ?   -> 21.857142857142857...

Maybe something like this works — in DBDictionary.mathFunction(), keep a separate cast type for the ROUND scale:

        int type = 0;
        int rtype = 0;
        if (requiresCastForMathFunctions && (lc != rc
            || (lhs.isConstant() || rhs.isConstant()))) {
            Class c = Filters.promote(lc, rc);
            type = getJDBCType(JavaTypes.getTypeCode(c), false);
            rtype = type;
            if (type != Types.VARBINARY && type != Types.BLOB) {
                castlhs = (lhs.isConstant() && rhs.isConstant()) || lc != c;
                castrhs = (lhs.isConstant() && rhs.isConstant()) || rc != c;
            }
        }

        boolean mod = "MOD".equals(op);
        boolean power = "POWER".equals(op);
        boolean round = "ROUND".equals(op);
        if (round && castrhs) {
            // the second argument of ROUND is an integer scale; casting it to
            // the promoted numeric type of the first argument produces e.g.
            // ROUND(x, CAST(? AS DOUBLE)), which databases reject
            rtype = Types.INTEGER;
        }

(and appendCast(buf, rhs, rtype) instead of type at the bottom of the method)

plus in HSQLDictionary (appendLength is only ever called from appendCast, so this does not affect DDL):

    @Override
    protected void appendLength(SQLBuffer buf, int type) {
        // HSQLDB gives CAST(x AS NUMERIC) without precision and scale a scale
        // of 0 and silently truncates all fractional digits, so casts written
        // for requiresCastForMathFunctions / requiresCastForComparisons would
        // corrupt decimal values; always spell out precision and scale.
        // 40/20 is an arbitrary but generous choice - adjust if needed
        if (type == Types.NUMERIC || type == Types.DECIMAL) {
            buf.append("(40,20)");
        } else {
            super.appendLength(buf, type);
        }
    }

With both changes TestEJBQLFunction on HSQLDB is down to a single remaining error (testExtractWEEK — HSQLDB does not know EXTRACT(WEEK ...), it wants WEEK_OF_YEAR, that one needs a separate field-name mapping), and Derby stays green (64/64).

[solomax]

Another weird case for HSSQL is this one and alike

- 0.4 part seems to be completely ignored

Object result0 = em.createQuery("SELECT 24 - 0.4 AS s, COUNT(c) FROM CompUser c").getSingleResult();

Results 24, 6 ....

how this can be debugged? :))

[rzo1]

This one turned out to have the same root cause as the wrong ROUND value (see the other comment). Debugged it the same way — standalone main() against an in-memory HSQLDB, comparing variations:

SELECT 24 - 0.4                                       -> 23.6   (inline literals)
SELECT ? - ?                    (24, 0.4)             -> 24  (!)
SELECT CAST(? AS NUMERIC) - ?   (24, 0.4)             -> 24  (!)   <- this is what we generate
SELECT CAST(? AS NUMERIC(30,10)) - CAST(? AS ...)     -> 23.6

OpenJPA renders JPQL literals as prepared-statement parameters, so the inline form never reaches the DB. For 24 - 0.4, mathFunction() promotes Long/BigDecimal to BigDecimal and emits CAST(? AS NUMERIC) - ?. On HSQLDB that goes wrong twice:

1. CAST(? AS NUMERIC) without precision/scale is NUMERIC with scale 0 → 24 stays 24, and
2. the bare ? infers its type from the other operand — also NUMERIC scale 0 — so 0.4 is truncated to 0.

Result: 24 - 0 = 24. So the - 0.4 isn't ignored, it's truncated :)

The appendLength override from the ROUND comment fixes this one as well: the cast becomes CAST(? AS NUMERIC(40,20)) - ?, the bare parameter inherits the scaled type, and the result is 23.6. The CEILING/FLOOR tests (SUM(c.age) + 0.4 etc.) are covered by the same fix.

Re "how can this be debugged": I set openjpa.Log=SQL=TRACE to capture the generated SQL, then replayed it 1:1 in a small main() against jdbc:hsqldb:mem: and mutated the casts until the error code / wrong value pointed at the responsible piece.