[jdbc-v2] `DataLakeCatalog` is not present in `getSchemas()` result

[chernser]

Description

Databases with engine DataLakeCatalog are not visible when requested via DataBaseMetadata because driver uses system.databases to fetch this information.

Workaround: show_data_lake_catalogs_in_system_tables=1 settings in session or profile.

Note

As SHOW DATABASES return better information - JDBC Driver should make it at least an option.

DBeaver and Metabase

When workaround applied these products getting exception trying to read tables information.
See https://clickhouse.com/docs/operations/settings/settings#database_datalake_require_metadata_access

2026-07-02T21:19:42+02:00 ERROR metabase.sync.fetch-metadata Error while fetching metadata with 'db-metadata',java.sql.SQLException: Code: 736. DB::Exception: Received error 736 while fetching table metadata for existing table 'product_analyst_team.vw_ap_integrations_2026'. If you want this error to be ignored, use database_datalake_require_metadata_access=0. Error: Code: 736. DB::Exception: Cannot read table `sandbox.product_analyst_team.vw_ap_integrations_2026` because it doesn't have storage location. It means that it's not a DeltaLake table, and it's unreadable with Unity catalog in ClickHouse. (...,	at com.clickhouse.jdbc.internal.ExceptionUtils.toSqlState(ExceptionUtils.java:68),	at com.clickhouse.jdbc.internal.ExceptionUtils.toSqlState(ExceptionUtils.java:43),	at com.clickhouse.jdbc.StatementImpl.executeQueryImpl(StatementImpl.java:196),	at com.clickhouse.jdbc.PreparedStatementImpl.executeQuery(PreparedStatementImpl.java:138),	at com.clickhouse.jdbc.metadata.DatabaseMetaDataImpl.getTables(DatabaseMetaDataImpl.java:998),	at com.mchange.v2.c3p0.impl.NewProxyDatabaseMetaData.getTables(NewProxyDatabaseMetaData.java:4174),	at metabase.driver.clickhouse_introspection$get_tables_from_metadata.invokeStatic(clickhouse_introspection.clj:109),	at metabase.driver.clickhouse_introspection$get_tables_from_metadata.invoke(clickhouse_introspection.clj:107),	at metabase.driver.clickhouse_introspection$get_all_tables_in_all_dbs$fn__173063.invoke(clickhouse_introspection.clj:128),	at metabase.driver.clickhouse$fn__173529$fn__173531.invoke(clickhouse.clj:102),	at metabase.driver.sql_jdbc.execute$do_with_resolved_connection.invokeStatic(execute.clj:366),	at metabase.driver.sql_jdbc.execute$do_with_resolved_connection.invoke(execute.clj:338),	at metabase.driver.clickhouse$fn__173529.invokeStatic(clickhouse.clj:79),	at metabase.driver.clickhouse$fn__173529.invoke(clickhouse.clj:77),	at clojure.lang.MultiFn.invoke(MultiFn.java:244),	at metabase.driver.clickhouse_introspection$get_all_tables_in_all_dbs.invokeStatic(clickhouse_introspection.clj:125),	at metabase.driver.clickhouse_introspection$get_all_tables_in_all_dbs.invoke(clickhouse_introspection.clj:119),	at metabase.driver.clickhouse_introspection$fn__173076.invokeStatic(clickhouse_introspection.clj:170),	at metabase.driver.clickhouse_introspection$fn__173076.invoke(clickhouse_introspection.clj:166)
2026-07-02T21:19:42+02:00 WARN metabase.sync.util Error in sync step Sync metadata for clickhouse Database 43 ''test-clickhouse-cloud'',java.sql.SQLException: Code: 736. DB::Exception: Received error 736 while fetching table metadata for existing table 'product_analyst_team.vw_ap_integrations_2026'. If you want this error to be ignored, use database_datalake_require_metadata_access=0. Error: Code: 736. DB::Exception: Cannot read table `sandbox.product_analyst_team.vw_ap_integrations_2026` because it doesn't have storage location. It means that it's not a DeltaLake table, and it's unreadable with Unity catalog in ClickHouse. (...,	at com.clickhouse.jdbc.internal.ExceptionUtils.toSqlState(ExceptionUtils.java:68),	at com.clickhouse.jdbc.internal.ExceptionUtils.toSqlState(ExceptionUtils.java:43),	at com.clickhouse.jdbc.StatementImpl.executeQueryImpl(StatementImpl.java:196),	at com.clickhouse.jdbc.PreparedStatementImpl.executeQuery(PreparedStatementImpl.java:138),	at com.clickhouse.jdbc.metadata.DatabaseMetaDataImpl.getTables(DatabaseMetaDataImpl.java:998),	at com.mchange.v2.c3p0.impl.NewProxyDatabaseMetaData.getTables(NewProxyDatabaseMetaData.java:4174),	at metabase.driver.clickhouse_introspection$get_tables_from_metadata.invokeStatic(clickhouse_introspection.clj:109),	at metabase.driver.clickhouse_introspection$get_tables_from_metadata.invoke(clickhouse_introspection.clj:107),	at metabase.driver.clickhouse_introspection$get_all_tables_in_all_dbs$fn__173063.invoke(clickhouse_introspection.clj:128),	at metabase.driver.clickhouse$fn__173529$fn__173531.invoke(clickhouse.clj:102),	at metabase.driver.sql_jdbc.execute$do_with_resolved_connection.invokeStatic(execute.clj:366),	at metabase.driver.sql_jdbc.execute$do_with_resolved_connection.invoke(execute.clj:338),	at metabase.driver.clickhouse$fn__173529.invokeStatic(clickhouse.clj:79),	at metabase.driver.clickhouse$fn__173529.invoke(clickhouse.clj:77),	at clojure.lang.MultiFn.invoke(MultiFn.java:244),	at metabase.driver.clickhouse_introspection$get_all_tables_in_all_dbs.invokeStatic(clickhouse_introspection.clj:125),	at metabase.driver.clickhouse_introspection$get_all_tables_in_all_dbs.invoke(clickhouse_introspection.clj:119),	at metabase.driver.clickhouse_introspection$fn__173076.invokeStatic(clickhouse_introspection.clj:170),	at metabase.driver.clickhouse_introspection$fn__173076.invoke(clickhouse_introspection.clj:166)
20

[chernser]

/triage

[github-actions]

Triage Report

Effort to Fix: Small
Type: Bug
Affected Module: jdbc-v2

Summary

DatabaseMetaDataImpl.getSchemas() (and getSchemas(catalog, schemaPattern)) query system.databases, which excludes DataLakeCatalog databases by default in ClickHouse. As a secondary problem, when the server-side workaround (show_data_lake_catalogs_in_system_tables=1) is applied, getTables() fails with ClickHouse error 736 for DataLake tables that lack a storage location, crashing tools like Metabase and DBeaver.

Recommended Labels & Priority

• Labels: bug, jdbc-v2, area:jdbc-metadata

Root Cause (Research)

Problem 1 — getSchemas() missing DataLakeCatalog databases

Both getSchemas() overloads in jdbc-v2/src/main/java/com/clickhouse/jdbc/metadata/DatabaseMetaDataImpl.java (lines 1023–1030 and 1770–1776) issue:

SELECT name AS TABLE_SCHEM, '' AS TABLE_CATALOG FROM system.databases ...

system.databases omits DataLakeCatalog-engine databases unless show_data_lake_catalogs_in_system_tables=1 is active in the session or server profile.

Switching to SHOW DATABASES (which returns all databases including DataLakeCatalog ones) is the suggested fix. The driver's SQL parser already handles SHOW DATABASES (ClickHouseParser.g4 line 936).

Problem 2 — getTables() throws error 736 once workaround is applied

getTables() at line 997–1001 runs:

SELECT ... FROM system.tables t JOIN system.databases d ON ... WHERE ...

When show_data_lake_catalogs_in_system_tables=1 is active, DataLake tables appear in system.tables, but ClickHouse raises error 736 when their metadata is fetched (they have no storage location). The server setting database_datalake_require_metadata_access=0 suppresses this error. The driver should either pass this setting or filter out DataLakeCatalog tables from the JOIN unless explicitly requested.

Missing Information / Questions for User

1. Which ClickHouse server version introduced DataLakeCatalog support? (Needed to know if a version guard is required.)
2. Which jdbc-v2 version is in use? (Confirm it maps to the DatabaseMetaDataImpl code above.)
3. Should DataLakeCatalog databases/tables be included by default (and the driver silently sets the required server settings), or should this be opt-in via a driver property?
4. Does SHOW DATABASES return enough columns / the right format to replace the system.databases query in the existing result-set shape expected by callers?

Tests to Add

Test 1: getSchemas() includes DataLakeCatalog database

// Requires a ClickHouse instance with a DataLakeCatalog database (e.g., Unity Catalog)
// and show_data_lake_catalogs_in_system_tables=1 (or driver-level fix applied)
@Test
public void testGetSchemasIncludesDataLakeCatalog() throws Exception {
    try (Connection conn = getJdbcConnection()) {
        DatabaseMetaData dbmd = conn.getMetaData();
        ResultSet rs = dbmd.getSchemas();
        List<String> schemas = new ArrayList<>();
        while (rs.next()) {
            schemas.add(rs.getString("TABLE_SCHEM"));
        }
        assertTrue("DataLakeCatalog schema must be visible via getSchemas()",
                schemas.contains("my_unity_catalog_db"));
    }
}

Test 2: getTables() does not throw on DataLakeCatalog tables with missing storage

// When DataLakeCatalog databases are visible, getTables() must not propagate
// ClickHouse error 736 for tables without a storage location.
@Test
public void testGetTablesDataLakeCatalogNoError() throws Exception {
    try (Connection conn = getJdbcConnection()) {
        DatabaseMetaData dbmd = conn.getMetaData();
        // Should complete without throwing SQLException with error code 736
        ResultSet rs = dbmd.getTables("", "my_unity_catalog_db", "%", new String[]{"TABLE"});
        assertNotNull(rs);
    }
}