Release Notes 2026-08-21

This release (Release 2026.08.21) introduces a range of new features, enhancements, performance improvements, and bug fixes. The overall update focuses on data version management, open table format writes, AI unstructured data processing, incremental computation, SQL compatibility, and high-concurrency query stability.

Please note that these updates will be rolled out in stages to the regions listed below. The rollout will complete within one to two weeks of the release date, depending on your region.

• Mainland China Regions

  • Alibaba Cloud (Shanghai)
  • Tencent Cloud (Shanghai / Beijing / Guangzhou)
  • AWS (Beijing)
  • Alibaba Cloud (Hangzhou)

• International Regions

  • Alibaba Cloud (Singapore)
  • AWS (Singapore)

New Features

Managed Iceberg

Managed Iceberg tables now support a complete write path. The engine can directly perform table creation, writes, updates, deletes, partition management, and Compaction on Iceberg tables, and is interoperable with the Spark write path.

Supported capabilities include:

• Creating Managed Iceberg tables using USING ICEBERG
• DML operations including INSERT, UPDATE, and DELETE
• Updates, deletes, and DROP PARTITION on partitioned tables
• Running table-level Compaction via OPTIMIZE
• Automatic cleanup of Managed Iceberg orphan files

Example:

CREATE TABLE iceberg_orders ( id BIGINT, amount DECIMAL(18,2), dt STRING ) USING ICEBERG PARTITIONED BY (dt); UPDATE iceberg_orders SET amount = 100 WHERE id = 1; DELETE FROM iceberg_orders WHERE dt = '2024-01-01'; OPTIMIZE iceberg_orders;

Iceberg Equality Delete

Iceberg Equality Delete has completed key pipeline gaps in this release, covering four areas: writes, replay, Compaction, and conflict handling:

• Deduplication: Equality delete files are deduplicated at the checkpoint snapshot layer to prevent the same delete record from being applied multiple times.
• Replay: Equality delete files can now participate in FMDT (file metadata) replay, ensuring data consistency after crash recovery.
• Compaction: SQL Compaction now supports local replay and local rewrite, converting equality delete files into position deletes to reduce read amplification at query time.

Together, these three capabilities significantly improve consistency and read performance for Iceberg tables in row-level update/delete and multi-engine co-write scenarios.

Semantic View Query Interface

Semantic View has expanded from "definable" to "queryable", supporting direct SQL access to semantic views via semantic_view(), with metric, dimension granularity, and relationship validation during query planning.

Key enhancements include:

• Support for querying semantic views via semantic_view()
• Support for cross-semantic-view join queries
• Support for the FACTS keyword and inline subqueries: the FACTS clause allows declaring row-level fact columns so that queries can return detail data directly rather than aggregated values; inline SELECT subqueries can now be used as logical table sources within view definitions to embed pre-filtering logic without creating additional views.
• Support for implicit data filtering (FILTER clause): declare a FILTER condition in the view definition and it is automatically applied before aggregation on every query, removing the need for users to manually add WHERE clauses. This is suitable for business metric views with a fixed data scope, such as counting only valid orders or showing only the current tenant's data.
• Semantic view dimension attributes visible in DESC: dimension annotations such as is_time, is_unique, and enum_values can now be viewed directly via DESC EXTENDED without needing to rely on the raw JSON format, making it easier to verify view definitions after table creation.
• Support for arithmetic metrics, window metrics, and multi-grain fan-out:
  • Arithmetic Metrics: Metric definitions can perform secondary calculations on aggregated results, for example SUM(amount)/COUNT(orders) for average price or SUM(profit)*100 for a percentage, without being limited to a single aggregation function.
  • Window Metrics: Window functions can be used in metric definitions, for example SUM(SUM(amount)) OVER (ORDER BY date) for cumulative values or SUM(SUM(amount)) OVER (PARTITION BY region) for grouped totals. This is suited for trend analysis and ranking scenarios.
  • Multi-Grain Fan-out: When metrics in a semantic view come from fact tables at different granularities (such as a total amount from an orders table and a line-item amount from a details table), the system first completes aggregation at each table's own granularity and then merges the results via FULL OUTER JOIN, avoiding duplicate calculations caused by row multiplication in multi-table joins.

AI Function Enhancements

AI Functions continue to expand unstructured data processing capabilities, covering multimodal inputs including PDF, images, and audio, with improved fault tolerance and parallel execution control for AI function requests.

Function	Description
AI_CONTEXT_LENGTH	Helps users estimate token consumption before batch calls to avoid exceeding budget
PDF_TO_JSON	Parses a PDF document into a structured JSON string containing document metadata, plain text per page, and extracted layout structure (blocks, lines, character coordinates, etc.). Suited for scenarios requiring programmatic processing of PDF content such as extracting tables, paragraphs, and coordinate information. This function does not interact with a large model and does not consume tokens.
PDF_TO_MD	Converts a PDF document to Markdown text. Suited for feeding PDF content into large model processing, knowledge base construction, or document summarization. This function does not interact with a large model and does not consume tokens.

Incremental Computation Enhancements

Dynamic Table and incremental computation in this release focus on three areas: expanded unique key detection scope, State Table cost control, and more granular refresh strategies.

• Support for declaring equivalent primary keys via EQUIV to enable incremental refresh for external tables without a PK; ARRAY/STRUCT/BINARY columns and partition keys can participate in unique key generation.
• The mode(expr, true) aggregate function now supports incremental maintenance.
• State Table strategy optimization: shared State Tables are exempt from count limits, and deep Join subtrees now use interval-based checkpointing to reduce total State Table overhead.
• A new switch provides precise control over full refresh trigger conditions to reduce unnecessary full recomputation.

External Table and Object Storage Extensions

External table and object storage access capabilities continue to expand in this release:

• Azure connections have a new access policy framework and authentication type
• GCS / Azure Blob writes now have overwrite protection: when concurrent writes target the same path, the server atomically rejects duplicate writes rather than silently overwriting, eliminating data overwrite risk in distributed scenarios and aligning with the consistency guarantees of OSS/S3.
• Object storage access now supports STS temporary credentials: when executors access storage such as OSS/S3, short-lived STS tokens scoped to specific buckets can be issued, improving storage access security in multi-tenant scenarios.
• Support for reading gzip / bzip2 compressed Hive text files
• Inverted index enhancements (aligned with Elasticsearch style): string columns can have both an exact index and a full-text index built simultaneously, with queries automatically routed based on condition type.
• Under an external schema (ext_hive) connected via Hive Metastore (HMS), you can now use CREATE EXTERNAL TABLE ... USING paimon to create external tables in Paimon format directly. Lakehouse automatically initializes the Paimon metadata files and registers the table information in HMS, allowing other engines such as Spark and Flink to access the same Paimon data through the Hive Catalog.

SQL Compatibility and Functions

Spark Compatibility

This release enhances Spark SQL compatibility, covering PIVOT / UNPIVOT, MERGE, type casting, and legacy behavior switches.

• Support for multi-column PIVOT / UNPIVOT
• MERGE INTO supports Spark-compatible syntax extensions
• Support for Spark overflow semantics when casting float / double to int
• Support for Spark legacy type conversion mode

Example:

MERGE INTO target t USING source s ON t.id = s.id WHEN MATCHED AND s.op = 'D' THEN DELETE WHEN MATCHED THEN UPDATE SET * WHEN NOT MATCHED THEN INSERT *; SET cz.sql.cast.double.as.int.clamp = true; SET cz.sql.spark_legacy.type_conversion = true;

Doris Compatibility

New Doris compatibility, adapting array subscripts, JSON functions, and MySQL-style semantics.

• Support for 1-based subscript indexing for one-dimensional arrays
• New ELEMENT_AT_DORIS function
• Setting cz.sql.compatible.target = doris automatically enables related compatibility behaviors such as cast.string.to.json.as.parse and mysql.having

Example:

SET cz.sql.array.subscript.one.based = true; SELECT ELEMENT_AT_DORIS(arr, 1) FROM t; SET cz.sql.compatible.target = doris;

Presto and SQL Server Compatibility

• Support for Presto decimal division rules
• New CHARINDEX function as a SQL Server-compatible alias for locate
• Parser support for SQL Server-style comparison operators !> and !<

New and Enhanced Functions

Function / Syntax	Description
TRY_URL_DECODE	Returns NULL on URL decode failure to avoid query failures
str_to_map	Duplicate key behavior changed to last-win, aligned with Spark behavior
substr / TIMESTAMPDIFF	Parser enhanced for compatibility with these function calls

Developer Tools and Management Enhancements

Security and Permissions

• Added Ranger authorization plugin capabilities, supporting Java-side Ranger authorization endpoint and plugin provider integration
• PIPE DDL in dedicated environments can now be automatically routed to serverless

Scheduling and Diagnostics Enhancements

DAG speculative execution has been further enhanced to identify slow tasks earlier and improve handling of long-tail tasks.

• Earlier lateral straggler detection — speculative execution is no longer deferred until a task is close to timing out
• Support for configuring the maximum number of tasks that can run speculatively at the same time
• Job Profile now records the reason for each rerun, such as straggler or failure
• A task can trigger speculative execution again after a previous speculative attempt fails or is killed
• Fixed a scheduling deadlock caused by NoOp task delays

Infrastructure Enhancements

• Introduced a NuRaft-based Raft KVStore module skeleton supporting CREATE, UPDATE, and REMOVE operations
• Full refresh reasons are now recorded in DESC HISTORY, with improved error message readability
• Pipe event-time states now support persistence so that state is not lost after a restart
• Arrow upgraded to 25.0.0.cz.1

Bug Fixes

Optimizer and Planner

• Fixed an issue where rules could enter an infinite loop when UNION ALL was followed by JOIN
• Fixed an issue where high-confidence NDV incorrectly inherited after LEFT ANTI JOIN caused underestimated row counts
• Fixed row count estimation for LEFT SEMI / LEFT ANTI in scenarios without equi-join keys
• Fixed incorrect splitting of grouping sets with DISTINCT
• Fixed incorrect pushdown of ORDER BY when output columns contained window functions
• Fixed a condition containing RAISE_ERROR blocking Join predicate pushdown
• Fixed chain-top Window nodes with different partition keys not triggering incremental rules
• Fixed a boundary issue with decimal zero literals in aggregation simplification

Types and Parser

• Fixed truncation of the millisecond portion when casting CAST(ts AS BIGINT)
• Fixed double-quoted column aliases being incorrectly recognized as typeConstructors
• Fixed a crash in UNION ALL caused by inconsistent DECIMAL nullable types inside structs
• Fixed backtick double-backtick escape parsing
• Fixed str_to_map duplicate key semantics, now adjusted to last-win

Storage and Transactions

• Fixed delta file name conflicts during concurrent upserts; retries are now supported
• Fixed ORC reads and writes not using the SQL session time zone
• Fixed time partition transforms not aligned to UTC semantics
• Fixed incremental replay potentially skipping some data after overwrite manifest splits

Metadata and Cache

• Fixed loss of the original table instance in Shared Table after a cross-RPC call
• Fixed inconsistent inner and outer spec IDs after Alter Table Spec retries
• Fixed incorrect cache hits for multi-datasource table partitions
• Fixed an AssertionError during updates caused by a stale snapshot in DeltaLogCache
• Fixed an infinite loop in Dynamic Table refresh when the interval was set to 0
• Fixed table cache not refreshing correctly after expiration

Other Fixes

• Fixed incorrect OOM job memory usage threshold calculations causing false positives
• Fixed a potential hang in the AWS SDK in certain scenarios
• Fixed native-layer OOM caused by COS / OFS FileSystem instances not being reused
• Changed fork + exec in multi-threaded environments to posix_spawn to reduce deadlock risk
• Fixed MySQL kill query control statements being incorrectly parsed as SQL