July 24, 2024 Lakehouse Studio Release Notes

In this release (Release 2024.07.24), we have introduced a series of new features, enhancements, and fixes. Please note that these updates will be gradually rolled out to the following regions over a period of one to two weeks from the release date, depending on your region.

Tencent Cloud Beijing Region
Tencent Cloud Shanghai Region
Tencent Cloud Guangzhou Region
Alibaba Cloud Shanghai Region
Alibaba Cloud Singapore Region
Amazon Beijing Region
Amazon Cloud Singapore

Incompatible Changes

None

New Features

Global: Added an invitation-based trial feature, allowing quick account registration with an invitation code.
Global: Introduced dynamic table task nodes, with support for operations management, monitoring, and alerting capabilities. Dynamic tables support incremental data processing and have incremental refresh optimization capabilities.
Global: Introduced streaming SQL task nodes, with support for operations management, monitoring, and alerting capabilities. Streaming SQL is a new task scheduling method that can provide sub-second or even millisecond data processing latency.
Data Synchronization: Offline synchronization now automatically generates table creation statements, with options to create partitioned tables.
Data Synchronization: Kafka real-time synchronization tasks now support defining source fields as variables or using computed columns.
Task Development: Added canvas (Dag) mode for task groups, allowing users to create/add task groups and establish dependencies in canvas mode; also supports batch submission of task groups.
Task Development: Supports setting workspace names via parameters in the URL.
Monitoring and Alerts: Added support for selecting specific weeks and days of the month for non-disturbance times.
Data Catalog: Exposed table lifecycle management in data asset management, allowing users to visually modify lifecycle ranges.
Data Catalog: Supports filtering by workspace.

Optimizations and Improvements

Global: Adjusted the left menu structure to expose the product's internationalization capabilities for switching between Chinese and English.
Global: Optimized table pagination display issues, defaulting to the top area when clicking to paginate.
Data Synchronization: Added a "validation configuration" feature for batch configuration based on file merging in multi-table real-time synchronization tasks, supporting legality checks for invalid data sources and source table primary keys, and providing information on tables that failed validation.
Data Synchronization: Optimized the behavior inconsistency of various metric monitoring charts in multi-table real-time tasks after stopping and restarting.
Data Synchronization: Data synchronization tasks now automatically reference the source table's comments when creating tables on the target end.
Task Development: Exposed the comment field information in the data tree when clicking to display table information.
Task Development: Optimized the external parameter area to address issues where excessive scheduling parameters obscure a large part of the development interface.
Operations Center: Task instances in periodic task details now support batch operations.
Operations Center: Exposed the "runtime" field information on instance nodes in the instance details page.
Data Assets: Added space-level selection capability on the data search page.
Job History: Optimized the memory capability of job history filter conditions, retaining filter conditions after clicking refresh.
Monitoring and Alerts: Non-disturbance dates now support filtering by day/month.

Bug Fixes

Task Development: Fixed an issue where the full download of SQL query results would return an error when clicking the button.
Data Assets: Fixed an issue where searching for information_schema would sometimes return empty results.
Operations Center: Fixed an issue where the overall failure and individual instance status of backfill tasks were inaccurately displayed in some cases.
Security Center: Fixed an issue where disabled users could illegally join workspaces and grant roles.

Known Limitations

Task Development: Does not support adding real-time task types to task groups, including real-time synchronization, multi-table real-time synchronization, dynamic tables, and streaming SQL.