April 10, 2024 Lakehouse Studio Release Notes

In this release (Release 2024.04.10), we have introduced a series of new features, enhancements, and fixes. Additionally, we have made some updates to the product's behavior. Please note that these updates will be gradually rolled out to the following regions:

Alibaba Cloud Shanghai Region
Tencent Cloud Shanghai Region
Alibaba Cloud Singapore Region
Tencent Cloud Beijing Region

Note: The updates will be completed within one to two weeks from the release date, depending on your region.

New Features

Account Registration: Now supports account and user registration using international phone numbers (only for Alibaba Cloud Singapore Region).
Workspace: Added a feature to modify the encryption properties of a workspace.
Billing: The billing page now supports filtering by "workspace," allowing users to view bills by workspace (preview version only).
Data Synchronization: Real-time synchronization now supports adding extended fields on the target side to record metadata information such as the source server, database name, and table name.
Data Synchronization: Real-time synchronization adds advanced naming methods, supporting mirror naming and custom naming.
Data Synchronization: Real-time synchronization now supports setting a composite primary key on the target side to resolve primary key conflicts in tables under multiple databases on the source side.
Data Synchronization: Real-time synchronization adds support for SQL Server data sources, including full database mirror synchronization and multi-table merge synchronization modes.
Data Synchronization: Offline synchronization adds support for REST API data sources.
Data Synchronization: Hive data sources now support connection via SSH Tunnel in addition to direct connection.
Task Development: Added a global search feature to search code in all tasks using a given keyword.
Task Development: Added a local code formatting feature, allowing users to format selected local code.
Task Development: Added a "Copy Downstream Links" feature, supporting the copying of the current task and all downstream link tasks in the current workspace.
Task Operations: Added a "Decommission (including downstream)" feature, supporting batch decommissioning of the current task and all downstream tasks.
Task Operations: Added an advanced rerun feature, allowing users to choose to rerun using historical or the latest version of code and configuration.
Compute Cluster: Added load monitoring, job queue monitoring, and job history list for compute clusters to better understand the operation of compute clusters.
Internationalization: Help documentation is now available in English.

Optimizations

Task Development: Optimized code formatting style to avoid unnecessary line breaks and improve code readability.
Task Development: Optimized the display of numeric types in query results, removing the default use of scientific notation for clearer readability.
Task Development: Optimized the display and intelligent parsing of scheduling dependency information, supporting the exposure of task scheduling time information and providing parsing feedback.
Data Quality: Rerun task instances trigger rerun of associated quality rules using the latest quality rule configuration.
Task Operations: Added audit information for supplementary task operations.
Monitoring and Alerts: Task instance failure now supports filtering by task type, distinguishing between periodic scheduling task types, supplementary task types, etc.

Bug Fixes

Data Synchronization: Fixed an issue where the full synchronization row count metric was inaccurate in real-time synchronization.
Task Operations: Fixed a rare issue where the task status remained as running after completion.
Data Security: Fixed an issue where users with the workspace_dev role encountered a lack of read metadata permissions when querying table data.

Known Limitations

Task Development: The "Copy Downstream Links" feature only copies downstream nodes located within the current workspace.
Offline Synchronization: StarRocks and Greenplum offline synchronization do not support updating writes via composite primary keys.