Using Explain to View Dynamic Table Refresh Mode
Use the explain command to check if incremental refresh (preview) is possible. You need to enable the switch by setting set cz.optimizer.explain.can.incrementalize=true;
and execute it together in data development.
Syntax
Usage Instructions
- The explain command is usually used to view the SQL execution plan. When refreshing a dynamic table, adding explain can be used to view the execution plan of the dynamic table. At the same time, Lakehouse will output whether it is an incremental plan field. CanBeIncrementalized, if it is Yes, it means that the incremental plan is executed. If the output is No, it means it is not an incremental plan No because ...
Specific Case
Use the Show Command to View Dynamic Table Refresh Mode
Check by using show dynamic table refresh history. The refresh_mode field in the output can be used to see if it is an incremental refresh, and the stats field records how many rows were incrementally refreshed.
Syntax
Parameter Description
WHERE <expr>
: (Optional) Allows users to filter based on the fields displayed by theSHOW JOBS
command. Users can filter the results through expressions to more accurately find the required data.LIMIT num
: (Optional) Limits the number of job records returned, ranging from 1-10000.
Return Results
Field | Description |
---|---|
workspace_name | Workspace name |
schema_name | Schema name |
name | Dynamic table name |
virtual_cluster | Computing cluster used |
start_time | Refresh start time, timestamp type |
end_time | Refresh end time, timestamp type |
duration | Refresh duration, interval type |
state | Job status |
refresh_trigger | MANUAL (manually triggered by user calling refresh, including studio scheduled refresh) SYSTEM_SCHEDULED (scheduled refresh by lakehouse) |
suspended_reson | Reserved field with no special meaning |
refresh_mode | NO_DATA FULL INCREMENTAL |
error_message | Information on refresh failure |
source_tables | Records the base table names used by the dynamic table |
stats | Information such as the number of incremental refreshes |
completion_target | Reserved field with no special meaning |
job_id | Job ID, by clicking the job ID you can see the job profile |