Using Explain to View Dynamic Table Refresh Mode

Use the explain command to check if incremental refresh (preview) is possible. You need to enable the switch by setting set cz.optimizer.explain.can.incrementalize=true; and execute it together in data development.

Syntax

set cz.optimizer.explain.can.incrementalize=true;
EXPLAIN REFRESH DYNAMIC TABLE dtname;

Usage Instructions

  • The explain command is usually used to view the SQL execution plan. When refreshing a dynamic table, adding explain can be used to view the execution plan of the dynamic table. At the same time, Lakehouse will output whether it is an incremental plan field. CanBeIncrementalized, if it is Yes, it means that the incremental plan is executed. If the output is No, it means it is not an incremental plan No because ...

Specific Case

set cz.optimizer.explain.can.incrementalize=true;
explain refresh dynamic table event_gettime;

Use the Show Command to View Dynamic Table Refresh Mode

Check by using show dynamic table refresh history. The refresh_mode field in the output can be used to see if it is an incremental refresh, and the stats field records how many rows were incrementally refreshed.

Syntax

 SHOW DYNAMIC TABLE REFRESH HISTORY [where <expr>] [LIMIT num];

Parameter Description

  • WHERE <expr>: (Optional) Allows users to filter based on the fields displayed by the SHOW JOBS command. Users can filter the results through expressions to more accurately find the required data.
  • LIMIT num: (Optional) Limits the number of job records returned, ranging from 1-10000.

Return Results

FieldDescription
workspace_nameWorkspace name
schema_nameSchema name
nameDynamic table name
virtual_clusterComputing cluster used
start_timeRefresh start time, timestamp type
end_timeRefresh end time, timestamp type
durationRefresh duration, interval type
stateJob status
refresh_triggerMANUAL (manually triggered by user calling refresh, including studio scheduled refresh) SYSTEM_SCHEDULED (scheduled refresh by lakehouse)
suspended_resonReserved field with no special meaning
refresh_modeNO_DATA FULL INCREMENTAL
error_messageInformation on refresh failure
source_tablesRecords the base table names used by the dynamic table
statsInformation such as the number of incremental refreshes
completion_targetReserved field with no special meaning
job_idJob ID, by clicking the job ID you can see the job profile