Task Parameters

Task parameters can be used for dynamic value replacement during the actual execution of the task.

Special note: This feature is currently in the grayscale stage, and the complete parameter functionality is only available to some customers. You can use the method at the end of the document to verify if it is supported.

Basic Concepts

Parameter Reference and Assignment

• In all task types, the ${} placeholder is used to mark the position as a variable parameter that needs to be replaced. The replacement only replaces the value of the corresponding placeholder.

  • Assume the value of the dt parameter is 2023-09-22
  • The code is written as follows: where pt=${dt}, the code will be replaced with where pt=2023-09-22
  • The code is written as follows: where pt='${dt}', the code will be replaced with where pt='2023-09-22'

• Assigning values to user-defined parameters can be done in two ways: constants and variables

  • Constant: Direct input, no decoration needed
  • Variable: Includes system built-in parameters and system built-in time functions, directly use the name

Parameter Usage Example

Taking a Lakehouse SQL task as an example, write the following code in the data development interface:

SELECT
  table1.*
FROM
  table1
WHERE
  city = '${city}'
  AND t = '${time}'
  AND bizdate = '${lastDay}';

In the above code example, the system will recognize three parameters: city, lastDay, and format. During actual execution, values can be assigned to them, for example:

Assuming the task's trigger execution time is 2023-09-20 18:00:00, after assigning and replacing the parameters, the following code will be executed:

SELECT
  table1.*
FROM
  table1
WHERE
  city = 'Shanghai'
  AND t = '2023-09-20 18:00:00'
  AND bizdate = '2023-09-19';

Supported Built-in Parameters

The system supports the following built-in parameters:

Built-in system parameters can be directly used when assigning values to user-defined parameters. As shown below, in the code, you can directly write ${param} to reference the parameter. When assigning a value to param, you can use sys_plan_day as the right value.

Preview

Supported Built-in Time Expressions

The system supports built-in time expressions, primarily various time processing functions, such as the commonly used yyyyMMdd combinations. Time function expressions refer to the ISO-8601 standard and are case-sensitive.

Time Expressions

Usage: $[Time Expression]

Examples of the above combinations: $[yyyy-MM-dd], $[yyyy-MM-dd HH:mm:ss], $[yyyy-MM-dd HH:mm:ss.SSSZZ], $[yyyyMMddHHmmss], $[HHmmss], $[MM], etc. Each element is the smallest unit, combined in the corresponding order, such as direct concatenation, separated by -, separated by /, etc.

Time Expression Increments and Decrements

To meet the needs of increasing or decreasing the corresponding time, the system supports time increments and decrements, defined as follows:

Usage: $[Time Expression, Increment/Decrement]

Similar to the usage of time expressions, a second parameter needs to be passed in to indicate the amount and unit to be increased or decreased, separated by a comma in English after the expression. For example: $['yyyy-MM-dd HH','-1d'] means subtract one day, $['yyyy-MM-dd HH','-1h'] means subtract one hour, $['yyyy-MM-dd HH','1h'] means add one hour, and so on.

Supported Built-in Time Functions

In addition to time expressions, the system also supports complex calculations based on time, known as built-in time functions. The list is as follows:

Verify if full parameter functionality is supported

As mentioned above, task parameters are open in grayscale, you can use the following methods to verify if they are supported:

1. Write SQL

SELECT '${lastDay}';

2. Execute the code to assign the value to lastDay as add_days('yyyy-MM-dd', -1)

3. Check if the parameter replacement and task execution return results are normal. If normal (assuming the current date is 2023-11-12, the return value is 2023-11-11), it indicates that it is within the grayscale range and supports the full parameter functionality; otherwise, it indicates that it is not supported.