CREATE EXTERNAL VOLUME

Mounts external object storage (Alibaba Cloud OSS, Tencent Cloud COS, Amazon S3) and creates an External Volume object in Lakehouse.

Prerequisites

Before creating an External Volume, you need to create the corresponding Storage Connection:

-- Create an Alibaba Cloud OSS storage connection
CREATE STORAGE CONNECTION IF NOT EXISTS oss_conn
    TYPE oss
    ENDPOINT = 'oss-cn-hangzhou.aliyuncs.com'
    ACCESS_ID = 'your_access_key_id'
    ACCESS_KEY = 'your_access_key_secret';

Syntax

CREATE EXTERNAL VOLUME [IF NOT EXISTS] [schema_name.]<volume_name>
    LOCATION '<storage_url>'
    USING CONNECTION <connection_name>
    DIRECTORY = (
        enable = { true | false },
        auto_refresh = { true | false }
    )
    RECURSIVE = { true | false };

Parameters

ParameterDescription
IF NOT EXISTSIf the Volume already exists, skip without error
schema_nameName of the owning schema; current schema is used if omitted
volume_nameVolume name, must be unique within the same schema
LOCATIONObject storage path, format: oss://bucket_name/path/, cos://bucket_name/path/, s3://bucket_name/path/
USING CONNECTIONThe Storage Connection name to reference
DIRECTORY.enableWhether to enable the directory feature; recommended to set to true
DIRECTORY.auto_refreshWhether to automatically refresh file metadata
RECURSIVEWhether to recursively scan subdirectories

Examples

  1. Mount an Alibaba Cloud OSS bucket:
CREATE EXTERNAL VOLUME my_oss_vol
    LOCATION 'oss://mcp-data-hangzhou/test/'
    USING CONNECTION oss_conn
    DIRECTORY = (enable = true, auto_refresh = true)
    RECURSIVE = true;
  1. Mount a Tencent Cloud COS bucket:
CREATE EXTERNAL VOLUME my_cos_vol
    LOCATION 'cos://my-bucket-1234567890/data/'
    USING CONNECTION cos_conn
    DIRECTORY = (enable = true, auto_refresh = true)
    RECURSIVE = true;
  1. Mount an Amazon S3 bucket:
CREATE EXTERNAL VOLUME my_s3_vol
    LOCATION 's3://my-s3-bucket/data/'
    USING CONNECTION s3_conn
    DIRECTORY = (enable = true, auto_refresh = true)
    RECURSIVE = true;
  1. Create under a specific schema with IF NOT EXISTS:
CREATE EXTERNAL VOLUME IF NOT EXISTS my_schema.my_oss_vol
    LOCATION 'oss://mcp-data-hangzhou/test/'
    USING CONNECTION oss_conn
    DIRECTORY = (enable = true, auto_refresh = true)
    RECURSIVE = true;

Notes

  • External Volumes only store path metadata; actual data is stored in the external cloud storage, so there are no additional storage costs on the Lakehouse side
  • Cross-cloud creation is not supported: Alibaba Cloud instances can only create OSS Connections, Tencent Cloud instances can only create COS Connections, etc.
  • Deleting an External Volume does not delete the actual files in the external storage

Required Privileges

PrivilegeDescription
CREATE VOLUMECreate a Volume under the current schema