Skip to content

Snowflake: Support warehouse attribute on dynamic_table HCL block #3670

@hagai-arad

Description

@hagai-arad

Problem

When defining Snowflake dynamic tables in Atlas HCL, there is no way to specify which warehouse should be used for the dynamic table's refresh operations. The warehouse is always inherited from the connection URL's ?warehouse= query parameter.

In Snowflake, CREATE DYNAMIC TABLE supports a WAREHOUSE clause:

CREATE DYNAMIC TABLE my_table
  TARGET_LAG = '20 seconds'
  WAREHOUSE = WH_DTS_PROD
  AS SELECT ...

This is important because organizations typically use different warehouses for different workloads — e.g., a small warehouse for Atlas migrations/DDL operations and a larger compute-optimized warehouse for dynamic table refreshes.

Proposed Solution

Add a warehouse attribute to the dynamic_table block in the Snowflake HCL dialect:

dynamic_table "DIM_ACCOUNTS" {
  schema       = schema.PROCESSING
  target_lag   = "20 seconds"
  warehouse    = "WH_DTS_PROD"
  refresh_mode = INCREMENTAL
  as           = var.dim_accounts_query
}

This would generate:

CREATE DYNAMIC TABLE "PROCESSING"."DIM_ACCOUNTS"
  TARGET_LAG = '20 seconds'
  WAREHOUSE = "WH_DTS_PROD"
  REFRESH_MODE = INCREMENTAL
  AS ...

When warehouse is omitted, the current behavior (inheriting from the connection) should remain.

Current Workaround

Running ALTER DYNAMIC TABLE ... SET WAREHOUSE = '...' as a post-migration step, which is fragile and not declarative.

Additional Context

  • The Snowflake HCL reference documents dynamic_table with target_lag, refresh_mode, initialize, and as — but no warehouse.
  • Snowflake's CREATE DYNAMIC TABLE documentation lists WAREHOUSE as a required clause.
  • This also causes false positives in schema drift checks, since the live DB reflects the actual warehouse while Atlas normalizes to the connection warehouse.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions