Using defer in dbt Cloud
Defer is a powerful feature that allows developers to only build and run and test models they've edited without having to first run and build all the models that come before them (upstream parents). This is powered by using a production manifest for comparison, and dbt will resolve the {{ ref() }}
function with upstream production artifacts.
By default, dbt follows these rules:
- Defers to the production environment when there's no development schema.
- If a development schema exists, dbt will prioritize those changes, which minimizes development time and avoids unnecessary model builds.
Both the dbt Cloud IDE and the dbt Cloud CLI allow users to natively defer to production metadata directly in their development workflows.
For specific scenarios:
- Use
--favor-state
to always use production artifacts to resolve the ref. - If facing issues with outdated tables in the development schema,
--favor-state
is an alternative to defer.
For a clean slate, it's a good practice to drop the development schema at the start and end of your development cycle.
Required setup
- You must select the Production environment checkbox in the Environment Settings page.
- This can be set for one deployment environment per dbt Cloud project.
- You must have a successful job run first.
When using defer, it compares artifacts from the most recent successful production job, excluding CI jobs.
Defer in the dbt Cloud IDE
To enable defer in the dbt Cloud IDE, toggle the Defer to production button on the command bar. Once enabled, dbt Cloud will:
- Pull down the most recent manifest from the Production environment for comparison
- Pass the
--defer
flag to the command (for any command that accepts the flag)
For example, if you were to start developing on a new branch with nothing in your development schema, edit a single model, and run dbt build -s state:modified
— only the edited model would run. Any {{ ref() }}
functions will point to the production location of the referenced models.
Defer in dbt Cloud CLI
One key difference between using --defer
in the dbt Cloud CLI and the dbt Cloud IDE is that --defer
is automatically enabled in the dbt Cloud CLI for all invocations, compared with production artifacts. You can disable it with the --no-defer
flag.
The dbt Cloud CLI offers additional flexibility by letting you choose the source environment for deferral artifacts. You can set a defer-env-id
key in either your dbt_project.yml
or dbt_cloud.yml
file. If you do not provide a defer-env-id
setting, the dbt Cloud CLI will use artifacts from your dbt Cloud environment marked "Production".
defer-env-id: '123456'
dbt_cloud:
defer-env-id: '123456'