Understanding the run command

The operations behind the run command and the different ways to select models

Part of the “Mastering dbt” series. Access to the full Study Guide. Let’s connect on LinkedIn!
Notes from this documentation.

When we execute the “dbt run” command, dbt connects to the warehouse and runs the necessary operations to materialise the objects according to the configuration chosen.

The models are materialised in the order of the DAG and using multi-threading to minimise runtime while still executing the models in the right order.

Also, when existing models are run again, they are first materialised with a temporary name while the prior version of the model is dropped. Then, it gets renamed to its correct name. These operations take place in a single transaction for the adapters that support transactions.

Running specific models

You can select specific models when you don’t need the entire project to be run. You can do that by using these arguments:

--select, --exclude, --selector, --defer

Example of model selection. Source: node selector documentation

Difference between select and selector:

The select argument allows you to select models based on their names or specific methods, like “tag:”, “version:” or “group:”.

However, the selector allows for a more reusable selection. You can define groups of models in a separate selector.yml file and use the selector argument to run these models.

An example selector.yml file. Source: YAML selectors documentation

Graph operators

The + operator

The + operator allows you to include upstream and downstream models in your command.

How to use the + operator. Source: graph operator documentation

You can also specify how many models downstream and/or upstream you’d like the command to apply to.

Using the n-plus operator. Source: graph operator documentation

The @ operator

By using the @ operator, you can include all the models necessary for the successful build of the model selected.

Source: graph operator documentation

In the example above, the selector @snowplow_web_page_content would run all 3 models shown.

Set operators

Unions

This allows you to run multiple groups of models. In the example below, the models run would be: snowplow_sessions, all of its upstream models, fct_orders, and all of its upstream models. The whitespace between the model names is what makes it a union.

Source: set operators documentation

Intersection

If instead of whitespace, you use a comma between the models, dbt will run only the models included in both arguments. The example below would run all the models under the marts.finance directory that are also tagged as nightly.

Source: set operators documentation

The - -empty flag

You can check for the dependencies and ensure that your models will be built correctly without fully materialising the models into the database and incurring costs.

To do this, you can use the - -empty flag. dbt will execute the compiled SQL, but limit sources and refs to zero rows, generating empty tables and views while still running the code.