How to create a minimal test project with dbt Core?

A test project can be a very good way to try out new techniques. With dbt Core you can easily create a minimal dbt Core project on your own computer. To do this, however, dbt Core must already be installed and a connection to Snowflake, for example, must already be in place. For information on setting up such a connection with RSA encryption, go to my blog about it.

In this blog, we will set up a small test project. By minimal, in this case, we mean with as few files and lines of code as possible. Such a small dbt project can be useful because it lets us concentrate on the techniques we want to test. Because everything else is left out, those things don't distract either.

The connection

In a previous blog, I showed how to apply RSA encryption in a connection with Snowflake. In the end there came the following dbt profiles file:

default:
  target: dev
  outputs:
    dev:
      type:      snowflake
      account:   voor-beeld
      user:      esempio
      role:      sysadmin
      warehouse: magazijn     
      database:  ontwikkel_db
      schema:    dbt_esempio
      
      private_key_path:       /Users/esempio/.dbt/rsa.p8 
      private_key_passphrase: t+gsjivVXODDzhujYLW/ZdjVuYrIz4n5

Perhaps needless to say, but the passphrase is a randomly generated example. In principle, this kind of passphrase should not be shared. You don't have to if you keep creating one yourself. What is more important here is that default the profile (also known as connection profile) is the one we will use later.

This profile we could call anything (including kaas or peper). It is useful to use the default to mention when we also dbt Cloud want to use. In the profiles.yml file may contain this one profile. But there can also be several. Later we will explicitly name which profile we are going to use (even if there is only 1 profile in the profiles.yml state).

Targets

A dbt connection profile can have multiple outputs. These are listed after outputs:. Above there is only 1 output dev, but this too could be several. Behind target is the default output. So we don't have to do anything with that later. If we want to use a different output with a dbt command, we can specify it explicitly: for example, with --target dev.

dbt_project.yml

Now we are ready to create a new project. We start by creating a new folder (directory). Let's create this minimaal name. Next, we are going to create a file in this folder named dbt_project.yml. We fill these with the following lines:

name: minimaal
version: '0.0.0'
config-version: 2
profile: default

The version doesn't really matter in this situation. We choose 0.0.0 as a starting point. For name we can choose anything. We choose the same name as the folder. config-version must 2 are. profile is very important. That's where we take default, the name of the profile at profiles.yml!

Now we can already see a dbt debug to see if everything works. If that's successful, all we really need to do is create a folder under minimaal create: models. If we create a SQL file in it with the techniques we want to try, we can get started. models is by default the directory where dbt will look for our models (SQL files). We can choose another directory with model-paths, but that is not necessary. dbt itself creates all the other folders it needs and those are the dafault folder names!

Discussion (minimum dbt Core test project).

Once dbt is installed and connected to the data platform, we only need the following for a minimal dbt Core test project:

• A folder (directory).
• A dbt_project.yml in that folder.
• A models folder in the first folder.
• SQL files in the models folder.

Then we can get started!

Thanks for reading this blog. Check out our blog page for more blogs about Tableau, Alteryx, and Snowflake.

Work together with one of our consultants and get more out of your data. 

Contact us and we will help you.