Hey folks, Does anyone tried using dbt in flyte wo...
# ask-the-communitys
Sree Rama Chinthamsetty
11/10/2023, 11:01 AMHey folks, Does anyone tried using dbt in flyte workflow ? I am trying a use case to run flyte dbt workflow with dbt plugins? Is there a way to cache the pods directories when you run dbt deps in one step and therefore I no need to run in downstream pods for the build, test and freshness tasks (pods) ?
Also checking this as an idea 💡 https://docs.getdbt.com/reference/programmatic-invocations ? Is flyte plugins is developed on top of this ?
k
Ketan (kumare3)
11/10/2023, 2:56 PM
It is not built on this please feel free to propose a change
s
Sree Rama Chinthamsetty
11/10/2023, 4:10 PMThank you Ketan!
Sree Rama Chinthamsetty
11/10/2023, 4:10 PMHow can I cache the result from previous pod (task) to the next task ?
Sree Rama Chinthamsetty
11/10/2023, 4:11 PMBecause, in this case its a target directory having compiled packages installed, compiled SQL and artefacts of dbt.
k
Ketan (kumare3)
11/10/2023, 4:15 PM
Just enable caching
Ketan (kumare3)
11/10/2023, 4:15 PM
It will auto cache
s
Sree Rama Chinthamsetty
11/10/2023, 4:18 PMLet me try that, thanks Ketan!
4 Views