Hey folks, Does anyone tried using dbt in flyte wo...
# flyte-supportw
wide-balloon-43886
11/10/2023, 11:01 AMHey folks, Does anyone tried using dbt in flyte workflow ? I am trying a use case to run flyte dbt workflow with dbt plugins? Is there a way to cache the pods directories when you run dbt deps in one step and therefore I no need to run in downstream pods for the build, test and freshness tasks (pods) ?
Also checking this as an idea 💡 https://docs.getdbt.com/reference/programmatic-invocations ? Is flyte plugins is developed on top of this ?
f
freezing-airport-6809
11/10/2023, 2:56 PM
It is not built on this please feel free to propose a change
👍 1
w
wide-balloon-43886
11/10/2023, 4:10 PMThank you Ketan!
wide-balloon-43886
11/10/2023, 4:10 PMHow can I cache the result from previous pod (task) to the next task ?
wide-balloon-43886
11/10/2023, 4:11 PMBecause, in this case its a target directory having compiled packages installed, compiled SQL and artefacts of dbt.
f
freezing-airport-6809
11/10/2023, 4:15 PM
Just enable caching
freezing-airport-6809
11/10/2023, 4:15 PM
It will auto cache
w
wide-balloon-43886
11/10/2023, 4:18 PMLet me try that, thanks Ketan!
4 Views