When I execute a flyte workflow, do the nodes in t...
# ecosystem-unionml
r
When I execute a flyte workflow, do the nodes in that workflow run in separate Kubernetes pods? If so, How can I adjust the amount of cpu/memory I allocate to a particular kubernetes pod?
k
Yes they do
You can adjust by passing resources to every task
Even in unionml
Cc @Eduardo Apolinario (eapolinario) can you share an example
n
hi @Ryan Delgado so these two unionml decorators take in all the
kwargs
of the flyte
@task
decorator: • Dataset.readerModel.trainer So you can do
flytekit.Resources(cpu="4")
for the dataset reading step and another configuration for the training step. FYI, the
dataset.loader
,
parser
,
splitter
, are all invoked within the training task to reduce overhead of spinning up pods.
134 Views