https://flyte.org logo
Title
r

Ryan Delgado

09/08/2022, 11:07 PM
When I execute a flyte workflow, do the nodes in that workflow run in separate Kubernetes pods? If so, How can I adjust the amount of cpu/memory I allocate to a particular kubernetes pod?
k

Ketan (kumare3)

09/09/2022, 12:55 AM
Yes they do
You can adjust by passing resources to every task
Even in unionml
Cc @Eduardo Apolinario (eapolinario) can you share an example
n

Niels Bantilan

09/09/2022, 2:07 PM
hi @Ryan Delgado so these two unionml decorators take in all the
kwargs
of the flyte
@task
decorator: • Dataset.readerModel.trainer So you can do
flytekit.Resources(cpu="4")
for the dataset reading step and another configuration for the training step. FYI, the
dataset.loader
,
parser
,
splitter
, are all invoked within the training task to reduce overhead of spinning up pods.