https://flyte.org
Join Slack
Menu
Home
Archive
Is it possible to mix `@flytekit.dynamic` & `with_overrides`? Curious if we could pass a resourc...
# flytekit
e
Is it possible to mix 
@flytekit.dynamic
& 
with_overrides
? Curious if we could pass a resource configuration object to a workflow at runtime and have the 
@dynamic
step specify overrides at runtime
f
Yes you should be able to
That was one of the intended usecases
e
Ah, I was trying to parameterize a workflow with `cpu`/`mem` so that we could dynamically specify them as arguments (in lieu of https://github.com/flyteorg/flyte/issues/475)
Looks like this successfully compiles w/ dynamic but if there's a sub-workflow (
@workflow
, not 
@dynamic
) then it leads to the following error
Copy code
AttributeError: 'Promise' object has no attribute 'cpu'
Although, when actually examining the pod in the cluster it looks like supplying args to 
with_overrides
for limits doesn't have any effect/I still see the following when running 
kubectl describe
on one of the pods (when I specified 
cpu="1"
& 
mem=100Mi
)
Copy code
Limits:
      cpu:     100m
      memory:  200Mi
    Requests:
      cpu:     100m
      memory:  200Mi
If I hard-code the values in it seems to work fine but it doesn't appear to be possible to specify 
with_overrides
w/ an argument to a workflow
Copy code
'VoidPromise' object has no attribute 'with_overrides'
If anyone on the team could provide an example of setting overrides in a dynamic step based on an argument that'd be very helpful..seems like it's possible per https://flyte-org.slack.com/archives/CP2HDHKE1/p1655135984234209, but I must be missing something…
t
I guess mixing 
@flytekit.dynamic
and 
with_overrides
isn’t yet possible. @billowy-sundown-31926, I remember you telling that this didn’t work. Could you elaborate more on this, please?
f
b
Thats right @tall-lock-23197. Facilitating 
dynamic
with 
with_overrides
is a work in progress. Right now, it can't be used. cc @thankful-minister-83577 If we try to use 
with_overrides
with 
dynamic
workflow, it fails to execute.
f
@elegant-australia-91422 I am sorry in that case, let us look into this tomorrow. For some reason this feels like a bug
🙏 1
e
Filed https://github.com/flyteorg/flyte/issues/2730 w/ some more details of the issue
f
cc @thankful-minister-83577
@elegant-australia-91422 I did not get the error - https://github.com/flyteorg/flyte/issues/2730#issuecomment-1199995173
It works for me on a production cluster
it does not work locally, but that is because of the way we interpret it and that should be a more complex fix to make it run locally
e
Ah this was on a production cluster with the sandbox helm chart deployed. To confirm I was running on remote. I'll upgrade all the deps and give this a shot next week
f
sure
if you hop on a call i can show you as well
e
Sure - thanks very much for digging into it so quickly @freezing-airport-6809 ! Will reach out next week
f
absolutely
e
Confirmed this is no longer an issue on the most recent version of the chart that I deployed. Thanks again @freezing-airport-6809!
f
Interesting, I don't even know we fixed it
e
This is definitely going to be very powerful and is the final blocker we had to fully moving towards flyte 🙂
❤️ 1
f
Hopefully you will spread the word
👍 1
162 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by