Hi Community - Is there a way within a dynamic workflow to use the result of one task to calculate some values (e.g. memory request) to use with _with_overrides()_ on a subsequent task? It would seem that, since any result returned is a promise, you can never get at any actual values to be passed via _with_overrides()_. e.g.:

# inside an @dynamic workflow...

some_result = first_task(in=inputs)

mem_required = some_result.some_computed_metric * 8  # oops, can't de-ref here, some_result is a promise!

second_task(in=other_inputs).with_overrides( requests=Resources(mem=f"{mem_required}Gi" ) )

How to accomplish this? Do I need to use multiple @dynamic workflows so that the result returned by the first one can be passed into the second one and thus be dereferenced for use by _with_overrides()_ ?

hmm can you not use the first_task? i guess it depedns on how compute intensive it is. the way we did is we just added concrete python code instead of wrapping it in a task like below

resource_override = get_res_override(input)
second_task(...).with_overrides(resource_override)

yeah at first blush, nested is the way to go here. but why not move the first task out of the dynamic?

@Jay Ganbat @Yee Thanks for your responses. The first task is pretty compute intensive. I think I was just sanity checking my conclusion which is: "The only values you can dereference/use in a task or @dynamic workflow are those passed as inputs - you can never make use of anything returned from a task, which will be a promise - you can of course pass this to a downstream task, but can never get "dereference-able/usable" values back into the dynamic or task". So I think Yee's recommendation is correct for us -- move first_task out of the dynamic. The reason we didn't do this initially is just writing the code with as simple a call-graph as possible -- conceptually it's a single "workflow", so we model it as such in code.