Hey all, does anyone have experience using a dynamic with_overrides to set resource requests/limits at scale? We've been testing the following example in our dev cluster and its been working okay, wanted to start using this at scale and wanted to see if people are also doing this in larger workflows (e.g., 100's of dynamic nodes). I can't see any K8s or Flyte reason why this would have issues in the future (maybe funky interactions between heterogenous pod requests and K8s/EKS cluster autoscaling?), but its so nice and simple that it seems too good to be true.

@task
def test_task(input: int) -> int:
    import time
    time.sleep(600)
    return input + 1


@dynamic
def test_dynamic(list_of_ints: List[int]) -> List[int]:
    results = []
    for integer in list_of_ints:
        result = test_task(input=integer).with_overrides(requests=Resources(cpu=f"{integer}", mem=f"{integer}Gi"))
        results.append(result)
    return results

does this run locally?

Naw we just tested in our remote dev cluster

yup. thanks for the unit test idea

@Calvin Leather this is ok and designed for specifically