Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

I am kind of a newbee with Flyte. But I am trying to understand how one can make certain properties such as resource constraints for tasks configurable from command line. I have a pipeline that is configured from command line and uses Flyte remote to launch a workflow.

Say I want to change the resource configuration of one of my tasks, say number of CPUs or memory size. Do I need to use a @dynamic task for that and rely on with_overrides?

you mean you want to change the cpu/memory at execution time?

does the cli mean - you also want to change it from the ui?

if not, then you can write a wrapper to execute, which can set some constant values that are read prior to module load and then configure

for example a simple way could be an env var to use high mem


```MEM = "10Gi"
if os.getenv("HIGH_MEM", False):
   MEM = "100Gi"

@task(resources=Resources(mem=MEM))
def foo():
  ...```

if you are using `pyflyte run`  this will auto-register a new version

Ok, I'll try this out (using environment variables).

Is there some plan to introduce a notion of global configs in Flyte? Might be useful (a proprietary orchestration system I'm familiar with added such a feature recently and it was widely adopted).

We do have attributes config
<https://docs.flyte.org/en/latest/deployment/configuration/customizable_resources.html#customizing-project-domain-and-workflow-resources-with-flytectl|https://docs.flyte.org/en/latest/deployment/configuration/customizable_resources.html#customizing-project-domain-an[…]rkflow-resources-with-flytectl>?

You can set attributes in project/project-domain/workflow levels.
For example
```flytectl update task-resource-attribute --attrFile tra.yaml```
```# tra.yaml
domain: development
project: flytesnacks
workflow: core.control_flow.merge_sort.merge_sort
defaults:
  cpu: "1"
  memory: "150Mi"
limits:
  cpu: "2"
  memory: "450Mi"```
