@Glime Can you consume environment variables from a static workflow?

No, you cannot directly consume environment variables from a static workflow in Flyte. Static workflows are designed to be reproducible and shareable without depending on external infrastructure or runtime environment variables. This design ensures that static workflows can be executed consistently across different environments. However, for dynamic behavior based on external inputs or environment variables, Flyte supports dynamic workflows. Dynamic workflows allow for the construction of execution graphs that adapt to external inputs, including environment variables, while maintaining the reproducibility and shareability of static graphs. To pass environment variables to tasks within a workflow, you can use the

--env

or

--envvars

parameters with the

pyflyte run

command. This approach allows tasks within the workflow to access environment variables. However, this method applies to tasks and not directly to the workflow itself. For workflows, especially when dealing with sensitive information or configurations, it's recommended to use secrets management rather than directly passing environment variables.

@Glime Can you override the map_task concurrency values from within a task?

No, you cannot override the

concurrency

values of a

map_task

from within a task. The

concurrency

parameter for a

map_task

is defined at the time of the

map_task

declaration and is used to limit the number of mapped tasks that can run in parallel. This parameter is set to manage how many instances of the task can execute concurrently, and it is not designed to be dynamically adjusted from within a task itself. The

concurrency

setting is part of the task's configuration and is intended to be a static value that controls the execution behavior of the

map_task

. Changing this value dynamically from within a task would go against the design principle of having predictable and controlled task execution, especially in terms of resource management and scheduling. For tasks that require dynamic control over concurrency or other execution parameters, you might need to consider alternative approaches, such as breaking down the tasks differently, using dynamic workflows, or adjusting the design of your workflow to accommodate these requirements through other means.