@Glime, can I provide inputs to a launch plan dynamically on runtime? For example, based on environment variables?

Yes, you can provide inputs to a launch plan dynamically at runtime, including using environment variables. This can be achieved by using the

os.getenv

function within your workflow definition to retrieve environment variable values and pass them as parameters to your tasks or workflows. For instance, you can define a workflow that takes inputs which are set dynamically based on the environment variables at runtime. Here's a simple example using Flytekit:

from flytekit import task, workflow import os  @task def my_task(param: str) -> str:     return f"Received: {param}"  @workflow def my_workflow():     env_var_value = os.getenv("MY_ENV_VAR", "default_value")     result = my_task(param=env_var_value)     return result

In this example,

MY_ENV_VAR

is an environment variable whose value will be used as an input to the

my_task

task. If

MY_ENV_VAR

is not set,

"default_value"

will be used. This approach allows you to parameterize your workflows based on the environment without needing to create multiple copies of the launch plan for different environments.

@Glime, what if I want to set

max_parallelism

for LaunchPlan based on environment?

To set

max_parallelism

for a LaunchPlan based on environment variables, you would typically handle this during the setup or registration of your LaunchPlan in your Flyte workflow code. You can use Python's

os

module to access environment variables and dynamically set the

max_parallelism

parameter when you create or get a LaunchPlan. Here’s an example of how you might do this:

import os from flytekit import LaunchPlan, workflow  @workflow def my_workflow():     # Your workflow definition here     pass  # Get max_parallelism from an environment variable, with a default fallback max_parallelism_env = os.getenv('MAX_PARALLELISM', '10')  # Default to 10 if not set max_parallelism = int(max_parallelism_env)  # Convert to integer  # Create or get a LaunchPlan with dynamic max_parallelism my_launch_plan = LaunchPlan.get_or_create(     name="dynamic_parallelism_launch_plan",     workflow=my_workflow,     max_parallelism=max_parallelism )

In this example, the

max_parallelism

for the LaunchPlan is set based on the

MAX_PARALLELISM

environment variable. If the environment variable is not set, it defaults to 10. This allows you to control the parallelism setting externally via environment configurations, which can be particularly useful for different deployment environments (e.g., development, staging, production). Make sure that the environment variable is properly set in the environment where your Flyte workflows are registered and executed.