Is it possible for `pyflyte register` to pick up e...
# ask-the-community
e
Is it possible for
pyflyte register
to pick up environment variables when I set them like this?
Copy code
@task(
    task_config=Databricks(databricks_token=os.environ.get("DATABRICKS_TOKEN"))
)
def etl():
k
absolutely
is it not picking it up ?
e
Great to know it should pick up on them. I thought it was picking up and then it stopped! I will debug on my end. Thanks for the help.
I believe the reason that flyte stopped picking up the environment variables is because I started using a @dynamic workflow. I just tested 2 versions of the workflow: 1. Not using a dynamic workflow 2. Wrapping the task in a dynamic workflow Scenario 1 worked and scenario 2 failed. I searched the docs and found the
environment
options for @dynamic. Setting it in
@dynamic
seemed to propagate the env variable to the inner task. This worked great!
Copy code
@task(
    task_config=Databricks(
        databricks_conf={
            "run_name": "rfy_etl",
            "existing_cluster_id": "0302-183318-vi1yrbkk"
        },
        databricks_token=os.environ.get("DATABRICKS_TOKEN"),
    ),
)
def test_task() -> int:
    return 10


@dynamic(environment={"DATABRICKS_TOKEN": os.environ.get("DATABRICKS_TOKEN")})
def wrapper() -> int:
    return test_task()

@workflow
def wf() -> int:
    return wrapper()
k
Ya this is interesting- Flyte does not capture env vars by default
Maybe in environment setting we can use delegate. But this is potentially dangerous. As you may have secrets etc
e
I think it works great as is, I just needed to a do a bit more research.
Thanks for your help!!!!!!
104 Views