sparse-window-1536
04/05/2022, 9:22 PMGOOGLE_APPLICATION_CREDENTIALS
and MLFLOW_TRACKING_USERNAME
. To make things easier for the team, I defined a decorator called @setup_task
, that receives strings specifying which integrations we want to use. For instance,
@task(secret_requests=MLFLOW_SECRETS, ...)
@task_setup(integration_requests=['mflow'])
def hello_mlflow() -> str:
In the above case, task_setup
is responsible for setting up the environment variables for MLflow. The problem is that these variables are defined as secrets on the Kubernetes cluster, so when I try to register the tasks, I receive errors such as
ValueError: Unable to find secret for key MLFLOW_TRACKING_URI in group mlflow-credentials in Env Var:_FSEC_MLFLOW-CREDENTIALS_MLFLOW_TRACKING_URI and FilePath: /etc/secrets/mlflow-credentials/mlflow_tracking_uri
Which is (kinda) expected, since I don't have the secret locally. But my whole logic is that these tasks won't be running locally. Is there no way that I can avoid this? The only workaround I can think is transforming the decorator into a normal function, and calling it inside the task. Is this the only way?