sparse-window-1536
03/15/2022, 2:37 PMgenerate_regression_dataset
and fit_polynomial_regression
. The second task connects to Google Cloud Storage and MLflow to log artifacts and model to the servers, and to do so it needs passwords and service account files, and I pass them as Secrets to the task (passwords as environment variables, and service accounts as files).
When I build the image and serialize the tasks, everything works perfectly (see Image 1). But then, if I use fast registration, suddenly the task does not have access to Secrets passed as environment variables (see Image 2). Keep in mind, the only changes I made to the task was to add a print()
to debug my current environment variables and the /etc/flyte/secrets
directory.
When I print the contents of /etc/flyte/secrets
, I see that the service account was correctly mounted as a file, because I specified that mount_requirement=Secret.MountType.FILE
on the Secret definition. All the other secrets, that were specified as mount_requirement=Secret.MountType.ENV_VAR
, are unavailable. When I print os.environ
, I can't find a single variable that starts with _FSEC_
. I think that fast registration is breaking the process of setting the secret environment variables.
I'd help to fix this problem, but I have no idea where to start on the repository (where fast registration occurs, when the task is initialized, etc.).