Ena Škopelja
02/27/2023, 3:42 PM~/.flyte/config.yaml
but can't figure out the proper formatKevin Su
02/27/2023, 4:41 PMEduardo Apolinario (eapolinario)
02/28/2023, 1:15 AM~/.flyte/config.yaml
like so:
admin:
endpoint: dns:///<your-admin>
images:
xyz: <http://docker.io/xyz:latest|docker.io/xyz:latest>
abc: <http://docker.io/abc|docker.io/abc>
Ena Škopelja
02/28/2023, 9:46 AMERRO[0000]
strict mode is on but received keys [map[images:{}]] to decode with no config assigned to receive them: failed strict mode check src="main.go:13"
from flytectl config validate
for the config you suggested.sandbox.config
to the docker image somehow other than having it in the project root, something like this? And what exactly should be in it? It's not clear to me how flyte reads it and how it interacts with the global config.
To clarify, this is my project setup:
|- workflows
| |- package_1
| | |- package_1
| | | |- __init__.py
| | | |- sandbox.config
| | | |- tasks.py
| | | |- workflows.py
| | |- pyproject.toml
| | |- default.Dockerfile
| | |- some_special.Dockerfile
| |- package_2
| | |- package_2
| | | |- __init__.py
| | | |- sandbox.config
| | | |- tasks.py
| | | |- workflows.py
| | |- pyproject.toml
| | |- default.Dockerfile
and package_1
depends on package_2
. What I would like is that each project has a registry of images that I can reference as described here. Is that possible?
The way I have it it's not getting recognized by flyte. I tried moving sandbox.config up one level but that doesn't help either.Eduardo Apolinario (eapolinario)
02/28/2023, 9:37 PMflytectl
strict mode. Mind filing an issue (using this template)? Here's an example PR that adds a new flag to flytectl config file: https://github.com/flyteorg/flytectl/pull/361/
2. Here's the logic used to find the config file: https://github.com/flyteorg/flytekit/blob/master/flytekit/configuration/file.py#L237-L270. As you can see, we assume that the file is going to be named either flytekit.config
or it lives under ~/.flyte
or its path is set via an environment var called FLYTECTL_CONFIG_ENV_VAR
or by specifying --config <filename>
when invoking pyflyte register / serialize
. The images
section is needed only when you're registering a task, so as long as you configure flytekit to find the correct file you can have it anywhere you want, including multiple config files, one per package. Again, just to reinforce the message, this is only needed at registration time, so you have a few options.
Please reach out if this is not clear.Ena Škopelja
02/28/2023, 11:32 PMflytekit.config
interact with my config in ~/.flyte/config.yaml
that I generated using pyflyte config init
, does it override it? I'm asking because I vaguely remember giving the --config
option a try but getting some registration error (something caused by the domain not being set correctly, which is not the case in my global config file).
Also, at which level are the docker images available? I'm assuming project+domain, right?Eduardo Apolinario (eapolinario)
03/01/2023, 12:04 AMpyflyte register
or using the combo pyflyte serialize + flytectl register
.
I'm curious to see which error you were seeing. We can set up a call tomorrow if you want.Ena Škopelja
03/01/2023, 10:31 AMdebug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30080: Failed to connect to remote host: Connection refused {created_time:"2023-03-01T11:24:21.493993+01:00", grpc_status:14}"
Or, more decriptively, I get Registering against localhost:30080
when running with --dry-run
What I have in the config file is:
[admin]
endpoint = dns:///some.internal.domain
authtype = Pkce
insecure = False
[logger]
show-source = True
level = 0
[images]
...
Registration works if I just reference ~/.flyte/config.yaml
that has:
admin:
# For GRPC endpoints you might want to use dns:///flyte.myexample.com
endpoint: dns:///some.internal.domain
authType: Pkce
insecure: false
logger:
show-source: true
level: 0
and in --dry-run
reports Registering against some.internal.domain
yaml
file in this format (note: has to have .yaml
suffix):
admin:
authType: Pkce
endpoint: dns:///some.internal.domain
insecure: false
images:
...
logger:
level: 0
show-source: true
Eduardo Apolinario (eapolinario)
03/01/2023, 8:04 PM.yml
in flytekit 1.1.0 (precisely in this commit: https://github.com/flyteorg/flytekit/commit/195d26ab0cc03260bb96b828219fcf7e6ea768a1)Ena Škopelja
03/02/2023, 9:49 AMyaml
is working properly, .config
is not picking up the admin stuffEduardo Apolinario (eapolinario)
03/02/2023, 3:29 PMEna Škopelja
03/02/2023, 3:31 PM.config
format as well?Eduardo Apolinario (eapolinario)
03/02/2023, 3:32 PM