Hi, I'm a bit confused about how to specify images if I don't use the sandbox.config file here: https://docs.flyte.org/projects/cookbook/en/latest/auto/core/containerization/multi_images.html#sphx-glr-auto-core-containerization-multi-images-py I tried in

~/.flyte/config.yaml

but can't figure out the proper format

have to create sandbox.config in workflow directory

@Ena Škopelja,you can add a stanza to your

~/.flyte/config.yaml

like so:

admin:
  endpoint: dns:///<your-admin>
images:
  xyz: <http://docker.io/xyz:latest|docker.io/xyz:latest>
  abc: <http://docker.io/abc|docker.io/abc>

@Eduardo Apolinario (eapolinario) I get this error:

ERRO[0000]
strict mode is on but received keys [map[images:{}]] to decode with no config assigned to receive them: failed strict mode check  src="main.go:13"

from

flytectl config validate

for the config you suggested.

I see now, thanks for clarifying this. Two things: 1. you found a bug in

flytectl

strict mode. Mind filing an issue (using this template)? Here's an example PR that adds a new flag to flytectl config file: https://github.com/flyteorg/flytectl/pull/361/ 2. Here's the logic used to find the config file: https://github.com/flyteorg/flytekit/blob/master/flytekit/configuration/file.py#L237-L270. As you can see, we assume that the file is going to be named either

flytekit.config

or it lives under

~/.flyte

or its path is set via an environment var called

FLYTECTL_CONFIG_ENV_VAR

or by specifying

--config <filename>

when invoking

pyflyte register / serialize

. The

images

section is needed only when you're registering a task, so as long as you configure flytekit to find the correct file you can have it anywhere you want, including multiple config files, one per package. Again, just to reinforce the message, this is only needed at registration time, so you have a few options. Please reach out if this is not clear.

Thank you, I'll give it another go tomorrow. Just one follow up, how does

flytekit.config

interact with my config in

~/.flyte/config.yaml

that I generated using

pyflyte config init

, does it override it? I'm asking because I vaguely remember giving the

--config

option a try but getting some registration error (something caused by the domain not being set correctly, which is not the case in my global config file). Also, at which level are the docker images available? I'm assuming project+domain, right?

flytekit is going to use the first config file that it can find for that particular invocation. So, in your case, you'll need the image information at registration time, e.g. as part of the call to

pyflyte register

or using the combo

pyflyte serialize + flytectl register

. I'm curious to see which error you were seeing. We can set up a call tomorrow if you want.

Here's the error I'm seeing, it's trying to post a workflow to the wrong endpoint:

debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30080: Failed to connect to remote host: Connection refused {created_time:"2023-03-01T11:24:21.493993+01:00", grpc_status:14}"

Or, more decriptively, I get

Registering against localhost:30080

when running with

--dry-run

What I have in the config file is:

[admin]
endpoint = dns:///some.internal.domain
authtype = Pkce
insecure = False

[logger]
show-source = True
level = 0

[images]
...

Registration works if I just reference

~/.flyte/config.yaml

that has:

admin:
# For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///some.internal.domain
  authType: Pkce
  insecure: false
logger:
  show-source: true
  level: 0

and in

--dry-run

reports

Registering against some.internal.domain

What version of flytekit are you running? We added support for

.yml

in flytekit 1.1.0 (precisely in this commit: https://github.com/flyteorg/flytekit/commit/195d26ab0cc03260bb96b828219fcf7e6ea768a1)

Why is that relevant?

yaml

is working properly,

.config

is not picking up the admin stuff

ok, so you're unblocked ? We're supporting both config formats for compatibility purposes, but yaml is our recommendation going forwad

Yes, thank you for your help 🙏 ! Do you want me to file a bug for the

.config

format as well?

sure and thank you!