incalculable-ice-13425
08/04/2023, 3:47 PMinit_image = Image(name="custom", fqn="custom_fqn", tag="custom_tag")
image_cfg = ImageConfig(images=[init_image])
remote_wf = remote.register_script(entity=aeb_sanity_workflow, copy_all=True, version=args.version, image_config=image_cfg, source_path=".")
When I execute it, it gives the error saying ERROR - failed to register the workflow: An image is required for PythonAutoContainer tasks
Do we need to specify default_image
as well which is an attribute of ImageConfig
, if Yes, what would be the value ?glamorous-carpet-83516
08/04/2023, 4:15 PMincalculable-ice-13425
08/04/2023, 4:15 PMglamorous-carpet-83516
08/04/2023, 4:15 PMglamorous-carpet-83516
08/04/2023, 4:15 PMincalculable-ice-13425
08/04/2023, 4:21 PMimage_config= ImageConfig.auto_default_image()
, it picks default image.
But, since, I want to include custom + default both, isn’t there option where default image can be picked.
I see we have package flytekit.configuration.DefaultImages
but, it doesn’t give of type Image
incalculable-ice-13425
08/04/2023, 4:24 PMDefaultImages.default_image()
and convert it into type Image
and put as default_image
in ImageConfig
glamorous-carpet-83516
08/04/2023, 4:24 PMincalculable-ice-13425
08/04/2023, 5:01 PMdefault_image
.
One more thing, right now I’m passing the custom_image as part of wf registration, and then later executing it.
Is there any way, I can pass the custom_image at the time of execution as an input, so that I can set the container_image
parameter for a task during execution like below
@task(container_image="custom-image-passed-as-input-during-execution")
Or any other way to achieve this ?glamorous-carpet-83516
08/04/2023, 9:15 PMincalculable-ice-13425
08/07/2023, 6:00 AMimageName, cmd, args, envs
that we need to pass to the container image. I want these inputs to be given to WF in the form of config.yaml file which inturn can be passed to the ContainerTask.
In other sense, I would like to register a WF once, and then during each execution, pass the config.yaml file which should determine the imageName, args, cmd, env that will be passed to the container image.
How can I pass the inputs that I got from WF to pass to the ContainerTask
ContainerTask(
image="",
command=[],
args=[],
environment=[]
incalculable-ice-13425
08/07/2023, 7:53 AMcustom_image = "my-custom-image" // set image at compile time
@task(container_image=custom_image,
cache=True,
cache_version="1.0",
...)
But, if I set the global variable through some task whose value is passed as input during execution, it doesn’t work. It instead picks up the default python flyte image. Tried with making setImage
and task1
dynamic as well, still it picks default image
Not working >>> (picks up default image)
custom_image = ""
@dynamic/task
def setImage(data: Dict):
global custom_image
// set image at runtime
custom_image = data['tasks']['initialize_input']['image']
return
@dynamic/task(container_image=custom_image,
cache=True,
cache_version="1.0",
...)
def task1(data: Dict):
...
@workflow
def wf(data: Dict):
setImagePromise = setImage(data=data)
out = task1(data=data)
setImagePromise >> out
return
tall-lock-23197
container_image
to with_overrides
?
@workflow
def wf(container_image: str):
t1().with_overrides(container_image=container_image)
I'm not sure if this will work but please give it a try.incalculable-ice-13425
08/07/2023, 9:06 AMincalculable-ice-13425
08/07/2023, 9:21 AMwith_overrides
, it still uses the default image, I think the issue is we want the container_image
value to to be resolved based on the input during WF execution at runtime. I think workflow is using default image as it’s getting container_image
as None during compilation. But, that’s just my theory.
I tried to make the wf
@dynamic to check if that helps, but there I’m hitting another issue that I’m not able to register the workflow using register_script
. Scripts run successfully, but it doesn’t show up in the workflows. I guess register_script
only for non-dynamic workflows.tall-lock-23197
container_image
has to work, right?tall-lock-23197
incalculable-ice-13425
08/07/2023, 12:58 PMtall-lock-23197
register_script
😅incalculable-ice-13425
08/07/2023, 4:34 PM