Gaurav Kumar
08/04/2023, 3:47 PMinit_image = Image(name="custom", fqn="custom_fqn", tag="custom_tag")
image_cfg = ImageConfig(images=[init_image])
remote_wf = remote.register_script(entity=aeb_sanity_workflow, copy_all=True, version=args.version, image_config=image_cfg, source_path=".")
When I execute it, it gives the error saying ERROR - failed to register the workflow: An image is required for PythonAutoContainer tasks
Do we need to specify default_image
as well which is an attribute of ImageConfig
, if Yes, what would be the value ?Kevin Su
08/04/2023, 4:15 PMGaurav Kumar
08/04/2023, 4:15 PMKevin Su
08/04/2023, 4:15 PMGaurav Kumar
08/04/2023, 4:21 PMimage_config= ImageConfig.auto_default_image()
, it picks default image.
But, since, I want to include custom + default both, isn’t there option where default image can be picked.
I see we have package flytekit.configuration.DefaultImages
but, it doesn’t give of type Image
DefaultImages.default_image()
and convert it into type Image
and put as default_image
in ImageConfig
Kevin Su
08/04/2023, 4:24 PMGaurav Kumar
08/04/2023, 5:01 PMdefault_image
.
One more thing, right now I’m passing the custom_image as part of wf registration, and then later executing it.
Is there any way, I can pass the custom_image at the time of execution as an input, so that I can set the container_image
parameter for a task during execution like below
@task(container_image="custom-image-passed-as-input-during-execution")
Or any other way to achieve this ?Kevin Su
08/04/2023, 9:15 PMGaurav Kumar
08/07/2023, 6:00 AMimageName, cmd, args, envs
that we need to pass to the container image. I want these inputs to be given to WF in the form of config.yaml file which inturn can be passed to the ContainerTask.
In other sense, I would like to register a WF once, and then during each execution, pass the config.yaml file which should determine the imageName, args, cmd, env that will be passed to the container image.
How can I pass the inputs that I got from WF to pass to the ContainerTask
ContainerTask(
image="",
command=[],
args=[],
environment=[]
custom_image = "my-custom-image" // set image at compile time
@task(container_image=custom_image,
cache=True,
cache_version="1.0",
...)
But, if I set the global variable through some task whose value is passed as input during execution, it doesn’t work. It instead picks up the default python flyte image. Tried with making setImage
and task1
dynamic as well, still it picks default image
Not working >>> (picks up default image)
custom_image = ""
@dynamic/task
def setImage(data: Dict):
global custom_image
// set image at runtime
custom_image = data['tasks']['initialize_input']['image']
return
@dynamic/task(container_image=custom_image,
cache=True,
cache_version="1.0",
...)
def task1(data: Dict):
...
@workflow
def wf(data: Dict):
setImagePromise = setImage(data=data)
out = task1(data=data)
setImagePromise >> out
return
Samhita Alla
container_image
to with_overrides
?
@workflow
def wf(container_image: str):
t1().with_overrides(container_image=container_image)
I'm not sure if this will work but please give it a try.Gaurav Kumar
08/07/2023, 9:06 AMwith_overrides
, it still uses the default image, I think the issue is we want the container_image
value to to be resolved based on the input during WF execution at runtime. I think workflow is using default image as it’s getting container_image
as None during compilation. But, that’s just my theory.
I tried to make the wf
@dynamic to check if that helps, but there I’m hitting another issue that I’m not able to register the workflow using register_script
. Scripts run successfully, but it doesn’t show up in the workflows. I guess register_script
only for non-dynamic workflows.Samhita Alla
container_image
has to work, right?Gaurav Kumar
08/07/2023, 12:58 PMSamhita Alla
register_script
😅Gaurav Kumar
08/07/2023, 4:34 PM