https://flyte.org logo
Powered by
n

Nada Saiyed

09/14/2022, 5:05 AM
Hello, i have extended flytekit to create a 
PythonInstanceTask
type plugin. I need to reference the default image that this task will run on, during remote execution. how can i reference that? thanks!
s

Samhita Alla

09/14/2022, 5:34 AM
@Nada Saiyed, I believe you could send an image as a `container_image`: https://github.com/flyteorg/flytekit/blob/2ccaed77c92e6ff97d6692e4bf91b003f560b088/flytekit/core/python_auto_container.py#L36. @Kevin Su, is this the right way to do?
n

Nada Saiyed

09/14/2022, 6:08 AM
@Samhita Alla right, but then I will need to provide 
container_image
.during task definition. I need to reference the default image uri that was provided while serializing the task. How can that be done?
cc : @Kevin Su
s

Samhita Alla

09/14/2022, 6:15 AM
Would you be using the same image to serialize all the tasks in a specific workflow?
n

Nada Saiyed

09/14/2022, 6:16 AM
Yes
s

Samhita Alla

09/14/2022, 6:17 AM
Then you needn’t provide a container image to your task. On serializing the workflow, the same image will be used for your 
PythonInstanceTask
as well.
n

Nada Saiyed

09/14/2022, 6:20 AM
Exactly. So, how can I reference that image uri within my custom plugin? E.g. When serializing I will be providing the image uri as part of pyflyte command and I need to reference that
Copy code
pyflyte —-pkgs workflows.example package —-image <image_uri>
s

Samhita Alla

09/14/2022, 6:21 AM
You need not reference at all. The image you give in your pyflyte command will be used to serialize the custom task.
n

Nada Saiyed

09/14/2022, 6:22 AM
Right. But I need to access that image uri to perform some logic in the 
execute()
. method
After serialization how can I retrieve that image uri?
k

Kevin Su

09/14/2022, 6:24 AM
Oh, I see. Let me dig into it
I think you should add a 
ENV FLYTE_INTERNAL_IMAGE=<image_uri>
to the dockerfile like this. and then you are able to retrieve it by using
Copy code
def execute(self, **kwargs) -> Any:
        image = self.container_image or os.getenv("FLYTE_INTERNAL_IMAGE")
n

Nada Saiyed

09/14/2022, 2:59 PM
yes. that’s an option., but lets say, i dont want to pass it via a docker file. is it possible to get the task details via any flyte api call giving an execution id and then i can parse the image out of the task details? @Kevin Su
k

Kevin Su

09/14/2022, 3:08 PM
you can use flyte remote to fetch task spec, and task spec has image url, but it’s wired to use flyte remote in the task. another way to achieve it is append an image_url to env by default. https://github.com/flyteorg/flyteplugins/blob/9e354af3d43f8f56ba46a97ffe67e26b0986937d/go/tasks/pluginmachinery/flytek8s/container_helper.go#L217 you can do something like 
append(r(taskContainer.GetEnv()), {FLYTE_INTERNAL_IMAGE: taskContainer.GetImage()})
May I ask why you need to use image_url in the task?
n

Nada Saiyed

09/14/2022, 5:52 PM
i would be using this uri to inturn launch a sagemaker service
another way to achieve it is append an image_url to env by default
this would require change in the backend plugin? @Kevin Su
k

Kevin Su

09/15/2022, 4:14 AM
I see, yes it require change in the backend
Powered by