Hello, i have extended flytekit to create a `Pytho...
# flytekitn
Nada Saiyed
09/14/2022, 5:05 AMHello, i have extended flytekit to create a
PythonInstanceTasks
Samhita Alla
09/14/2022, 5:34 AM
@Nada Saiyed, I believe you could send an image as a `container_image`: https://github.com/flyteorg/flytekit/blob/2ccaed77c92e6ff97d6692e4bf91b003f560b088/flytekit/core/python_auto_container.py#L36.
@Kevin Su, is this the right way to do?
Samhita Alla
09/14/2022, 5:35 AM
Also Nada, for a pre-built container, take a look at https://docs.flyte.org/projects/cookbook/en/latest/auto/core/extend_flyte/prebuilt_container.html doc page.
n
Nada Saiyed
09/14/2022, 6:08 AM@Samhita Alla right, but then I will need to provide
container_imageNada Saiyed
09/14/2022, 6:12 AMcc : @Kevin Su
s
Samhita Alla
09/14/2022, 6:15 AM
Would you be using the same image to serialize all the tasks in a specific workflow?
n
Nada Saiyed
09/14/2022, 6:16 AMYes
s
Samhita Alla
09/14/2022, 6:17 AM
Then you needn’t provide a container image to your task. On serializing the workflow, the same image will be used for your
PythonInstanceTaskn
Nada Saiyed
09/14/2022, 6:20 AMExactly. So, how can I reference that image uri within my custom plugin?
E.g. When serializing I will be providing the image uri as part of pyflyte command and I need to reference that
Copy code
pyflyte —-pkgs workflows.example package —-image <image_uri>s
Samhita Alla
09/14/2022, 6:21 AM
You need not reference at all. The image you give in your pyflyte command will be used to serialize the custom task.
n
Nada Saiyed
09/14/2022, 6:22 AMRight. But I need to access that image uri to perform some logic in the
execute()Nada Saiyed
09/14/2022, 6:23 AMAfter serialization how can I retrieve that image uri?
k
Kevin Su
09/14/2022, 6:24 AMOh, I see. Let me dig into it
Kevin Su
09/14/2022, 6:47 AMI think you should add a
ENV FLYTE_INTERNAL_IMAGE=<image_uri> Copy code
def execute(self, **kwargs) -> Any:
image = self.container_image or os.getenv("FLYTE_INTERNAL_IMAGE")n
Nada Saiyed
09/14/2022, 2:59 PMyes. that’s an option., but lets say, i dont want to pass it via a docker file. is it possible to get the task details via any flyte api call giving an execution id and then i can parse the image out of the task details? @Kevin Su
k
Kevin Su
09/14/2022, 3:08 PMyou can use flyte remote to fetch task spec, and task spec has image url, but it’s wired to use flyte remote in the task.
another way to achieve it is append an image_url to env by default.
https://github.com/flyteorg/flyteplugins/blob/9e354af3d43f8f56ba46a97ffe67e26b0986937d/go/tasks/pluginmachinery/flytek8s/container_helper.go#L217
you can do something like
append(r(taskContainer.GetEnv()), {FLYTE_INTERNAL_IMAGE: taskContainer.GetImage()})Kevin Su
09/14/2022, 3:09 PMMay I ask why you need to use image_url in the task?
n
Nada Saiyed
09/14/2022, 5:52 PMi would be using this uri to inturn launch a sagemaker service
Nada Saiyed
09/14/2022, 5:56 PManother way to achieve it is append an image_url to env by defaultthis would require change in the backend plugin? @Kevin Su
k
Kevin Su
09/15/2022, 4:14 AMI see, yes it require change in the backend
156 Views