Hello, i have extended flytekit to create a `Pytho...
# flytekitp
plain-carpenter-67621
09/14/2022, 5:05 AMHello, i have extended flytekit to create a
PythonInstanceTaskt
tall-lock-23197
09/14/2022, 5:34 AM
@plain-carpenter-67621, I believe you could send an image as a `container_image`: https://github.com/flyteorg/flytekit/blob/2ccaed77c92e6ff97d6692e4bf91b003f560b088/flytekit/core/python_auto_container.py#L36.
@glamorous-carpet-83516, is this the right way to do?
👍 1
tall-lock-23197
09/14/2022, 5:35 AM
Also Nada, for a pre-built container, take a look at https://docs.flyte.org/projects/cookbook/en/latest/auto/core/extend_flyte/prebuilt_container.html doc page.
p
plain-carpenter-67621
09/14/2022, 6:08 AM@tall-lock-23197 right, but then I will need to provide
container_imageplain-carpenter-67621
09/14/2022, 6:12 AMcc : @glamorous-carpet-83516
t
tall-lock-23197
09/14/2022, 6:15 AM
Would you be using the same image to serialize all the tasks in a specific workflow?
p
plain-carpenter-67621
09/14/2022, 6:16 AMYes
t
tall-lock-23197
09/14/2022, 6:17 AM
Then you needn’t provide a container image to your task. On serializing the workflow, the same image will be used for your
PythonInstanceTaskp
plain-carpenter-67621
09/14/2022, 6:20 AMExactly. So, how can I reference that image uri within my custom plugin?
E.g. When serializing I will be providing the image uri as part of pyflyte command and I need to reference that
Copy code
pyflyte —-pkgs workflows.example package —-image <image_uri>t
tall-lock-23197
09/14/2022, 6:21 AM
You need not reference at all. The image you give in your pyflyte command will be used to serialize the custom task.
p
plain-carpenter-67621
09/14/2022, 6:22 AMRight. But I need to access that image uri to perform some logic in the
execute()plain-carpenter-67621
09/14/2022, 6:23 AMAfter serialization how can I retrieve that image uri?
g
glamorous-carpet-83516
09/14/2022, 6:24 AMOh, I see. Let me dig into it
👍 1
glamorous-carpet-83516
09/14/2022, 6:47 AMI think you should add a
ENV FLYTE_INTERNAL_IMAGE=<image_uri> Copy code
def execute(self, **kwargs) -> Any:
image = self.container_image or os.getenv("FLYTE_INTERNAL_IMAGE")p
plain-carpenter-67621
09/14/2022, 2:59 PMyes. that’s an option., but lets say, i dont want to pass it via a docker file. is it possible to get the task details via any flyte api call giving an execution id and then i can parse the image out of the task details? @glamorous-carpet-83516
g
glamorous-carpet-83516
09/14/2022, 3:08 PMyou can use flyte remote to fetch task spec, and task spec has image url, but it’s wired to use flyte remote in the task.
another way to achieve it is append an image_url to env by default.
https://github.com/flyteorg/flyteplugins/blob/9e354af3d43f8f56ba46a97ffe67e26b0986937d/go/tasks/pluginmachinery/flytek8s/container_helper.go#L217
you can do something like
append(r(taskContainer.GetEnv()), {FLYTE_INTERNAL_IMAGE: taskContainer.GetImage()})glamorous-carpet-83516
09/14/2022, 3:09 PMMay I ask why you need to use image_url in the task?
p
plain-carpenter-67621
09/14/2022, 5:52 PMi would be using this uri to inturn launch a sagemaker service
plain-carpenter-67621
09/14/2022, 5:56 PManother way to achieve it is append an image_url to env by defaultthis would require change in the backend plugin? @glamorous-carpet-83516
g
glamorous-carpet-83516
09/15/2022, 4:14 AMI see, yes it require change in the backend
158 Views