Good morning<@UNR3C6Y4T> <@UNZB4NW3S>. We recently...
# ask-the-community
n
Good morning@Yee @Ketan (kumare3). We recently updated to flytekit 1.3.2 and flyte 1.3.0. I’m seeing an issue where a task’s specific memory requests/limits and custom image is not being used. This is happening for a task where we set
task_config=Pod()
as we needed to customize some of the k8s spec. Usually for simpler tasks we set
requests
and
limits
for the
task
decorator It seems like the way we should be setting these values may have changed based on this? I’m reviewing this documentation now
d
Hey @Nicholas LoFaso so sorry about this. Using the configuration you mentioned - ie. existing
PodTask
should still be fully supported. We implemented the new
PodTemplate
to allow finer-grained configuration accross task types - for example
ContainerTask
. This is supposed to be fully backwards compatible, if it's not, it is a problem. Can you provide an example that reproduces this issue?
n
Right now the task definition is very specific to our environment. Let me try and reproduce on a separate example
False alarm it was operator error on our part!
d
😅 that's relieving! Hopefully debugging wasn't too difficult!
And if you want to chat more about the PodTemplate stuff, would be happy to. We're working on some more extensions here. Basically, you can define a
PodTemplate
in the task decorator which can also support
ContainerTask
in addition to different plugins (ex. Dask, Ray, etc). Ultimately, this will achieve the same thing as the
task_config=Pod
(ie.
PodTask
) that you're currently using, but will apply to more task types offering more configurability.
m
@Dan Rammer (hamersaw) I am trying to apply k8s labels using
pod_template
for
ContainerTask
but the labels doesn't reflect. I have tried
flytekit==1.4.2
and task def looks like below, Can you suggest if I am missing something here :
Copy code
calculate_ellipse_area_shell = ContainerTask(
    name="ellipse-area-metadata-shell",
    input_data_dir="/var/inputs",
    output_data_dir="/var/outputs",
    inputs=kwtypes(a=float, b=float),
    outputs=kwtypes(area=float, metadata=str),
    image="<http://ghcr.io/flyteorg/rawcontainers-shell:v2|ghcr.io/flyteorg/rawcontainers-shell:v2>",
    command=[
        "./calculate-ellipse-area.sh",
        "{{.inputs.a}}",
        "{{.inputs.b}}",
        "/var/outputs",
    ],
    pod_template=PodTemplate(labels=labels),
)
d
@Mohd Shahid Khan Afridi what version of flytepropeller are you running? This PR added support for this functionality into flyteplugins, which made it's way into flytepropeller in the v1.1.71 release. This flytepropller version seems to be included in Flyte v1.5.
101 Views