Hank Huang
07/13/2023, 9:37 AMFROM python:3.9-slim-buster
USER root
WORKDIR /root
ENV PYTHONPATH /root
RUN apt-get update && apt-get install build-essential -y
RUN apt-get install git -y
# The following line is an example of how to install your own plugins. In this case, it demonstrates how to install the 'deck' plugin.
# RUN pip install -U git+<https://github.com/Yicheng-Lu-llll/flytekit.git@>"demo#egg=flytekitplugins-deck-standard&subdirectory=plugins/flytekit-deck-standard" # replace with your own repo and branch
RUN pip install -U git+<https://github.com/hhcs9527/flytekit.git@master>
ENV FLYTE_INTERNAL_IMAGE "localhost:30000/flytekit:master"
Here is my workflow (relatively short, so I just paste it here)
import typing
from typing import Tuple
from flytekit import task, workflow
@task(disable_deck=False)
def t1(a: int) -> typing.NamedTuple("OutputsBC", t1_int_output=int, c=str):
return a + 2, "world"
@task(disable_deck=False)
def t2(a: str, b: str) -> str:
return b + a
@workflow
def wf_deck_normal(a: int, b: str) -> Tuple[int, str]:
x, y = t1(a=a)
d = t2(a=y, b=b)
return x, d
I think I get the problem, the running tasks shows the following, anyone know about it?
7/13/2023 9:16:22 AM UTC task submitted to K8s
7/13/2023 9:16:22 AM UTC Unschedulable:0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
Yicheng Lu
07/13/2023, 5:26 PMHank Huang
07/14/2023, 2:04 AMYicheng Lu
07/14/2023, 2:13 AMkubectl describe node
to see the actual cpu resources.Hank Huang
07/14/2023, 2:16 AMYicheng Lu
07/14/2023, 2:23 AMdisable_deck=True
, this may reduce the cpu usageHank Huang
07/14/2023, 2:25 AMdisable_deck=True
to test the result.Yicheng Lu
07/14/2023, 2:54 AM@task(
requests=Resources(mem="1Gi",cpu="1"),
limits=Resources(mem="1Gi",cpu="1"),
disable_deck=False,
)
Hank Huang
07/14/2023, 2:55 AMYicheng Lu
07/14/2023, 3:14 AMHank Huang
07/14/2023, 3:16 AMNoSuchKeyThe specified key does not exist.metadata/propeller/flytesnacks-development-fbf6961c9f62546baa08/n1/data/0/deck.htmlmy-s3-bucket/my-s3-bucket/metadata/propeller/flytesnacks-development-fbf6961c9f62546baa08/n1/data/0/deck.html17719D7997601E5385f2f3ba-30a3-46d3-b10b-a0caf4074cef
Yicheng Lu
07/14/2023, 3:17 AMHank Huang
07/14/2023, 3:21 AMYicheng Lu
07/14/2023, 3:22 AMHank Huang
07/14/2023, 3:22 AMYicheng Lu
07/14/2023, 3:22 AMHank Huang
07/14/2023, 3:22 AMYicheng Lu
07/14/2023, 3:23 AMHank Huang
07/14/2023, 3:26 AMYicheng Lu
07/14/2023, 3:28 AM# The following line is an example of how to install your own plugins. In this case, it demonstrates how to install the 'deck' plugin.
# RUN pip install -U git+<https://github.com/Yicheng-Lu-llll/flytekit.git@>"demo#egg=flytekitplugins-deck-standard&subdirectory=plugins/flytekit-deck-standard" # replace with your own repo and branch
Hank Huang
07/14/2023, 3:54 AMRUN pip install -U git+<https://github.com/hhcs9527/flytekit.git@>"master#egg=flytekitplugins-deck-standard&subdirectory=plugins/flytekit-deck-standard" plotly
Yicheng Lu
07/14/2023, 3:58 AMpip install plotly
Hank Huang
07/14/2023, 3:59 AMYicheng Lu
07/14/2023, 4:00 AMHank Huang
07/14/2023, 4:01 AMYicheng Lu
07/14/2023, 4:02 AMHank Huang
07/14/2023, 4:20 AMNoSuchKeyThe specified key does not exist.metadata/propeller/flytesnacks-development-f41ce8dcaf22f41819b7/n0/data/0/deck.htmlmy-s3-bucket/my-s3-bucket/metadata/propeller/flytesnacks-development-f41ce8dcaf22f41819b7/n0/data/0/deck.html1771A1115FF4F6E685f2f3ba-30a3-46d3-b10b-a0caf4074cef
Yicheng Lu
07/15/2023, 4:07 AMHank Huang
07/15/2023, 4:14 AMYicheng Lu
07/15/2023, 5:31 AMHank Huang
07/15/2023, 5:46 AM