https://flyte.org
Join Slack
Hi community, I would like to contribute to the flytekit repo. When I following this <doc>, and try ...
# flyte-support
r
Hi community, I would like to contribute to the flytekit repo. When I following this doc, and try testing it with sandbox. But I’ve encounter the problem for the task would never stop, even though I tried to re-build the image / restart the cluster, and test on my master branch. Still the same. Any good suggestions? Here is my dockerFile
Copy code
FROM python:3.9-slim-buster
USER root
WORKDIR /root
ENV PYTHONPATH /root
RUN apt-get update && apt-get install build-essential -y
RUN apt-get install git -y
# The following line is an example of how to install your own plugins. In this case, it demonstrates how to install the 'deck' plugin.
# RUN pip install -U git+<https://github.com/Yicheng-Lu-llll/flytekit.git@>"demo#egg=flytekitplugins-deck-standard&subdirectory=plugins/flytekit-deck-standard" # replace with your own repo and branch
RUN pip install -U git+<https://github.com/hhcs9527/flytekit.git@master>
ENV FLYTE_INTERNAL_IMAGE "localhost:30000/flytekit:master"
Here is my workflow (relatively short, so I just paste it here)
Copy code
import typing
from typing import Tuple
from flytekit import task, workflow


@task(disable_deck=False)
def t1(a: int) -> typing.NamedTuple("OutputsBC", t1_int_output=int, c=str):
    return a + 2, "world"


@task(disable_deck=False)
def t2(a: str, b: str) -> str:
    return b + a


@workflow
def wf_deck_normal(a: int, b: str) -> Tuple[int, str]:
    x, y = t1(a=a)
    d = t2(a=y, b=b)
    return x, d
I think I get the problem, the running tasks shows the following, anyone know about it?
Copy code
7/13/2023 9:16:22 AM UTC task submitted to K8s

7/13/2023 9:16:22 AM UTC Unschedulable:0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
a
Seem like you do not have enough cpu resource for you task pod to run. the k3s will use whatever CPU resources are available on the host system.
r
Hi @acceptable-window-92672 , I think you are right.
But I am still figuring out how to fix it.
a
Could you increase the num of cpu for you vm, or eliminate other processes that may be consuming CPU resources. You can use 
kubectl describe node
to see the actual cpu resources.
r
looks like we are able to allocate?
Hi @acceptable-window-92672, just ask chatgpt, does the following mean my cpu resource is running out? • `ephemeral-storage: 252895796231`: The node has 252895796231 bytes (or about 252.9 GB) of ephemeral storage available. Ephemeral storage refers to the temporary storage available on the node. • `hugepages-1Gi: 0`: There are no 1GB huge pages available on this node. Huge pages are large memory pages that can be used to improve performance in specific workloads. • `hugepages-2Mi: 0`: There are no 2MB huge pages available on this node.
Here is the current cpu usage
a
Would you mind setting 
disable_deck=True
, this may reduce the cpu usage
r
Sure! However, the issue I am addressing needs 
disable_deck=True
to test the result.
a
Maybe try set:
Copy code
@task(
    requests=Resources(mem="1Gi",cpu="1"),
    limits=Resources(mem="1Gi",cpu="1"),
    disable_deck=False,
)
r
ok! thanks
it work! looks like the cpu is running out.
thanks for the help on the setting!
Hi @acceptable-window-92672 I turn off desk_disable for the performance measurement mentioned in this pr #1581 Measuring the time taken for each component when executing a task. but looks like the attachment not showing the result you did, do you know how to let it show?
a
The flyte deck button~
r
I see, so the result would upload it to s3? Looks like I am unable to test it on flyte-deployment, may be some s3 setting Here is what I have.
Copy code
NoSuchKeyThe specified key does not exist.metadata/propeller/flytesnacks-development-fbf6961c9f62546baa08/n1/data/0/deck.htmlmy-s3-bucket/my-s3-bucket/metadata/propeller/flytesnacks-development-fbf6961c9f62546baa08/n1/data/0/deck.html17719D7997601E5385f2f3ba-30a3-46d3-b10b-a0caf4074cef
a
Looks like deck.html is not in you S3
If you are using flytectl demo, You can access the mino: Access the minio console at: http://localhost:30080/minio/login. The default credentials are: - Username: ``minio`` - Password: ``miniostorage``
r
I am using that, should I do something to setup the minio with my env? I just login and re-run => but still no such key
a
r
no
a
You do not need to
r
let me run the demo code, I think it would be better
a
Sure
In principle, as long as you set disable_deck=False, there should have a deck.html
should see deck.html in this path:
r
looks like we need flytekitplugins => do you know where to install it?
a
Copy code
# The following line is an example of how to install your own plugins. In this case, it demonstrates how to install the 'deck' plugin.
# RUN pip install -U git+<https://github.com/Yicheng-Lu-llll/flytekit.git@>"demo#egg=flytekitplugins-deck-standard&subdirectory=plugins/flytekit-deck-standard" # replace with your own repo and branch
r
thanks! but I think we need to install plotly ModuleNotFoundError: No module named ‘plotly’ => not sure how you handle this in sand box? here is how I do.
Copy code
RUN pip install -U git+<https://github.com/hhcs9527/flytekit.git@>"master#egg=flytekitplugins-deck-standard&subdirectory=plugins/flytekit-deck-standard" plotly
still not fix the problem
a
Copy code
pip install plotly
r
you mean run in my running machine? (I’ve already installed)
a
in you dockerFile, You could: RUN pip install plotly
r
yeah… I think I should add a new line for it.
let me re-build it
a
sure
r
Thanks for the help! but I am still get (even I ran the sample code)
Copy code
NoSuchKeyThe specified key does not exist.metadata/propeller/flytesnacks-development-f41ce8dcaf22f41819b7/n0/data/0/deck.htmlmy-s3-bucket/my-s3-bucket/metadata/propeller/flytesnacks-development-f41ce8dcaf22f41819b7/n0/data/0/deck.html1771A1115FF4F6E685f2f3ba-30a3-46d3-b10b-a0caf4074cef
However my minio looks a bit different from yours
I have only deployment folder not metadata, Any good suggestion?
a
I am not sure what happened actually. Maybe try running the task with the default Flytekit image to see if you can see the deck.html.
r
I see, thanks!
🙌 1
By the way, @acceptable-window-92672 are you familiar with the dependency setting in flyte? I am import mashumaro to speed up the sterilize / de sterilize process. Here is the related PR(https://github.com/flyteorg/flytekit/pull/1735) The setup.py part is already set! But I found that there are many *requirement.txt need to add such information, that’s why I try to add it through the pip-compile command. Take lib marshmallow as example, not sure if I should do the same in my PR. Any good suggestion?
a
Sorry, I am not familiar with the dependency setting😞.
r
OK! thanks for the anwsering!
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by