flyte-org #ask-the-community

Frank Shen

10/10/2022, 7:00 PM

Hello, I am new to Flyte. I’ve setup flyte locally and started the demo cluster. I created a launch plan below. How do I register it and activate it so that it will show up in the UI? The found the documentation not clear about LP registration. Thank yiou!

Copy code

from flytekit import LaunchPlan, current_context, task, workflow

@task
def square(val: int) -> int:
    return val * val

@workflow
def my_wf(val: int) -> int:
    result = square(val=val)
    return result

default_lp = LaunchPlan.get_default_launch_plan(current_context(), my_wf)
square_3 = default_lp(val=3)

seunggs

10/11/2022, 3:23 AM

Hi, I’m just ran a workflow execution that outputs a numpy ndarray but I’m not sure how to retrieve the actual data? In the console, it just shows the s3 url with

.npy

extension:

seunggs

10/11/2022, 3:23 AM

Copy code

{
  "o0": {
    "type": "single (NumpyArray) blob",
    "uri": "<s3://sidetrek-flyte-cluster-flyte-bucket/j7/s8df0ape621iv5k8jz0t-n3-0/5a5182507b2e1d3759ca877860fb780c/5d8c4cacddf4ae178bc3987478b0cb24.npy>"
  }
}

seunggs

10/11/2022, 3:25 AM

I suppose you can open it with

np.load

after downloading the file, but is there any easier way to retrieve the data?

Adwitiya

10/11/2022, 7:14 AM

thanks @Samhita Alla, will run using the docker image

varsha Parthasarathy

10/11/2022, 1:53 PM

HI team, QQ, for flyte map tasks, we have can use resources to specify the requests and limits, is it also possible to pass in the node pool to run this task on?

Krishna Agarwal

10/12/2022, 10:42 AM

Hello 👋 Sorry for being impatient but can someone review my PR, It's been more than 3 days. https://github.com/unionai-oss/unionml/pull/187 cc @Eduardo Apolinario (eapolinario) and @Niels Bantilan

Chris Hairfield

10/12/2022, 9:26 PM

Hello 👋, I recently joined a team using Flyte, and I’m tasked with introducing Flyte’s native scheduler. I’ve gotten my code running in the

sandbox

environment with

--fast

registration, which is pretty awesome. But now I’m stuck trying to set inputs specific to developing in

sandbox

(e.g., hostname of an external service running on my local machine). Question: How do I enable a schedule with specified inputs that override

default_inputs

whenever the scheduler invokes my workflow? What I think I want is a commandline like:

Copy code

flytectl update launchplan -p {project} -d {domain} {lp_name} --version {version} --activate --execFile exec_spec.yaml

As a test, when I try to follow this line from the docs and omit a required input from the

LaunchPlan.get_or_create

call, then my schedule simply doesn’t run when actived:

Copy code

Launch plans do not necessarily contain the entire set of required workflow inputs

I hope I’m just missing something! Thanks.

SeungTaeKim

10/13/2022, 1:37 AM

Hello, Flyte Team flyte I would like to ask a best usecase of making flyte project. Those who are currently processing ML work in my company are using flyte as a ml pipeline. You know, some research engineers who have not so much known software engineering have learning curve to learning flyte. So, my team builds CI process with github action, the example image is below. research engineers do not need to know how to write flytectl codes. However, there is a problem. This CI process basically made on flyte project per github repo, which make three namespaces in kubernetes (development/staging/production) (My team guess it will be hard to manage those namespaces all) It is too much building namespace per projct, so I would like get solutions or usecase of flyte users. Thank you so much

KS Tarun

10/13/2022, 5:24 AM

Hi, We've tried to setup flyte on an EKS cluster. On checking with

kubectl get pods -n flyte

, the output is as shown below:

Copy code

NAME                                              READY   STATUS                   RESTARTS          AGE
flyte-deps-contour-contour-cd4d956d9-2xlwk        1/1     Running                  0                 19d
flyte-deps-contour-envoy-244xb                    2/2     Running                  0                 19d
flyte-deps-contour-envoy-rbwp7                    2/2     Running                  0                 19d
flyte-deps-kubernetes-dashboard-8b7d858b7-x9ncd   1/1     Running                  0                 19d
flyte-pod-webhook-85ff4b5f9d-t9l2f                1/1     Running                  0                 19d
flyteadmin-b46f578c7-qww88                        1/1     Running                  0                 17d
flyteconsole-77f665cc84-8gmv7                     1/1     Running                  0                 19d
flytepropeller-655664f9f9-6c69v                   1/1     Running                  0                 14d
flytepropeller-655664f9f9-l67tf                   0/1     Error                    0                 14d
flytepropeller-655664f9f9-zz2c2                   0/1     Error                    0                 19d
flytescheduler-6db897d545-mlntd                   0/1     ContainerStatusUnknown   1                 14d
flytescheduler-6db897d545-mmhtl                   1/1     Running                  563 (8m45s ago)   10d
minio-5cbbccf6cd-bm6ms                            1/1     Running                  0                 17d
postgres-565f59484-qcnjq                          1/1     Running                  0                 19d
syncresources-7b4b44f7cc-p4h9j                    1/1     Running                  0                 19d

But when we run any command to run/register a workflow

pyflyte run ........  etc.

, we're getting the below error:

Copy code

Traceback (most recent call last):
  File "/home/tarun/.local/bin/pyflyte", line 5, in <module>
    from flytekit.clis.sdk_in_container.pyflyte import main
  File "/home/tarun/.local/lib/python3.8/site-packages/flytekit/__init__.py", line 259, in <module>
    load_implicit_plugins()
  File "/home/tarun/.local/lib/python3.8/site-packages/flytekit/__init__.py", line 253, in load_implicit_plugins
    discovered_plugins = entry_points(group="flytekit.plugins")
TypeError: entry_points() got an unexpected keyword argument 'group'

Can someone explain, what might be the cause of this error ?

Yash Panchwatkar

10/13/2022, 11:43 AM

hello guys i am trying to install flyte on aws eks i have created the initial cluster and nodes also i have created db as well but db connection is not getting establish some one please help

Yash Panchwatkar

10/13/2022, 12:21 PM

Thanks @Shivay Lamba issue is resolved.. will ping if there is any Thanks community

Yash Panchwatkar

10/13/2022, 1:30 PM

aws-load-balancer-controller has not scheduled or some one please verify the same weather I am on right path or not

Sanjay Chouhan

10/13/2022, 4:32 PM

We've tried to setup flyte on an EKS cluster. But getting some issue with Ingress as attached.

Yee

10/13/2022, 5:22 PM

do you have any error messages/stack traces you can paste?

Chandramoulee K V

10/14/2022, 5:39 AM

Hi, I have a private ECR image and i tried binding it to a task and register the workflow . It worked and now the task is binded with the private ECR image but when i run the workflow i get this following error:

containers with unready status: [a4tbv4qblclb2d8kdqg6-n1-0]|Back-off pulling image {ECR IMAGE}

What all accesses should be provided to the ECR image? Any idea on this

Jun Kwan

10/14/2022, 11:34 AM

Hi guys, I am currently following tutorial https://docs.flyte.org/projects/cookbook/en/stable/auto/integrations/flytekit_plugins/onnx_examples/pytorch_onnx.html However, when I want to save image to my current directory by changing to code to

Copy code

final_img = Image.merge(
        "YCbCr",
        [
            img_out_y,
            img_cb.resize(img_out_y.size, Image.BICUBIC),
            img_cr.resize(img_out_y.size, Image.BICUBIC),
        ],
    ).convert("RGB")

    img_path = (
        Path("cat_superres_with_ort.jpg")
    )
    final_img.save(img_path)

Where can I find cat_superres_with_ort.jpg? Thanks!

Sanjay Chouhan

10/14/2022, 1:43 PM

Getting SSL error while running pyflyte register command for an AWS EKS setup. ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1131)

Sky Wolf

10/15/2022, 2:50 AM

Hello I was wondering if anyone could guide me through some basic open source contributions to flyte so I can get a handle on things.

Sky Wolf

10/15/2022, 2:51 AM

Please dm me if possible!

Yash Panchwatkar

10/15/2022, 6:47 AM

hello... i have a query ... is it okay to remove redishosturl from values-eks.yaml

Yash Panchwatkar

10/15/2022, 6:51 AM

btw all steps got well executed until now....

Yash Panchwatkar

10/15/2022, 6:53 AM

anyone....

Yash Panchwatkar

10/15/2022, 6:57 AM

Thanks @Samhita Alla for reply...🙏

Yash Panchwatkar

10/15/2022, 7:20 AM

hello guys I am trying to install Flyte with Flyte AWS Scheduler executed the command by replacing values in vales-eks.yaml here 2 pods were goes in the crashloopbackoff can any one help

Yash Panchwatkar

10/15/2022, 7:21 AM

message has been deleted

Yash Panchwatkar

10/15/2022, 7:28 AM

here are the logs for same

Yash Panchwatkar

10/15/2022, 7:28 AM

message has been deleted

Thusal Ranawaka

10/15/2022, 10:53 AM

Yash Panchwatkar

10/16/2022, 2:37 PM

hello community.. does flyte have the AWS SQS dependency?? @anyone