https://flyte.org logo
Join the conversationJoin Slack
Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
sig-large-models
workflow-building-ui-proj
writing-w-sfloris
Powered by Linen
ask-the-community
  • f

    Frank Shen

    11/15/2022, 12:12 AM
    Hello, I am trying to return a flytekit.core.resources.Resources object in a task.
    @task
    def get_xgb_task_resource_request(df: pd.DataFrame) -> Resources:
    ...
  • f

    Frank Shen

    11/15/2022, 12:12 AM
    Unfortunately, I got error:
    flytekit/core/type_engine.py", line 319, in get_literal_type
        f"Dataclass {t} should be decorated with @dataclass_json to be " f"serialized correctly"
    AssertionError: Dataclass <class 'flytekit.core.resources.Resources'> should be decorated with @dataclass_json to be serialized correctly
    • 1
    • 1
  • f

    Frank Shen

    11/15/2022, 12:13 AM
    Since I cannot change flytekit code, what is my solution?
  • f

    Frank Shen

    11/15/2022, 12:14 AM
    What I intent to do is to request resource overrides if the dataset size is too big.
    s
    e
    • 3
    • 22
  • v

    varsha Parthasarathy

    11/15/2022, 5:45 AM
    Hi team . Does flyte provide a decorator of sorts which would output time it takes to run each task or will these be updated in backend DB?
    s
    • 2
    • 2
  • v

    varsha Parthasarathy

    11/15/2022, 3:21 PM
    Enabling task cache for Map tasks throws the error in 🧵 .. Is this an issue with my config or a real bug? Thanks 🙏
    d
    k
    a
    • 4
    • 73
  • a

    Aphra Bloomfield

    11/15/2022, 8:32 PM
    Hey all, does the Sandbox have an external events flow you can hook into? I would like to be able to capture job metadata in a proof of concept.
    k
    • 2
    • 8
  • a

    Aphra Bloomfield

    11/16/2022, 12:50 AM
    I’m also wondering if it’s possible to share tasks between workflows. For, if I have a
    github.fetch_repos
    task, can i employ it in
    snyk.scan_repos
    ?
    k
    • 2
    • 3
  • l

    Louis DiNatale

    11/16/2022, 1:17 AM
    Does anyone know if you can get more robust logs than “Dynamic workflow failed”
    s
    d
    • 3
    • 3
  • k

    Kamakshi Muthukrishnan

    11/16/2022, 7:22 AM
    I am trying to execute the knn_classifier.py but getting the below error...flytesnacks/cookbook/integrations/flytekit_plugins/modin_examples/knn_classifier.py /
  • k

    Kamakshi Muthukrishnan

    11/16/2022, 7:22 AM
    File "knn.py", line 105, in pipeline accuracy=calc_accuracy(split_data_vals.test_labels,predicted_vals_output) File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages/flytekit/core/base_task.py", line 283, in call return flyte_entity_call_handler(self, *args, **kwargs) File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages/flytekit/core/promise.py", line 1003, in flyte_entity_call_handler f"When calling tasks, only keyword args are supported. " flytekit.exceptions.user.FlyteAssertion: When calling tasks, only keyword args are supported. Aborting execution as detected 2 positional args (Promise(node:n0.test_labels), Promise(node:n1.o0))
  • k

    Kamakshi Muthukrishnan

    11/16/2022, 7:22 AM
    what needs to be done here ?
    s
    • 2
    • 7
  • k

    KS Tarun

    11/16/2022, 8:27 AM
    Hi, I'm able to run a certain model on an instance with 32 GiB RAM, but the same model when run with flyte is throwing OOM Error on the same instance type and for the same input parameters. Any thoughts on why this might be happening ?
    s
    • 2
    • 6
  • b

    Byron Murillo

    11/16/2022, 2:47 PM
    Hello guys, does anybody have experience with the deployment on aws, I’m currently working with the manual guide for EKS and automate most of the steps except the ALB, when I’m push the chart values I have this error: Note: the private subnets for the eks nodes have the tags from the guide.
    d
    • 2
    • 5
  • r

    Robin Eklund

    11/16/2022, 4:14 PM
    Hey! Do anyone know if it is possible/where to set session duration on the role which is assumed when running a workflow? I have a task which is running for some time (~5min) and then i get this error:
    flytekit.exceptions.scopes.FlyteScopedUserException: An error occurred (ExpiredToken) when calling the AssumeRole operation: The security token included in the request is expired
    and after a while it also prints this error:
    Called process exited with error code: 1.  Stderr dump:\n\nb'upload failed: ../../tmp/flyte-kvw3xxto/sandbox/local_flytekit/engine_dir/error.pb to s3://<s3_bucket>/metadata/propeller/<project_name>-development-<execution_id>/n1/data/0/error.pb An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
    I trigger this workflow with a Launch Plan which specify a specific service account (not default). It was created like this:
    security_context = SecurityContext(
            run_as=Identity(
                iam_role=None,
                k8s_service_account=f"my-aws-role",
            ),
        )
    
    LaunchPlan.get_or_create(
        name="my_lp",
        workflow=my_wf,
        security_context=security_context,
    )
    Anyone who faced the same error before?
    s
    p
    a
    • 4
    • 24
  • t

    Tarmily Wen

    11/16/2022, 5:24 PM
    Hi, I followed the guide for setting up flyte on GKE. I tried port forwarding the console to my local machine for testing purposes, but that resulted in an error. The command I used is
    kubectl -n flyte port-forward service/flyteconsole 8080:80
    The only thing I didn't follow in the guide is having a domain name. We don't want to expose this externally. But I am surprised the port forwarding doesn't work. Is the domain name used beyond just the ingress point?
    c
    f
    h
    • 4
    • 9
  • a

    Aphra Bloomfield

    11/16/2022, 5:38 PM
    Is it possible to import tasks from their own Python modules to get separation of concerns? I have tried this, but get import errors when deploying my workflows.
    y
    • 2
    • 12
  • s

    Sujith Samuel

    11/16/2022, 5:54 PM
    #ask-the-community I am having issues with flyte propeller picking up stale resources and giving error. In my k8s cluster, I have deleted the flyte installation and then reinstalled flyte again. But the flytepropeller still shows the logs from the workflows that failed in the previous installation. When i try to submit a new workflow I get the below error in the UI Request failed with status code 500 failed to create workflow in propeller create not allowed while custom resource definition is terminating What could the reason be for this.?
    k
    d
    • 3
    • 7
  • l

    Laura Lin

    11/16/2022, 9:54 PM
    how do you chain a task to run after a map_task inside a dynamic workflow
    y
    • 2
    • 1
  • a

    Aphra Bloomfield

    11/17/2022, 12:18 AM
    Would anyone be able to point me in the direction of where Flyte’s DAG model lives in the source code repository?
    d
    • 2
    • 14
  • k

    KS Tarun

    11/17/2022, 1:58 PM
    Hi, In a model pipeline, the tasks are taking longer time to run than usual. A func. which would take ~2 mins to run is taking 35-40 min when using flyte. When I checked the cloudwatch logs, I see that there is a dormant period in every task which is taking up >98% of run time. In the image attached below, after 18 min (@ 17:27), the actual process is running and it takes seconds to complete. In between the initialization and the actual code run, when I checked the logs, these are available:
    flyteadmin-c9cccb94d-fcq5s_flyte_flyteadmin-51bb75904a0581e506a547fc4e434e29b564ec2bcf9d41c343575ffdb29c5bc4
    flytepropeller-cbf5cb9f-9kn76_flyte_flytepropeller-b5c4de694b13a1687857b30012166b8218f8ddd20e141f94492b441fa7fdf425
    flyteconsole-5c7b456999-jqt42_flyte_flyteconsole-7f461bce1be5bcf45010474a20c7b6602ed0892374d968838ca2ed72f9c1d878
    Are these contributing to the increased runtime ?
    d
    k
    +2
    • 5
    • 16
  • b

    Byron Murillo

    11/17/2022, 7:27 PM
    Hello, I followed the guide for EKS and I have the following error, refering to the last step, the yaml where is located? I think is for that reason its not working, because I make a portforward to open the console, I can`t enter thought the alb link
    d
    • 2
    • 1
  • s

    Sujith Samuel

    11/18/2022, 2:10 PM
    #ask-the-community Hello All, My flyte workflows are running on a k8s cluster. Workflow was 6 nodes and each node requests 1 CPU. What happens is that by the end of the workflow 6 nodes are requesting 6 CPU's. The workflow succeeds but the CPU is not released from request. This means that once I run this workflow like 20 times, my cluster is already at 120 CPU's and after that the jobs get OOMKilled. Has anyone gone through this. How to get out of this pickle....
    k
    k
    • 3
    • 5
  • j

    Jay Ganbat

    11/18/2022, 6:07 PM
    So i have a workflow with dict input with type specified so its
    map
    type in FlyteConsole. However i cannot give it an empty dict, flyteconsole is not just wont submit the job and no error msg shows up, like shown below i kept clicking on
    Launch
    but nothing happens. this LP has default input as empty dict, does flyte not accept empty dict
    k
    • 2
    • 3
  • f

    Felix Ruess

    11/18/2022, 8:03 PM
    Is there a way to delete flyte tasks and workflows again?
    flytectl delete
    doesn't seem to have that...
    k
    • 2
    • 6
  • f

    Felix Ruess

    11/18/2022, 8:09 PM
    And how can I add a description to my flyte tasks and workflows.... I can see in flyte console that this should be possible, but couldn't find any docs on how to add the description...
    k
    • 2
    • 5
  • b

    Byron Murillo

    11/18/2022, 8:18 PM
    Hello guys, in the last step of the AWS manual guide after apply the changes on the config.yaml using flytectl to add the dns, I have this error on the ingresses
    s
    a
    +2
    • 5
    • 12
  • a

    Aphra Bloomfield

    11/18/2022, 10:42 PM
    Hey channel, I’m getting the following sandbox error when trying to start a workflow from the UI:
    Request failed with status code 500 failed to create workflow in propeller namespaces "sat-development" not found
    . Not sure why this is coming up, as the project
    sat
    exists and so should the
    development
    namespaces…
    s
    • 2
    • 1
  • t

    Tarmily Wen

    11/18/2022, 11:43 PM
    I followed the guide to installing MPI Operator, and tried running the example workflows for MPI. I run into this error.
    k
    k
    +3
    • 6
    • 21
  • r

    Rahul Mehta

    11/19/2022, 12:09 AM
    Is there a way to serialize workflows without docker? Ideally we’d like to serialize workflows using our regular python environment, since we can guarantee that the deps at any given point will match the container image due to our build system
    k
    • 2
    • 5
Powered by Linen
Title
r

Rahul Mehta

11/19/2022, 12:09 AM
Is there a way to serialize workflows without docker? Ideally we’d like to serialize workflows using our regular python environment, since we can guarantee that the deps at any given point will match the container image due to our build system
k

Ketan (kumare3)

11/19/2022, 12:15 AM
yes absolutely
you dont have to use docker
pyflyte serialize should work without docker
and pyflyte register should work without docker
you just have to pass the path to python inside the container
View count: 2