Hello, I am trying to return a flytekit.core.resources.Resources object in a task.

@task
def get_xgb_task_resource_request(df: pd.DataFrame) -> Resources:
...

Unfortunately, I got error:

flytekit/core/type_engine.py", line 319, in get_literal_type
    f"Dataclass {t} should be decorated with @dataclass_json to be " f"serialized correctly"
AssertionError: Dataclass <class 'flytekit.core.resources.Resources'> should be decorated with @dataclass_json to be serialized correctly

Since I cannot change flytekit code, what is my solution?

What I intent to do is to request resource overrides if the dataset size is too big.

Hi team . Does flyte provide a decorator of sorts which would output time it takes to run each task or will these be updated in backend DB?

Enabling task cache for Map tasks throws the error in 🧵 .. Is this an issue with my config or a real bug? Thanks 🙏

Hey all, does the Sandbox have an external events flow you can hook into? I would like to be able to capture job metadata in a proof of concept.

I’m also wondering if it’s possible to share tasks between workflows. For, if I have a

github.fetch_repos

task, can i employ it in

snyk.scan_repos

?

Does anyone know if you can get more robust logs than “Dynamic workflow failed”

I am trying to execute the knn_classifier.py but getting the below error...flytesnacks/cookbook/integrations/flytekit_plugins/modin_examples/knn_classifier.py /

File "knn.py", line 105, in pipeline accuracy=calc_accuracy(split_data_vals.test_labels,predicted_vals_output) File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages/flytekit/core/base_task.py", line 283, in call return flyte_entity_call_handler(self, *args, **kwargs) File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages/flytekit/core/promise.py", line 1003, in flyte_entity_call_handler f"When calling tasks, only keyword args are supported. " flytekit.exceptions.user.FlyteAssertion: When calling tasks, only keyword args are supported. Aborting execution as detected 2 positional args (Promise(node:n0.test_labels), Promise(node:n1.o0))

what needs to be done here ?

Hi, I'm able to run a certain model on an instance with 32 GiB RAM, but the same model when run with flyte is throwing OOM Error on the same instance type and for the same input parameters. Any thoughts on why this might be happening ?

Hello guys, does anybody have experience with the deployment on aws, I’m currently working with the manual guide for EKS and automate most of the steps except the ALB, when I’m push the chart values I have this error: Note: the private subnets for the eks nodes have the tags from the guide.

Hey! Do anyone know if it is possible/where to set session duration on the role which is assumed when running a workflow? I have a task which is running for some time (~5min) and then i get this error:

flytekit.exceptions.scopes.FlyteScopedUserException: An error occurred (ExpiredToken) when calling the AssumeRole operation: The security token included in the request is expired

and after a while it also prints this error:

Called process exited with error code: 1.  Stderr dump:\n\nb'upload failed: ../../tmp/flyte-kvw3xxto/sandbox/local_flytekit/engine_dir/error.pb to s3://<s3_bucket>/metadata/propeller/<project_name>-development-<execution_id>/n1/data/0/error.pb An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

I trigger this workflow with a Launch Plan which specify a specific service account (not default). It was created like this:

security_context = SecurityContext(
        run_as=Identity(
            iam_role=None,
            k8s_service_account=f"my-aws-role",
        ),
    )

LaunchPlan.get_or_create(
    name="my_lp",
    workflow=my_wf,
    security_context=security_context,
)

Anyone who faced the same error before?

Hi, I followed the guide for setting up flyte on GKE. I tried port forwarding the console to my local machine for testing purposes, but that resulted in an error. The command I used is

kubectl -n flyte port-forward service/flyteconsole 8080:80

The only thing I didn't follow in the guide is having a domain name. We don't want to expose this externally. But I am surprised the port forwarding doesn't work. Is the domain name used beyond just the ingress point?

Is it possible to import tasks from their own Python modules to get separation of concerns? I have tried this, but get import errors when deploying my workflows.

#ask-the-community I am having issues with flyte propeller picking up stale resources and giving error. In my k8s cluster, I have deleted the flyte installation and then reinstalled flyte again. But the flytepropeller still shows the logs from the workflows that failed in the previous installation. When i try to submit a new workflow I get the below error in the UI Request failed with status code 500 failed to create workflow in propeller create not allowed while custom resource definition is terminating What could the reason be for this.?

how do you chain a task to run after a map_task inside a dynamic workflow

Would anyone be able to point me in the direction of where Flyte’s DAG model lives in the source code repository?

Hi, In a model pipeline, the tasks are taking longer time to run than usual. A func. which would take ~2 mins to run is taking 35-40 min when using flyte. When I checked the cloudwatch logs, I see that there is a dormant period in every task which is taking up >98% of run time. In the image attached below, after 18 min (@ 17:27), the actual process is running and it takes seconds to complete. In between the initialization and the actual code run, when I checked the logs, these are available:

flyteadmin-c9cccb94d-fcq5s_flyte_flyteadmin-51bb75904a0581e506a547fc4e434e29b564ec2bcf9d41c343575ffdb29c5bc4

flytepropeller-cbf5cb9f-9kn76_flyte_flytepropeller-b5c4de694b13a1687857b30012166b8218f8ddd20e141f94492b441fa7fdf425

flyteconsole-5c7b456999-jqt42_flyte_flyteconsole-7f461bce1be5bcf45010474a20c7b6602ed0892374d968838ca2ed72f9c1d878

Are these contributing to the increased runtime ?

Hello, I followed the guide for EKS and I have the following error, refering to the last step, the yaml where is located? I think is for that reason its not working, because I make a portforward to open the console, I can`t enter thought the alb link

#ask-the-community Hello All, My flyte workflows are running on a k8s cluster. Workflow was 6 nodes and each node requests 1 CPU. What happens is that by the end of the workflow 6 nodes are requesting 6 CPU's. The workflow succeeds but the CPU is not released from request. This means that once I run this workflow like 20 times, my cluster is already at 120 CPU's and after that the jobs get OOMKilled. Has anyone gone through this. How to get out of this pickle....

So i have a workflow with dict input with type specified so its

map

type in FlyteConsole. However i cannot give it an empty dict, flyteconsole is not just wont submit the job and no error msg shows up, like shown below i kept clicking on

Launch

but nothing happens. this LP has default input as empty dict, does flyte not accept empty dict

Is there a way to delete flyte tasks and workflows again?

flytectl delete

doesn't seem to have that...

And how can I add a description to my flyte tasks and workflows.... I can see in flyte console that this should be possible, but couldn't find any docs on how to add the description...

Hello guys, in the last step of the AWS manual guide after apply the changes on the config.yaml using flytectl to add the dns, I have this error on the ingresses

Hey channel, I’m getting the following sandbox error when trying to start a workflow from the UI:

Request failed with status code 500 failed to create workflow in propeller namespaces "sat-development" not found

. Not sure why this is coming up, as the project

sat

exists and so should the

development

namespaces…

I followed the guide to installing MPI Operator, and tried running the example workflows for MPI. I run into this error.