Hi there, does anyone have any tips on debugging why workflows are stuck in queue indefinitely?

Hey, I just noticed there is a single binary deployment available for Flyte. We are looking to migrate our current deployment to a new EKS cluster, and I was thinking of deploying it using a single binary as looks awesome! We are currently leveraging OAuth2 authentication, based on the docs. However, with the new binary I would imagine the auth setup to be simplified, but the docs for single cluster production-grade deployment yields a 404 in terms of authentication (https://docs.flyte.org/en/latest/deployment/deployment/cloud_production.html#authentication). I would assume the app for flytepropeller is obsolete for binary deployments as there is no need for authenticated requests between the flyte components? Is there an updated guide that we can use?

Hey all, I'm trying to experiment with "fast registration" via pyflyte but am unsuccessful and having a hard time finding docs that work. The docs tell me to use

--fast

in conjunction with

--image

to do a fast register of local code that will be "overlayed" onto to the specified image (as I understand it). But in my trials I find that: 1. --fast is not a recognized option, eventually realized it is a command (without dashes), are the docs are wrong? 2. Then it complains "No such option --image". Again, are docs are wrong? Or I'm misreading something. I was trying to modify a command that works without "fast" into one that runs with "fast" to see if I can avoid building and pushing a new image (the image I specify in this case already exists). The command looks like this, where I've inserted $FAST_REG into a command that otherwise works to serialize against a given image:

pyflyte --pkgs plaster.genv2 serialize $FAST_REG --image <my-ecr-registry-path>/<my-image-name>:<tag> --local-source-root . workflows -f /tmp/_pb_outputs

...where the $FAST_REG is where I'm attempting to insert "fast" and run my local code overlayed with what is in the image I'm specifying. I'm probably misunderstanding something, and the docs aren't helping me. Thanks for any tips!

Hello team, While applying a map function on spark rdd, job failed unexpectedly. flyte Spark job is executed on local machine. Any suggestion on how to resolve this failure.

flytekit.exceptions.scopes.FlyteScopedUserException: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.runJob.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 3) (localhost executor driver): org.apache.spark.SparkException: Python worker exited unexpectedly (crashed)


INFO:py4j.clientserver:Closing down clientserver connection

Hi team.. I’m seeing my map tasks get evicted from k8s even with the

interruptible=False

flag.. which ends up dragging the task on much longer than it should (since it retries). Is there something different for map tasks?

Hi, I am getting error while executing ray with optuna in flyte.

We're aware of an issue with the

flyte-core

helm chart version

1.1.0

(which is the latest release). This is impacting all scenarios, including flyte deploys and also sandbox. Fix coming up shortly.

Anyone know a good way to do non-grid (eg. some kind of bayesian) hyperparameter optimization with Flyte, with multiple trials in parallel? (ie. is there some library that makes this easy or do I have to implement most of the optimization stuff myself? eg. a library which spits out parameters to try and I give it back the results would be pretty easy to use with Flyte, rather than the library calling an objective function like hyperopt does)

Hey, I made a PR for a polars plugin (we're using that and converting from/to pandas so far, which wasted a lot of memory hence this plugin) based on the Modin plugin. Do I need to build/upload the pypi package myself or do you have some automation?

When using the API, what is the format for specifying filters. ex: I want to fetch a project with a certain name. The API docs all say:

Indicates a list of filters passed as string. More info on constructing filters : +optional.

but I don't see anywhere where it explains the structure of the filters