Hello everyone! Is there any way of mounting a data volume in a ContainerTask? I need to provide a directory with an arbitrary number of files, read them, and apply some processing. The way I do it ususally is by mounting a volume when running docker run, but now I’m transforming it to a Flyte pipeline and have had some issues. What I’m trying to do is to pass it as a FlyteDirectory, but it’s not working as I expected. I’m doing it like this:

ContainerTask(
    name="my_container",
    image="localhost:30000/my_container:latest",
    input_data_dir="/input",
    output_data_dir="/output",
    inputs=kwtypes(
        config_file=FlyteFile,
        my_dir=FlyteDirectory,),
    outputs=kwtypes(out=FlyteDirectory),
    command=[
        "python app.py --input_dir=/input/my_dir --config=/input/config_file"
    ]
)

I was expecting the

/input

directory to contain a dir called

my_dir

and a file called

config_file

. However, it seems to be empty. Can someone help me figure out how to mount data? Thanks 🙌

Is there any examples of using ContainerTasks with FlyteFile and FlyteDirectory? I’m trying to use a ContainerTask that has a FlyteFile as an input, and the output is a directory but struggling to make anything work. Any pointers?

Does the sandbox cluster support running in a podman rootless cluster?

Why does Recover rerun map tasks that succeeded? Shouldn't it only rerun tasks that failed.

Hi there, I've just started installing flyte (the binary) on an existing EKS cluster and have been walking through the getting started docs. I'm port-forwarding from the flyte-binary service on the cluster, have created a test project, and have created a local workflow script containing this example. However, when I run:

pyflyte run -p testflyte --remote example.py training_workflow --hyperparameters '{"C": 0.1}'

I get a 403 error. There's a signed url, but this is rejected by the metadata bucket. The cluster itself has a flyte service account with read/write access to the bucket, but the above is all happening locally. Any ideas how I can debug this as I'm currently a bit stumped! Thanks a lot 🙂

Hey we ran into an interesting problem today... we have a map task that calls a C binary that uses a large reference dataset from disk to do some computations on a new smaller dataset. It keeps failing with a rather mysterious error:

[0]: code:"ResourceDeletedExternally" message:"resource not found, name [e2e-workflows-development/fb2xnzxy-n2-0-0]. reason: pods \"fb2xnzxy-n2-0-0\" not found"

We then checked control plan logs and they suggested the pod was being evicted due to memory pressure (137 = k8s OOM status code):

"containerStatuses": [
                {
                    "name": "fb2xnzxy-n2-0-0",
                    "state": {
                        "terminated": {
                            "exitCode": 137,
....

However when we look at grafana, we see that memory used is really low, way below requests/limits... however, we found that the memory cache was quite high. We then found a k8s issue about memory cache being incorrectly counted as "used" memory by kubelet when it looks at memory pressure. Note quite a flyte issue, more of a k8s issue, but the log was a bit mysterious and we're still figuring out resolution.

Hello, I have a flyte cluster running in eks with all

flyte-*

pods running in

test-flyte

namespace instead of default

flyte

namespace. I am able to create projects and register workflows, but when i try to execute the workflow from the console it goes into an

Unknown

state.

Hi, Flyte Team! I wonder flyte support Grafana Logging with Loki in Flyte console. My team currently is using Kibana (EFK), but we consider it to replace Loki.

I'm on Windows 10. I've installed

flytekit

using:

pip install flytekit

tried to reinstall it, used conda to install it, removed it and then installed it again using pip. But when I use

flytekit status

on my vscode terminal I get this error:

'flytekit' is not recognized as an internal or external command,
operable program or batch file.

I've added these PATHS to my environment variables [the image is attached] Can anyone please give me a clue? Why is

flytekit

or any of its features like

pyflyte

is not recognized on my terminal (either vscode or cmd)? Any help would be really appreciated. Thanks.

I'm getting some unexpected results using the map_task and want to make sure I'm using it correctly

map_task(move_and_split_txns, concurrency=1)(
    move_and_split_txns_config_file=config_files
).with_overrides(retries=5)

config_files

is a list with 3 FlyteFiles returned from a dependent task

move_and_split_txns

is a task that reads the FlyteFile and does the heavy lifting The nodes I was using didn't have enough memory and the

move_and_split_txns

tasks all got oomKilled by kubernetes. The

map_task

says it completed successfully with no reties I would expect the map_task to at least fail. What am I doing wrong?