Is there any examples of using ContainerTasks with FlyteFile and FlyteDirectory? I’m trying to use a ContainerTask that has a FlyteFile as an input, and the output is a directory but struggling to make anything work. Any pointers?

Does the sandbox cluster support running in a podman rootless cluster?

Hi there, I've just started installing flyte (the binary) on an existing EKS cluster and have been walking through the getting started docs. I'm port-forwarding from the flyte-binary service on the cluster, have created a test project, and have created a local workflow script containing this example. However, when I run:

pyflyte run -p testflyte --remote example.py training_workflow --hyperparameters '{"C": 0.1}'

I get a 403 error. There's a signed url, but this is rejected by the metadata bucket. The cluster itself has a flyte service account with read/write access to the bucket, but the above is all happening locally. Any ideas how I can debug this as I'm currently a bit stumped! Thanks a lot 🙂

Why does Recover rerun map tasks that succeeded? Shouldn't it only rerun tasks that failed.

Hello, I have a flyte cluster running in eks with all

flyte-*

pods running in

test-flyte

namespace instead of default

flyte

namespace. I am able to create projects and register workflows, but when i try to execute the workflow from the console it goes into an

Unknown

state.

I'm getting some unexpected results using the map_task and want to make sure I'm using it correctly

map_task(move_and_split_txns, concurrency=1)(
    move_and_split_txns_config_file=config_files
).with_overrides(retries=5)

config_files

is a list with 3 FlyteFiles returned from a dependent task

move_and_split_txns

is a task that reads the FlyteFile and does the heavy lifting The nodes I was using didn't have enough memory and the

move_and_split_txns

tasks all got oomKilled by kubernetes. The

map_task

says it completed successfully with no reties I would expect the map_task to at least fail. What am I doing wrong?

Hello everyone! Is there any way of mounting a data volume in a ContainerTask? I need to provide a directory with an arbitrary number of files, read them, and apply some processing. The way I do it ususally is by mounting a volume when running docker run, but now I’m transforming it to a Flyte pipeline and have had some issues. What I’m trying to do is to pass it as a FlyteDirectory, but it’s not working as I expected. I’m doing it like this:

ContainerTask(
    name="my_container",
    image="localhost:30000/my_container:latest",
    input_data_dir="/input",
    output_data_dir="/output",
    inputs=kwtypes(
        config_file=FlyteFile,
        my_dir=FlyteDirectory,),
    outputs=kwtypes(out=FlyteDirectory),
    command=[
        "python app.py --input_dir=/input/my_dir --config=/input/config_file"
    ]
)

I was expecting the

/input

directory to contain a dir called

my_dir

and a file called

config_file

. However, it seems to be empty. Can someone help me figure out how to mount data? Thanks 🙌

I'm on Windows 10. I've installed

flytekit

using:

pip install flytekit

tried to reinstall it, used conda to install it, removed it and then installed it again using pip. But when I use

flytekit status

on my vscode terminal I get this error:

'flytekit' is not recognized as an internal or external command,
operable program or batch file.

I've added these PATHS to my environment variables [the image is attached] Can anyone please give me a clue? Why is

flytekit

or any of its features like

pyflyte

is not recognized on my terminal (either vscode or cmd)? Any help would be really appreciated. Thanks.

Hi, I am running a workflow locally on my machine using

pyflyte run

or

python -m

but, I am getting this error -

2023-06-20 17:04:51,354876 ERROR    {"asctime": "2023-06-20 17:04:51,354", "name": "flytekit", "levelname":  base_task.py:587
                                    "ERROR", "message": "Failed to convert outputs of task 'read_dataset' at                 
                                    position 0:\n  [Errno 28] Error writing bytes to file. Detail: [errno                    
                                    28] No space left on device"}

Is there a virtual directory or disk flyte creates when running on a local machine which I can purge? The data for training is being pulled from s3.

Hi, Flyte Team! I wonder flyte support Grafana Logging with Loki in Flyte console. My team currently is using Kibana (EFK), but we consider it to replace Loki.