Hello everyone.. I'm testing the "getting started" example in the flyte docs but I'm getting "module 'example' has no attribute 'wf'" using the pyflyte run command.. any hints?

I’m early in my Flyte Journey, but I want to build a workflow with different containers (easy!) with one team per container.. However one of these steps require much more resources than the rest (something like 32vcpu, 128 GB RAM) and can’t be broken down. Ideally I don’t want a cluster that size enabled all of the time. I have the option to choose pretty much any cloud, would you recommend something like AWS Batch for resources? Or can you configure your K8s cluster enough through Flyte to auto scale this?

Hello! I’m trying to convert a Python dict to a LiteralMap and found the DictTransformer class. It states:

Transformer that transforms a univariate dictionary Dict[str, T] to a Literal Map

But it only seems to be able to transform it into a Literal. So how do I go from a Literal to a LiteralMap?

Hi, what is the reason that launchplans cannot be started from the UI?

And launchplans seem the only way to specify default values for inputs?

Does anyone know how to setup email notifications? According to the docs, notifications are only supported on AWS, but this seems to be outdated...

Hi team I want to use map_tasks to process the data from a StructuredDataset.. It’s a large dataset so i’ll need to use

iter()

to access all of it and would like each map task to process 1 chunk. Can map task take a generator instead of a list as the input?

Hey all, Running into an module error. I have a sandbox environment running locally that is supposed to pull a function from a module in our code outside the root flyte folder. Seems like it should be simple... Is this possible?

Hi. As the new Union.ai docs writer I will also be contributing to the Flyte docs.So, I'd like to get commiter access to all the repos that contain content surfaced on docs.flyte.org. I understand that this includes the repos

flyte

,

flytesnacks

and some others. Can someone help me with this?

Hello! I have trouble with python code not updating despite repackaging the files and running flytectl register files with a new version. I've also tried rebuilding the container image and the same problem persists.

Hey, is there any starter ticket to pick up?

Trying to get familiar with the code

I wanted to make a bingo card so that conference attendees can scratch off squares if they see it during a presentation, and that image should be one of the squares 🙂

Additionally, there are no logs why requests go multiple times to snowflake. There were just loaded data in snowflake

We’ve seen something similar lately (only happening in a

map_task

though) - so following along here for any debugging tips

We have a conditional branch that takes the output of two tasks whether they are true of false, takes the output of two different tasks to process in a final task. Once in a while, the inputs from the boolean checks are being bound to the branch instead of the other tasks (once or twice out of a couple hundred). This makes flyte’s input converter fail. Does anyone else have the case where the variable being conditionally evaluated is not the same variable being passed into the nested tasks? Happy to share more logging as needed. Running on GKE.

Traceback (most recent call last):

      File "/root/.varner_dp/venv/3.7.1/lib/python3.9/site-packages/flytekit/exceptions/scopes.py", line 165, in system_entry_point
        return wrapped(*args, **kwargs)
      File "/root/.varner_dp/venv/3.7.1/lib/python3.9/site-packages/flytekit/core/base_task.py", line 472, in dispatch_execute
        native_inputs = TypeEngine.literal_map_to_kwargs(exec_ctx, input_literal_map, self.python_interface.inputs)
      File "/root/.varner_dp/venv/3.7.1/lib/python3.9/site-packages/flytekit/core/type_engine.py", line 800, in literal_map_to_kwargs
        return {k: TypeEngine.to_python_value(ctx, lm.literals[k], python_types[k]) for k, v in lm.literals.items()}
      File "/root/.varner_dp/venv/3.7.1/lib/python3.9/site-packages/flytekit/core/type_engine.py", line 800, in <dictcomp>
        return {k: TypeEngine.to_python_value(ctx, lm.literals[k], python_types[k]) for k, v in lm.literals.items()}

Message:

    'n2.o0'

SYSTEM ERROR! Contact platform administrators.

Does anyone have examples or best practices on how to use the stats & logging properties of flytekit.current_context? I just see TODOs in the doc...

I am testing the DB plugin on our company cluster and got the following message. It the job was hanging. Curious if anyone can help trouble shoot cc @Kevin Su

Hi there, I am wondering if there is a way to dynamically set the default inputs of a launchplan after it has been serialized and I only have access to the archive of serialized artifacts and flytectl. The use case is that I've generated workflow / launchplan artifacts in a CI process but now want to register those files on flyte running in different environments where the default inputs could be environment specific without having to re-package everything (just tweak the inputs to the launchplans). More details in 🧵

Hi Everyone ! I was curious to find out a couple of questions regarding flyte. I’m curious to know if anyone has resources on: 1) integrating flyte with a larger kubernates cluster 2) if its possible to see the realtime console outputs for the workflow and pipe it somewhere else

Hey i wonder where is the dockerfile for ghcr.io/flyteorg/flytekit:py3.8-xxx

Hello, Flyte Team! I have a question about flyte retry policy. I have checked flyte re-run failed task in the same workflow. But tasks re-return in a long term, for example, the failed task run after a day. some succeeded task is also same. running again after a day or more than 10 hours. Does anyone know why flyte task run with a long term interval?

Have anyone seen this error?

byhsu@byhsu-ld1 ~/learn-repo/flyte-demo flyte ❯ ./bin/flytectl demo start
INFO[0000] [0] Couldn't find a config file []. Relying on env vars and pflags.
🧑‍🏭 Bootstrapping a brand new flyte cluster... 🔨 🔧
🐋 Going to use Flyte v1.3.0 release with image <http://cr.flyte.org/flyteorg/flyte-sandbox-bundled:sha-f69fb09ca189e8bf57e1a6a12db168274f640d15|cr.flyte.org/flyteorg/flyte-sandbox-bundled:sha-f69fb09ca189e8bf57e1a6a12db168274f640d15>
🐋 pulling docker image for release <http://cr.flyte.org/flyteorg/flyte-sandbox-bundled:sha-f69fb09ca189e8bf57e1a6a12db168274f640d15|cr.flyte.org/flyteorg/flyte-sandbox-bundled:sha-f69fb09ca189e8bf57e1a6a12db168274f640d15>
🧑‍🏭 booting Flyte-sandbox container
😬 Something went wrong: Failed to start Sandbox container 🐋, Please check your docker client and try again.
Error: Error response from daemon: invalid IP address in add-host: "host-gateway"
{"json":{},"level":"error","msg":"Error response from daemon: invalid IP address in add-host: \"host-gateway\"","ts":"2023-01-15T22:30:19-08:00"}

Hello Flyte Team! I want to use custom built docker image to use our custom dependencies. I tested on flyte sandbox on my local macbook. (Running from

flytectl demo start

) My workflow was successfully run on local (

pyflyte run cifar10.py wf

) But my workflow needs

torch

packages, so I built docker image with Dockerfile like below:

FROM python:3.9-slim-buster

RUN python -m pip install torch torchvision

RUN python -m pip install flytekit pandas numpy awscli

I pushed the image to dockerhub, it can be accessed publicly. And I submit flyte workflow with

pyflyte

cli

pyflyte run --remote --image {username}/flyte:0.0.4 cifar10.py wf

But it doesn’t work on remote (sandbox). How can I do?

qq, can pythonfunctask work without fast registration? If can, how the code is stored and pulled?

Just had a great offline chat with @Yee and @Peeter Piegaze. Thank you guys so much! The community is really very active and helpful for the new comers! flyte

Hi Community, I wonder if there is any tutorials for Gitlab openId authentication?

I tried to follow the auth tutorials (https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html), but seems it does work when try to use corresponding configuration from gitlab. To be honest, I am not quite clear about the implementation here, is there any extra efforts we should do?

What’s the minimum python version officially supported for flytekit? Was looking at a mypy improvement to allow the type checker to catch obvious errors in workflows but looks like it’ll need ParamSpec in order to correctly propagate the type annotations in the task/workflow/dynamic decorators. It’s part of Python 3.10, but is also available in typing_extensions for >= py3.7. Is it safe to assume that python 3.7 is the oldest version that flytekit should be guaranteed to work on?

Hi community, are there any published Flyte books?? (or any preprint ver.?)