Is the new single binary deployment intended to replace and phase out the multi-image/binary deployment?

Having some connection issues. flytectl seems to be working perfectly but pyflyte is erroring out.

UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: Peer name <http://k8s-flyte-XXXXX.us-west-1.elb.amazonaws.com|k8s-flyte-XXXXX.us-west-1.elb.amazonaws.com> is not in peer certificate

Hello, I have a flyte .py file that imports a module I created from a subdir called lib and it contains another workflow that is intended as subflow.

from lib.subwf import my_subwf

I can run the flyte flow successfully in local mode. However, it errored when running with --remote in the demo flyte cluster I started on my PC. The error msg is:

ModuleNotFoundError: No module named 'lib'

Hello, as someone who is creating a new flyte task in flytekit, I am a little bit confused about the difference between

PythonTask

and

PythonInstanceTask

. With the latter, I am having a difficult time writing a unit test that passes where I am trying to verify if my workflow serialized correctly. More in 🧵

Hi All, I want to run a script using a custom docker image. I have created the docker container and placed it in the home directory in the flyte-sandbox Now when I run the script using : pyflyte run --remote --image fibo:1.1 fibanocci_example.py wf --n 8 I'm getting the error mentioned in 2nd image. Even the .py file is present in the same directory. Can anyone please provide any insight into why this might be happening ?

Hey guys, trying to run the example workflow on my Flyte AWS cluster and seeing

Exceeded resourcequota: [BackOffError] The operation attempt was blocked by back-off [attempted at: 2022-08-15 22:25:40.938456125 +0000 UTC m=+13688.975277765][the block expires at: 2022-08-15 22:25:58.135850691 +0000 UTC m=+13706.172672341] and the requested resource(s) exceeds resource ceiling(s)

. However, following the config instructions, I manually edited the

ResourceQuota

to some absurdly high cpu and memory limit. I’m seeing this in Flyteconsole UI as well as propeller logs. I’m trying to spin up some more nodes for testing to see if that helps

Hey folks, currently investigating Flyte as an alternative to our current setup which uses Kubeflow/Argo -- we have several pipelines with a very high degree of fan-out (on the order of ~700-1000 individual tasks). Does anyone have data points/benchmarks on how Flyte scales wrt the size of the workflow? Many of the features in the SDK (especially how flytekit works w/ native python types) look quite promising and could eliminate a lot of workarounds we've implemented to use Kubeflow.

Hi, everyone! Another question: is it possible to start the sandbox without using flytectl? A member of my team is using Windows and wants to try Flyte. I was trying to accomplish this by starting the sandbox with Docker Compose, and having another container to communicate with it. I was able to run simple commands such as

flytectl create

and

flytectl get

from the container, but I can't build images, and I think the problem is on the sandbox side, since I can't do that even on my host machine. I get the error:

OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "docker": executable file not found in $PATH: unknown

This is the configuration for the sandbox. What am I missing?

services:
  flyte-sandbox:
    image: <http://cr.flyte.org/flyteorg/flyte-sandbox:latest|cr.flyte.org/flyteorg/flyte-sandbox:latest>
    container_name: flyte-sandbox
    privileged: true
    volumes:
      - ..:/usr/src           # The docker-compose file is inside a sandbox/ dir
    ports:
      - "30081:30081"         # Console at <http://localhost:30081/console>
      - "30084:30084"
      - "30088:30088"

I've gotten around the map_task "single input" limitation before by using a data class to consolidate all the inputs together, but it doesn't look like I can use those as inputs to the shell task since you can't (or I think you can't) deference multiple times (i.e.

{{ .<http://inputs.my|inputs.my>__input__data_class.input_1 }}

)

what's the best way to query bigquery from aws? it seems the plugin only supports querying from GCP