Hey quick question, how would you add a

nodeSelector

for a specific workflow/task? I’m not sure if taints are enough to trigger our cluster-autoscaler. I wanted to edit the pods on the fly but I can’t 😕

I'm looking to interact with the flyte REST API and admin database to be able to query for execution input / output data. I have 2 questions: 1. How to query input / output data from the database? I can find references to proto input files, but can't find any reference to where the outputs exist. I also wonder if there is an example of decoding the downloaded proto files. 2. How to authenticate the REST API? I found the API swagger docs, but I neither see examples of using the API or how to authenticate. I can generate JWT's easily enough against our IdP, but passing that as a standard Authorization header doesn't appear sufficient.

Hi everyone. I am trying to run

pyflyte run --remote awesome.py main_workflow

on flyte-binary deployed on aks. I am able to access the ui. my config is the standart one, although I tried some variations:

1 admin:
  2   # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  3   endpoint: dns:///flyte.<hostname>
  4   authType: Pkce
  5   insecure: false
  6 logger:
  7   show-source: true
  8   level: 0

when I run the command, I get

Failed with Exception Code: SYSTEM:Unknown
RPC Failed, with Status: StatusCode.UNAVAILABLE
	details: failed to connect to all addresses; last error: UNKNOWN: ipv4:<ip>:443: Cannot check peer: missing selected ALPN property.
	Debug string UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:<ip>:443: Cannot check peer: missing selected ALPN property. {grpc_status:14, created_time:"2023-08-16T16:11:24.832905759+00:00"}

We're using azure/application-gateway as ingress. I've read through the channel and saw similar issues, but non of the solutions seemed to solve the issue. Would be glad if someone has a hint 🙂

Hey folks we recently upgraded copilot and tasks are failing with

[1/1] currentAttempt done. Last Error: USER::[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[flyte-copilotdownloader] terminated with exit code (1). Reason [Error]. Message: 
  --storage.cache.max_size_mbs int             Maximum size of the cache where the Blob store data is cached in-memory. If not specified or set to 0,  cache is not used
      --storage.cache.target_gc_percent int        Sets the garbage collection target percentage.

I took a look at the config in the docs and looks like we need to add these. Is there anything else I’m missing and will these default values work for our deployment?

co-pilot:
  cpu: 500m
  default-input-path: /var/flyte/inputs
  default-output-path: /var/flyte/outputs
  image: <http://cr.flyte.org/flyteorg/flytecopilot:v0.0.15|cr.flyte.org/flyteorg/flytecopilot:v0.0.15>
  input-vol-name: flyte-inputs
  memory: 128Mi
  name: flyte-copilot-
  output-vol-name: flyte-outputs
  start-timeout: 1m40s
  storage: ""

Hi. I'm getting OOMKilled when running the diabetes demo on Kubernetes. Describing the pod I see

Limits:
      cpu:     1
      memory:  200Mi
    Requests:
      cpu:     1
      memory:  200Mi

How can I increase this? My helm yaml already has

task_resource_defaults:
    task_resources:
      defaults:
        cpu: 1000m
        memory: 1000Mi
        storage: 1000Mi
      limits:
        storage: 2000Mi

Hi, I have one question regarding the for loop in workflow. I wanted to have a code like the following

@task
def example_task(num:int) -> None:
    .....

@workflow
def test_workflow() -> None:
    for i in range(5):
        example_test(num=i)

It seems one way is that I can use dynamic workflow. However, in dynamic workflow, the tasks in the for loop will be running in parallel. I wanted the tasks can be running in sequence. So do you have a method to realize the for loop in sequence. Thanks!

Hi! I’m trying to add a kubernetes pod label to all pods generated as a result of launching a workflow. Execution code:

remote.execute(
        workflow,
        inputs=inputs,
        options=Options(labels=common_models.Labels({"flyte.user": os.getenv("USER", "na")}))

where

remote

is of type

FlyteRemote

. I can see that the workflow’s pod has the expected label, but none of the subworkflows do. Is there a way to have all pod labels be passed to all child pods of a workflow pod?

Hello, I have a super basic question! I’d like to run a local flyte cluster to test with, which I thought I would do by running

flyte start --config=flyte_local_k3d.yaml

, (after having run

make compile

already) for example when using k3d. Even though my

.k3d/kubeconfig-flyte.yaml

config file works with k3d directly, when I try to run the above command I always get an error:

{"json":{"src":"start.go:113"},"level":"error","msg":"Failed to create controller manager. error building Kubernetes Clientset: Error building kubeconfig: error loading config file \"/Users/ailinyu/.k3d/kubeconfig-flyte.yaml\": no kind \"Simple\" is registered for version \"<http://k3d.io/v1alpha3\|k3d.io/v1alpha3\>" in scheme \"pkg/runtime/scheme.go:100\"","ts":"2022-07-25T17:17:20-07:00"} {"json":{"src":"start.go:164"},"level":"panic","msg":"Failed to start Propeller, err: error building Kubernetes Clientset: Error building kubeconfig: error loading config file \"/Users/ailinyu/.k3d/kubeconfig-flyte.yaml\": no kind \"Simple\" is registered for version \"<http://k3d.io/v1alpha3\|k3d.io/v1alpha3\>" in scheme \"pkg/runtime/scheme.go:100\"","ts":"2022-07-25T17:17:20-07:00"}

Am I using this incorrectly or do I need to have something else in my k3d yaml config perhaps?

hello! Quick question - what's best pattern of triggering flyte workflows based on events (similar to argo-events)? Like, trigger a workflow when new file is uploaded to s3 bucket or a pubsub message is received?

Hi, everyone! I was wondering: is there a simple way to execute file persistence in Flyte? My use case: I have a workflow that generates a dataset. I want to save that dataset somewhere so I can call it on my training workflow without regenerating it. Is it possible? When I return a

FlyteFile

, the path is not really obvious/human-friendly.