Hi team.. Can

dynamic

be cached?

I am trying to use the flyte demo locally (OSX intel chip) using the

--remote

command. I am running into an issue. I am not sure where to start looking, so any advice is much appreciated. Inside of a new venv

$git clone <https://github.com/flyteorg/flytesnacks>
$cd flytesnacks/cookbook
$pip install -r core/requirements.txt

$flytectl demo start
$pyflyte run --remote core/flyte_basics/hello_world.py:my_wf

And then I get this

Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.

grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses"
	debug_error_string = "{"created":"@1655239314.572674000","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3129,"referenced_errors":[{"created":"@1655239314.572673000","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":163,"grpc_status":14}]}"

Hey folks.. quick question: If flyte is deployed on a k8s cluster, can it submit spark jobs on a different k8s cluster?

Hi all, is there an equivalent for

list-active-launch-plans

for

flytectl

? I really really miss this right now:

flyte-cli \
    -h HOST \
    -d ENV \
    -p PROJECT \
    list-active-launch-plans

Hi everyone! Has anyone experienced trouble with uploading to Minio? I'm getting an error where my task, which is trying to save a report to minio, can't connect to the minio endpoint. My deployment is a local kubeadm cluster with a single node, multiple pods. Using kubectl get pods -A I can see that the minio pod is up and running. I also can see the minio dashboard at localhost:30088. Any help would be much appreciated!

Hi everyone. I am trying to write a task to Insert a row into Postgres table. Because there is very little guide on Insert, so can anyone provide me any documentation on this issue? Thanks you for help.

Hi all. I have been trying Flyte for the past week or so and I love it! I am new to ML Ops style tools and workflows, so I don't think I get all of the details yet. Question on the sandbox?: Is there a way to download the task outputs to the sandbox container? Right now my outputs for a task that creates a directory is this:

{1 item
o0:{2 items
type:"multi-part blob"
uri:"<s3://my-s3-bucket/hi/ag5dp94bs4c846cn847s-n0-0/628c0053bbedf3f1db4d44362f48857e>"
}
}

Not sure where this multi-part blob resides

I want to deploy spark plugin in the Flyte backend. Step 1: I have stared a Flyte demo cluster by flytectl demo start Step 2: Install spark operator Step 3: Create “values-override.yaml” file and upgrade the Flyte helm

helm upgrade flyte flyteorg/flyte-core -f <https://raw.githubusercontent.com/flyteorg/flyte/master/charts/flyte-core/values-sandbox.yaml> -f values-override.yaml -n flyte

In step 3. I get the following error

Error: failed to download "flyteorg/flyte-core"

After checking, the flyte-core not installed after running the “flytectl demo start” command, and I not found the flyte-core repository in flyteorg. The flyte-core have still in use or has it been dropped?

Is there a REST API for the flytectl commands by any chance? (e.g. is there an API endpoint you can hit to run something like

flytectl create project

instead of running it as a CLI command?)

yes

everything has an api

hello.. new flyte user here.. had a question about how can i configure different tasks to use different images in a workflow? i am aware that i can do this by specifying

@task(
    container_image="{{.image.img1.fqn }}:{{.image.img1.version}}"
)

and the value of

image.img1.version

can be specified in a config file.. but how is that config linked to tasks while packaging the workflow?

I want to register my workflow. I have done the following steps Step 1: create project. Example:

flytectl create project --name myflyteproject --id myflyteproject --description "myflyteproject description" --labels app=myflyte

Step 2: build docker image Example:

docker build -t my-flyte-img:v1 .
docker tag my-flyte-img:v1 <http://ghcr.io/nguyenhuuvan51703223/my-flyte-img:v1|ghcr.io/nguyenhuuvan51703223/my-flyte-img:v1>
docker push <http://ghcr.io/nguyenhuuvan51703223/my-flyte-img:v1|ghcr.io/nguyenhuuvan51703223/my-flyte-img:v1>

Step 3: Example:

pyflyte -c sandbox.config --pkgs core serialize --in-container-config-path /root/sandbox.config --local-source-root /Users/admin/hvan/myproject/flytesnacks/cookbook --image <http://ghcr.io/nguyenhuuvan51703223/my-flyte-img:v1|ghcr.io/nguyenhuuvan51703223/my-flyte-img:v1> workflows -f _pb_output/

Step 4: Example

flytectl register files _pb_output/* -p myflyteproject -d development --version v1 k8sServiceAccount demo --outputLocationPrefix <s3://my-s3-bucket/raw_data>

Step 5: open UI and launch workflow I get the error:

[1/1] currentAttempt done. Last Error: USER::containers with unready status: [a5k8lnk9s5qh98szvgc8-n0-0]|Back-off pulling image "<http://ghcr.io/nguyenhuuvan51703223/my-flyte-img:v1|ghcr.io/nguyenhuuvan51703223/my-flyte-img:v1>

Did I miss or take wrong on any step?? Do I need push my image to my docker registry? Do I need define my repository name somewhere?

Hello! I'm having some issues with flyte when istio sidecars are present. When the flyte task container finishes, the pod does not terminate (because the istio sidecar is active) so it is stuck forever. Have you come across this? Do you have any tips or tricks? Thanks very much!

Hi team... quick question. If a task calls a task, do they both not show up in the Flyte console? I only see the outer task

This message was deleted.

Hey folks.. so I'm exploring how to set up the CI/CD pipeline now that I have some workflows running locally. In order to do this I was experimenting with the flyte-sandbox docker image to register the new version of the workflows and so on. I'm running into an error though

flytectl get workflows --admin.endpoint <remote_host>:30080 -p <project_name> -d development

is successful

docker run --rm --entrypoint flytectl <http://cr.flyte.org/flyteorg/flyte-sandbox|cr.flyte.org/flyteorg/flyte-sandbox> get workflows --admin.endpoint <remote_host>:30080 -p <project_name> -d development

throws an error

time="2022-06-27T20:18:25Z" level=info msg="[0] Couldn't find a config file []. Relying on env vars and pflags."
{"json":{},"level":"error","msg":"failed to initialize token source provider. Err: failed to fetch auth metadata. Error: rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: first record does not look like a TLS handshake\"","ts":"2022-06-27T20:18:26Z"}
{"json":{},"level":"warning","msg":"Starting an unauthenticated client because: can't create authenticated channel without a TokenSourceProvider","ts":"2022-06-27T20:18:26Z"}
{"json":{},"level":"info","msg":"Initialized Admin client","ts":"2022-06-27T20:18:26Z"}
Error: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"
{"json":{},"level":"error","msg":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: first record does not look like a TLS handshake\"","ts":"2022-06-27T20:18:27Z"}

hello! i am trying to run a workflow which has a task/step that fetches data from sandbox’s Minio and splits it into train/test. But the pod keeps dying with this error message

Pod failed. No message received from kubernetes.
[atg6lnllwptbr4j6thwc-n0-0] terminated with exit code (137). Reason [Error]. Message: 

.

the flyte cluster is local sandbox cluster

Hi, where can I check the latest Flyte helm chart version?

I'm still running into the issue here even with flytekit 1.0.3. Create project is asking for project?

Hi team.. I'm trying to serialize the workflows in order to run

flytectl register files

.. However pyflyte is not serializing anything. This is what I see when I run

pyflyte serialize...

in the container. Any ideas what's going on?

pyflyte serialize workflows -f /tmp/_pb_output
{"asctime": "2022-06-28 23:51:47,894", "name": "flytekit", "levelname": "INFO", "message": "Setting protocol to file"}
{"asctime": "2022-06-28 23:51:48,124", "name": "flytekit", "levelname": "INFO", "message": "Setting protocol to file"}
{"asctime": "2022-06-28 23:51:48,125", "name": "flytekit", "levelname": "INFO", "message": "Setting protocol to file"}
{"asctime": "2022-06-28 23:51:48,125", "name": "flytekit", "levelname": "INFO", "message": "Setting protocol to file"}
{"asctime": "2022-06-28 23:51:48,125", "name": "flytekit", "levelname": "INFO", "message": "We won't register bigquery handler for structured dataset because we can't find the packages google-cloud-bigquery-storage and google-cloud-bigquery"}
Serializing Flyte elements with image 
Writing output to /tmp/_pb_output
Loading packages [] under source root /root
Successfully serialized 0 flyte objects

Hey team.. so we have our custom UI that uses the REST API to trigger workflows in the Flyte backend. Now every time we bump the version of the workflows, the REST APIs invoked by the UI need to change as well to use the latest version. Is there a "latest" tag or something like that that can be used to alleviate this problem?

Hi community, I was trying to setup flyte-sandbox locally on my k3s cluster. I followed this instruction https://docs.flyte.org/en/latest/deployment/sandbox.html#deployment . First I was stuck at the point that

svclb-flyte-deps-contour-envoy-c519e84f-lnpvm

keeps pending in the namespace kube-system. I worked around this by upgradeing the flyte-deps config. However, I am still confronting the envoy pod pending issue. Terminal output are in the reply thread. Anyone know how should I fix this? In general, it seems to be some port allocation issue, but I really don’t have any idea how to debug this futher, lol. Really appreciate it! 🙏

Is there a way to setup "auto-cleanup" of old executions (e.g. older than couple days)? Asking because pods are not deleted until execution is deleted, so we are ending up with a like a gazillion of completed pods. Or there's another solution for not having completed pods lying around?

Hi flyte team, I integrated our OIDC provider for flyteadmin auth. But its failing during OIDC callback with the following error. It looks to be cookie that it generates using the auth_code becomes bigger than 4096 [gorilla securecookie implementation](https://github.com/gorilla/securecookie/blob/master/securecookie.go#L256-L259). Any ideas on how to fix this ?

{
  "json": {
    "src": "handlers.go:161"
  },
  "level": "error",
  "msg": "Error setting encrypted JWT cookie [SECURE_COOKIE_ERROR] Error creating secure cookie, caused by: securecookie: the value is too long",
  "ts": "2022-07-01T07:41:05Z"
}

With the way packaging and registration works with flyte, what is the recommended way to provide a common set of library methods across multiple flyte projects? (eg. our team wants to define a CustomTask that multiple projects want to use). Is there any example of this we can reference?

Hi using Google for flyte console auth and I can’t find documentation on how to use the refresh token - it logs me out constantly otherwise?

If anyone could point to the doc on this, it would be much appreciated!

Has anyone tried running Dask together with Flyte?

This message was deleted.