Hi, I am trying to use kuberay operator. I have created kuberay-operator and ray-cluster using helm chart and it is running in a new namespace. When I am trying to run an example code mentioned in the blog here , getting this below error.

File "/home/kkanagar/kkr/kuberay_test.py", line 15, in <module>
    def ray_task() -> list[int]:
TypeError: 'type' object is not subscriptable

Changing this list[int] to any is working fine in local but when trying to register to remote flyte. Need your help, please

I was told that we can use regular Python type hinting for files. But in the example: https://docs.flyte.org/projects/cookbook/en/latest/auto/case_studies/ml_training/mnist_classifier/pytorch_single_node_and_gpu.html I see

from flytekit.types.file import PythonPickledFile

and in the example: https://docs.flyte.org/projects/cookbook/en/latest/auto/case_studies/ml_training/house_price_prediction/house_price_predictor.html I see

from flytekit.types.file import JoblibSerializedFile

These are used to output the model state. Couldn’t we use regular Python type hints for these? May I know why

PythonPickedFile

and

JoblibSerializedFile

were used in the examples?

Hello, I am writing a custom flytekit plugin and I was wondering how can I retrieve the rendered image uri in

execute()

of a custom plugin, if i pass a templated value to

container_image

. E.g. is my task definition

@task(task_config=my_task_config, container_image="{{.image.img1.fqn }}:{{.image.img1.version}}"

I can see that the

container_image

is set as a property and if I retrieve it, I see it’s value is still a templated string. How can I get the actual rendered image uri for that task? (i.e. basically I want what this function returns)

Hi, using flyte again after a long break. What is the current equivalent of

from flytekit.sdk.types import Types

the

Types.Generic

? This used to be the type for arbitrary dict/json objects but it doesn't seem to be here anymore?

can you use docker images when you run

pyflyte run

without the remote option. e.g. if I define a

@task(container_image="XXXX"

, can I use this image locally when I run?

Hi , I am trying to integrate okta with Flyte. Following this page for setting up appAuth and userAuth. First tried only setting up userAuth in configmap. It worked, okta is enabled. Then included appAuth to configmap and re-deployed, facing the below error in flytescheduler-check container of flytescheduler. Using the same client_id for userAuth and appAuth.

panic: rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: oauth2: cannot fetch token: 400 Bad Request
Response: {"errorCode":"invalid_client","errorSummary":"Invalid value for 'client_id' parameter.","errorLink":"invalid_client","errorId":"oae4BTFncguRFCNZUfFEpIFhA","errorCauses":[]}

goroutine 1 [running]:
main.main()
        /go/src/github.com/flyteorg/flyteadmin/cmd/scheduler/main.go:12 +0x85

This is my sample configmap for this.

configmap:
    auth:
      appAuth:
        authServerType: External
        externalAuthServer:
          baseUrl:  <https://www.oktadev.com/oauth2/default>
        thirdPartyConfig:
          flyteClient:
            clientId:  "a1b2c3xxxxyz9N3" 
            redirectUri: <https://flytedev.myapp.com/callback>
            scopes:
            - offline
            - all
      userAuth:
        openId:
          baseUrl: <https://www.oktadev.com/oauth2/default>
          scopes:
            - profile
            - openid
          clientId: "a1b2c3xxxxyz9N3"
      authorizedUris:
        - <https://flytedev.myapp.com>
        - <http://flyteadmin:80>
        - <http://flyteadmin.flyte.svc.cluster.local:80>

this is probably a silly question but i'm looking to serialize

tf.tensor

. The

tf.io.serialize_tensor

returns a string, should we store it in

tf.train.Feature

? and further put this into

tf.train.Example

and use

SerializeToString()

method to serialize the proto? Is there a need to store it on to the disk? i'm not sure if storing on disk is possible or not

join the community sync here - https://zoom.us/j/93875115830?pwd=YWZWOHl1ODRRVjhjVkxSV0pmZkJaZz09

Hello, I’m using

fast register

to develop in the

sandbox

environment, and I’d like to configure it to use a storage backend other than

awscli

. Is there any documentation for doing this? We run exclusively in GCP and install

gsutil

in each of our Flyte-ready Docker images. I’m hoping to avoid installing

awscli

in our images just to support running in

sandbox

. Thanks!

Hi! I have LaunchPlan created in Python that I want to trigger in my CI/CD flow based on file changes. The documentation shows how to create an execution given a spec file, but how do I actually execute a launch plan that’s already created programmatically?

Hey everyone 🙂 I'm currently deploying a flyte sandbox at my new company and am struggling with getting auth to work. I followed this guide. Flyte is deployed, I am redirected to the SSO login screen, after that I can view the flyte console 👍 I can also run

flytectl get projects

, am redirected to the browser, and then retrieve the projects.

pyflyte register ...

works as well. So far so good. When I manually launch a registered workflow from the console, it doesn't start however.

k -n development describe <http://flyteworkflows.flyte.lyft.com|flyteworkflows.flyte.lyft.com> ah9ghn272q7v22dbsvvd

gives:

Status:
  Failed Attempts:  9
  Message:          Workflow[] failed. ErrorRecordingError: failed to publish event, caused by: EventSinkError: Error sending event, caused by [rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"]
  Phase:            0
Tasks:
  resource_type:TASK project:"3dod" domain:"development" name:"project.workflows.workflow.deploy_model" version:"2"

Details about config in 🧵 A pointer in the right direction would be much appreciated.

Hey Folks, new here but our team loves Flyte! I'm seeing a hang from a fork when I use flyte remote - should this being forking? I'm only just getting into this but if anyone has any quick thoughts or ideas, PLMK. I'll post more info soon. Running python3 in a venv, flytekit 1.1.3, Ubuntu 20.04 LTS on GCP. python3 -v tells me I'm hanging on popen_fork.

Hi, how can I run a ContainerTask on a GPU enabled node and set

runtimeClassName

to

nvidia

so that it can actually use the GPU? I added

requests=Resources(gpu="1")

but can I add the runtimeClassName?

Hi team, does anyone know what can be a good default value for type

FlyteFile

How do you scale up nodes inside a EKS cluster. If I set my desired + min to 1 and then I spawn a bunch of jobs, shouldn't I expect it to create new nodes if I don't hit the max node? But the default auto scaling group doesn't seem to be doing that. Is there something in the helm chart that I should change for that?

Hi team, I need to set some env variables (e.g. PATH/PYTHONPATH/LD_LIBRARY_PATH) in an image, but I cannot do so when building the docker image since their values are dynamically determined when building the image, and docker does not allow that. How can I source those env variables on a worker pod before

pyflyte-fast-execute

gets invoked? Or is there any other recommended way of doing this?

Hi all, I want to try the new ray plugin in flyte. Can I do that using a sandbox environment?

I'm getting this error on flytescheduler running on AWS EKS, which I guess is admin ip:

Error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp ...:81: i/o timeout"

panic: authentication error! Original Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken, Auth Error: failed to initialized token source provider. Err: failed to fetch auth metadata. Error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp ...:81: i/o timeout"

Could this be related to auth config? And if so where do I look to fix it?

Hello, I have some common libraries (including pyspark) from both public and private repos that I would like all my flyte workflows to share. To do that, I would like to build a custom flyte docker image. However, I am not able to find a clear documentation on how to do it and how to refer to the custom image when running or registering my flows. Could you please help? Thanks a lot.

Hi, I am trying to use kuberay operator and ray cluster with in the Flyte by following this documentation. https://blog.flyte.org/ray-and-flyte#heading-launch-a-ray-cluster I have deployed the kuberay-operator and ray-cluster by following the helm chart docs given here https://github.com/ray-project/kuberay/tree/master/helm-chart. The pods are up and running. I have also enabled the ray plugin in the flyte. When running the pyflyte remotely, I am getting this below error.(I tried deploying the ray pods both in separate namespace as well as in the namespace where flyte pods are running). Need help here on this error.

raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "failed to create workflow in propeller json: error calling MarshalJSON for type *v1alpha1.Inputs: Marshal called with nil"
        debug_error_string = "{"created":"@1666318439.715331527","description":"Error received from peer ipv4:35.172.9.181:443","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"failed to create workflow in propeller json: error calling MarshalJSON for type *v1alpha1.Inputs: Marshal called with nil","grpc_status":13}"

Hi, I am getting,

Requested MEMORY default [6Gi] is greater than the limit [150Mi]. Please fix your configuration

where and how to configure the limit? I am using the AWS EKS setup.

Hi, I have a strange error where I am creating several LaunchPlans with different names using

flytekit.LaunchPlan.get_or_create

. However, when I try to register this archive with the launchplans, it fails on a particular one with the name:

a-cst-old

, giving the error:

Error: rpc error: code = InvalidArgument desc = missing name

It’s just this name and the fixed input (string) that’s changed, nothing else, if I prefix this name with something else like, it works. And on a related note, is there a way to delete, not just archive, all launchplans?

Hey Flyte team! Can flytekit conditionals be used inside dynamics? I noticed that they say that have to be used within a workflow just trying to understand if its used in a dynamic that is then used in a workflow thats ok.

On the flyte UI, there's an option for

Only my executions

filtering. Right now when I click on that, I get

Request failed with status code 400 invalid value for filters

What needs to be done for this to work? Do I have to set up OAuth login for the UI?

With regards to https://github.com/flyteorg/flyte/issues/2992, should I update the link or just remove it? If I have to update it then what procedure should I follow?

Hello, I am having trouble building a custom image that will execute pyspark tasks.

Last Error: USER::Spark Job Failed with Error: driver container failed with ExitCode: 127, Reason: ContainerCannotRun

I have the following in the requirements.txt file for spark related dependencies.

flytekit>=0.20.0
flytekitplugins-snowflake
flytekitplugins-spark
scikit-learn
xgboost
unionml
SHAP
pandas
apache-beam
joblib
pyyaml==6.0
boto3==1.9.207
aws-secretsmanager-caching==1.1.1.5
aws-requests-auth==0.4.3

Could you tell me what are missing?

Hello, Regarding I found this https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/k8s_spark/index.html#spark-docker-image, and its example Dockerfile.

RUN python3 -m venv ${VENV}
ENV PATH="${VENV}/bin:$PATH"

RUN pip3 install wheel

# Install Python dependencies
COPY k8s_spark/requirements.txt /root
RUN pip install -r /root/requirements.txt

RUN flytekit_install_spark3.sh

Could you point me to the full repo location?

I think I found it https://github.com/flyteorg/flytesnacks/tree/master/cookbook/integrations/kubernetes/k8s_spark