Hi Everyone. i am trying to deploy flye in azure A...
# flyte-deployment
s
Hi Everyone. i am trying to deploy flye in azure AKS cluster. Pods are up and running after running helm command for single cluster deployment.i am able to access flyte service without authentication through port-forward. As per documentation I am trying to test workflow by cloning flytesnacks repo. This will require pyflyte cli but I am unable to find any link or reference where to download pyflyte binary from. Can somebody please suggest on it?
s
Hi @Shambhu!
pyflyte
gets automatically installed when you install flytekit.
s
Hi @Samhita Alla, thanks. I am trying to test the workflow now in azure AKS cluster where flyte is installed following single cluster deployment mode in flyte namespace .Could somebody please suggest if I am doing anything wrong here.i have brought up another pod with python and flytekit installed in it in default namespace. Then i am running below command git clone https://github.com/flyteorg/flytesnacks cd flytesnacks/cookbook pyflyte --config $HOME/.flyte/config.yaml run --remote core/flyte_basics/hello_world.py my_wf Content of config.yaml is:- admin: # For GRPC endpoints you might want to use dns:///flyte.myexample.com endpoint: dns///flyte flyte binary grpc.flyte.svc.cluater.local8089 authType: Pkce insecure: true logger: show-source: true level: 0 It is erroring out with below error: root@python-flytekit:~/flytesnacks/cookbook# pyflyte --config $HOME/.flyte/config.yaml run --remote core/flyte_basics/hello_world.py my_wf Failed with Exception Code: SYSTEM:Unknown RPC Failed, with Status: StatusCode.INTERNAL     details: failed to create a signed url. Error: NoCredentialProviders: no valid providers in chain. Deprecated.     For verbose messaging see aws.Config.CredentialsChainVerboseErrors     Debug string UNKNOWN:Error received from peer {grpc_message:"failed to create a signed url. Error: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors", grpc_status:13, created_time:"2023-05-19T173209.146454034+00:00"} Can somebody please suggest how to fix this?
s
I think you need to set
insecure
to false in your config.yaml file.
s
Thanks. Able to run the pyflyte command and get the workflow in flyte now. But workflow run for "say hello" project failed with error as - "FlyteAssertion - Failed to get data from s3, original exception - unable to locate credentials". I have installed and configured minio operator as I am deploying flyte in azure AKS. can somebody please suggest where exactly workflow is looking for credentials of bucket and it is not able to locate and how to fix it?
d
Hi @Shambhu how did you deploy? Could you share the
values
file you used?
s
Hi @David Espejo (he/him), yeah sure. Thanks for your response .Sharing storage config quickly, will this be fine?i deployed flyte in single cluster deployment mode and also configured postgres and minio in azure AKS and updated their config in configmap and deleted the pod so that changes reflects.kindly suggest if I missed any step to fix this issue
d
wondering if you'd find useful what @Mathias Andersen shared here: https://github.com/flyteorg/flyte/discussions/3591#discussioncomment-5612989 Not using minio but Azure blob storage
BTW you can use flytekit >= 1.5 which should work seamlessly without specific
fsspec
plugins
s
Ok, Thanks for sharing it.will try to check this.
Hi @David Espejo (he/him) is it possible to use service principal client I'd and client secret while using azure storage account for flyte deployment in azure aks? Can you kindly help with sample values.yaml for the same?
d
@Shambhu not sure if exactly what you need, but recently @Byron Hsu expanded the auth guide to include Azure AD as IdP: https://docs.flyte.org/en/latest/deployment/configuration/auth_setup.html#authentication-setup
s
@David Espejo (he/him) thanks for input. I tried using values-aks.yaml but getting below error. Any suggestions? i ran this command - helm upgrade --install flyte-backend flyteorg/flyte-binary --namespace flyte --values aks-starter.yaml and this pod "flyte-backend-flyte-binary-c68fc599-5zccr" errors out as : - time="2023-05-31T104540Z" level=info msg="Using config file: [/etc/flyte/config.d/000-core.yaml /etc/flyte/config.d/001-plugins.yaml /etc/flyte/config.d/002-database.yaml /etc/flyte/config.d/003-storage.yaml]" panic: type is of an invalid value [custom] goroutine 239 [running]: github.com/flyteorg/flyteadmin/pkg/server.newGRPCServer({0x3e83040, 0xc001043740}, 0xc001052a50, 0xc00011b080, 0x2930363432623930?, {0x0?, 0x0}, {0x3ea3c28, 0xc000851330}, {0x0, ...}) /go/pkg/mod/github.com/flyteorg/flyteadmin@v1.1.88/pkg/server/service.go:111 +0xbde github.com/flyteorg/flyteadmin/pkg/server.serveGatewayInsecure({0x3e83040?, 0xc001043740}, 0x0?, 0xc00011b080, 0xc000826d80, 0x7fab6a7e1108?, 0x46fbfa?, {0x3ea3c28, 0xc000851330}) /go/pkg/mod/github.com/flyteorg/flyteadmin@v1.1.88/pkg/server/service.go:278 +0x705 github.com/flyteorg/flyteadmin/pkg/server.Serve({0x3e83040, 0xc001043740}, 0xc001052a50?, 0x18?) /go/pkg/mod/github.com/flyteorg/flyteadmin@v1.1.88/pkg/server/service.go:59 +0x19f github.com/flyteorg/flyte/cmd/single.startAdmin.func3() /flyteorg/build/cmd/single/start.go:92 +0x147 golang.org/x/sync/errgroup.(*Group).Go.func1() /go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:72 +0xa5
k
it seems the generated config is wrong
we do not have access to an azure environment and hence we cannot test. this was a community generated yaml file
I would request the community to help
s
Thanks @Ketan (kumare3) for your response. Hi Everyone, can somebody please share installation guide for azure as i have tried multiple option but it always end with error. Appreciate your help.
113 Views