Channels
#flyte-deployment
Title
# flyte-deployment
a
Armaan Goel
09/20/2022, 10:44 PMStill having issues with the flyte deployment actually, it seems like I can run jobs but they never complete!
Not sure how to debug this, because there are no logs
Looks like its stuck in queued
k
Ketan (kumare3)
09/20/2022, 11:04 PM
hmm
stuck in queued?
can you check the kubernetes namespace
project-domainkubectl get pods -n <namespace>and see what the pods are, also in the UI
you should see a small
iohh you have not installed flytepropeller
this is reallllllllly odd
you installation seems bad - phew!!!!!!!!!
cc @Shivay Lamba?
cc @Alireza can you help here?
a
Alireza
09/20/2022, 11:23 PMCan you run this command and check the outputs:
kubectl get pods -n flyteThe error points out that
the server could not find the requested resourcesFlyteAdmina
Armaan Goel
09/20/2022, 11:37 PMGKE, yep
a
Alireza
09/20/2022, 11:41 PMDid you follow the instruction on docs.flyte.org?
a
Armaan Goel
09/20/2022, 11:43 PMYes, but I had to make some modifications to get to this point
a
Alireza
09/20/2022, 11:44 PMcan you also share the output of this command?
kubectl get nodes -n flyteis it ok if I open your doc?
Another Question. After reinstalling helm did you:
1. helm upgrade, or
2. helm install?
a
Armaan Goel
09/20/2022, 11:49 PMOh yeah feel free to check out the doc!
i did
helm uninstall flyte -n flytehelm install -n flyte -f values-gcp.yaml --create-namespace flyte flyteorg/flyte-corea
Alireza
09/20/2022, 11:51 PMhmm, too less resources! I am just suggesting the cluster is so small. Not sure if it is related to error though!
helm installhelm upgradecan you helm upgrade right now and see if the same error happens or not?
Meanwhile I take a look into your doc
Your installation process is correct! Good Job!
can you describe the failed pod?
a
Armaan Goel
09/21/2022, 1:28 AMhmm which pod?
I figured it would be
amt5m758z8xtv7rzqrnl-n0-0a
Alireza
09/21/2022, 1:29 AMWhat is the output of this?
kubectl get pods -n flyteI am suspicous that your venv is breaking this. Is it possible to try it with conda?
a
Armaan Goel
09/21/2022, 1:35 AMthe venv of the code I’m deploying?
a
Alireza
09/21/2022, 1:35 AMyeap
a
Armaan Goel
09/21/2022, 1:36 AMI’m actually just deploying this flyte example
a
Alireza
09/21/2022, 1:37 AM(I guess) you are using a virtual environment, right?
Copy code
Message: exec /opt/venv/bin/pyflyte-execute: exec format errorIt would be helpful to send the process you made the virtual environment so that I can replicate the process. Or try it with conda virtual environment. something like this:
conda create --name flyte python=3.9conda activate flytea
Armaan Goel
09/21/2022, 1:47 AMIf you look at the error message I sent there do seem to be some grpc errors in flytepropellor
but also I dont think i’m using a venv I made a docker container based on the guide you had
i have the link to the github i reference just above and here is my version of it https://github.com/armaangoel78/flytetest
a
Alireza
09/21/2022, 1:55 AMI see. Just a quick qustion do you know by any chance if your flyte config is setup correctly?
or how did you set it up?
a
Armaan Goel
09/21/2022, 1:59 AM./flyte/config.yaml or the values file for the flyte cluster
a
Alireza
09/21/2022, 2:00 AM./flyte/config.yamlhow did you set it up?
it seems there is a connection error
Please share your findings. I will discuss it with my colleagues. Sorry couldn't solve it all today. We'll check it tomorrow.
a
Armaan Goel
09/21/2022, 2:10 AM Copy code
admin:
# For GRPC endpoints you might want to use dns:///flyte.myexample.com
endpoint: dns:///flyte.gauntletai.com
authType: Pkce
insecure: false
logger:
show-source: true
level: 0%a
Alireza
09/21/2022, 2:20 AMI got it
a
Armaan Goel
09/21/2022, 4:08 AM@Alireza no worries thanks so much for you help it means a ton 🙂
7 Views