https://flyte.org
Join Slack
Hey everyone, I'm trying to run a wf on my AWS single binary deployment of Flyte. I've got port forw...
# flyte-support
b
Hey everyone, I'm trying to run a wf on my AWS single binary deployment of Flyte. I've got port forwarding running with
Copy code
kubectl -n flyte port-forward service/flyte-backend-flyte-binary 8088:8088 8089:8089
Config is this
Copy code
admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: localhost:8088
  authType: Pkce
  insecure: true
logger:
  show-source: true
  level: 0
And I'm getting this when running a wf:
Copy code
$ FLYTE_CONFIG=~/.flyte/config.yaml pyflyte run --remote workflows/greeting.py wf --name Sam
Failed with Exception: Reason: SYSTEM:Unknown
RPC Failed, with Status: StatusCode.UNAVAILABLE
	details: failed to connect to all addresses; last error: INTERNAL: ipv4:127.0.0.1:8088: Trying to connect an http1.x server
	Debug string UNKNOWN:failed to connect to all addresses; last error: INTERNAL: ipv4:127.0.0.1:8088: Trying to connect an http1.x server {grpc_status:14, created_time:"2023-04-25T15:31:54.782127+01:00"}
a
Hi @brief-fish-13206 Try adding 
insecureSkipVerify: true
b
Like this?
Copy code
admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: localhost:8088
  authType: Pkce
  insecure: true
  insecureSkipVerify: true
logger:
  show-source: true
  level: 0
Still no luck :-(
a
and change 
insecure
to 
false
b
Now I get this :-(
Copy code
Failed with Unknown Exception <class 'ssl.SSLError'> Reason: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)
[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)
a
is this running on the cloud?
t
8088 is http, for flytekit, use 8089
insecureskipverify doesn’t do anything if insecure is true
and insecure should be true.
a
right
I'll have a session with Samuel but meanwhile @brief-fish-13206 1. Remove the 
insecureSkipVerify
line 2. Leave 
insecure: true
3. Change the port from 
8088
to 
8089
b
Cool, that's "working" now. @average-finland-92144 I'm back to the 403 error now. Catch up tomorrow :-)
Thanks @thankful-minister-83577 🙂
v
Hi guys. Similar issue for me as well. With some WARNINGS first,
Copy code
~/data_ops/eks/flyte/flytesnacks/cookbook$ pyflyte run --remote core/flyte_basics/hello_world.py my_wf 

2023-05-24 15:01:01,173936 WARNING  {"asctime": "2023-05-24 15:01:01,173", "name": "flytekit", "levelname": "WARNING", "message": "Error while parsing a block mapping\n  in     file.py:185
                                    \"/home/enceladus/.flyte/config.yaml\", line 1, column 1\nexpected <block end>, but found '<block mapping start>'\n  in
                                    \"/home/enceladus/.flyte/config.yaml\", line 4, column 3 reading yaml config file at /home/enceladus/.flyte/config.yaml, ignoring..."}
2023-05-24 15:01:01,217704 WARNING  {"asctime": "2023-05-24 15:01:01,217", "name": "flytekit", "levelname": "WARNING", "message": "Error while parsing a block mapping\n  in     file.py:185
                                    \"/home/enceladus/.flyte/config.yaml\", line 1, column 1\nexpected <block end>, but found '<block mapping start>'\n  in
                                    \"/home/enceladus/.flyte/config.yaml\", line 4, column 3 reading yaml config file at /home/enceladus/.flyte/config.yaml, ignoring..."}
2023-05-24 15:01:01,221089 WARNING  {"asctime": "2023-05-24 15:01:01,221", "name": "flytekit", "levelname": "WARNING", "message": "Error while parsing a block mapping\n  in     file.py:185
                                    \"/home/enceladus/.flyte/config.yaml\", line 1, column 1\nexpected <block end>, but found '<block mapping start>'\n  in
                                    \"/home/enceladus/.flyte/config.yaml\", line 4, column 3 reading yaml config file at /home/enceladus/.flyte/config.yaml, ignoring..."}
2023-05-24 15:01:02,057700 WARNING  {"asctime": "2023-05-24 15:01:02,057", "name": "flytekit", "levelname": "WARNING", "message": "Error while parsing a block mapping\n  in     file.py:185
                                    \"/home/enceladus/.flyte/config.yaml\", line 1, column 1\nexpected <block end>, but found '<block mapping start>'\n  in
                                    \"/home/enceladus/.flyte/config.yaml\", line 4, column 3 reading yaml config file at /home/enceladus/.flyte/config.yaml, ignoring..."}
2023-05-24 15:01:02,060970 WARNING  {"asctime": "2023-05-24 15:01:02,060", "name": "flytekit", "levelname": "WARNING", "message": "Error while parsing a block mapping\n  in     file.py:185
                                    \"/home/enceladus/.flyte/config.yaml\", line 1, column 1\nexpected <block end>, but found '<block mapping start>'\n  in
                                    \"/home/enceladus/.flyte/config.yaml\", line 4, column 3 reading yaml config file at /home/enceladus/.flyte/config.yaml, ignoring..."}


Failed with Exception Code: SYSTEM:Unknown
RPC Failed, with Status: StatusCode.UNAVAILABLE
        details: failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30080: Failed to connect to remote host: Connection refused
        Debug string UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30080: Failed to connect to remote host: Connection refused {created_time:"2023-05-24T15:01:09.752070056+05:30", grpc_status:14}
heres the config -
Copy code
admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
        endpoint: dns:///localhost:8089
  authType: Pkce
  insecure: true
logger:
  show-source: true
  level: 0
I installed single cloud cluster flyte with latest 1.6.1 helm chart. both http and grpc are forwarded in other terminals. The flyte frontend is up on 8088. any thoughts?
a
@victorious-kilobyte-69570 have you ever used 
flytectl demo
in your system? From the error message, it's weird to see pyflyte trying to connect to 
30080
, the port for sandbox
v
hmm, i think I have a long time ago. is there something I should clear/remove?
a
sorry @victorious-kilobyte-69570 yeah you could 
flytectl demo teardown
and it will remove your local sandbox instance. Then port-forward your flyte-binary and try again please
t
flytectl demo teardown --volume
to remove the local db volume also
v
now the console isn't coming up at localhost8088. There are no errors or any other logs in the http port forward terminal, its just showing
Copy code
Forwarding from 127.0.0.1:8088 -> 8088
Forwarding from [::1]:8088 -> 8088
, and in config, ive kept
Copy code
admin:
        endpoint: dns:///localhost:8088
  authType: Pkce
  insecure: true
logger:
  show-source: true
  level: 0
t
localhost:30080
a
I think @victorious-kilobyte-69570 is running 
flyte-binary
is that correct?
t
oh you’re port forwarding a full helm install
oh your endpoint should be 8089
not 8088
8088 is http, 8089 is grpc
v
yes im at the stage of flyte-binary in AWS
ok will try. also will talk my colleagues here, there are many new security related changes going on, might be a AWS security group thats blocking me.
154 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by