acoustic-carpenter-78188
01/13/2023, 12:52 PMpyflyte run --remote core/flyte_basics/hello_world.py my_wf, it does not work because no configuration is provided. After calling flytectl config init, the config file ~/.flyte/config.yaml is created, but the endpoint port there is not 30080 as in the demo cluster, but 30081. I had to manually change (rewrite) the config file to have endpoint: dns:///localhost:30080. Then, everything works well.
Expected behavior
Following userguide_setup, I'd expect that following all steps from top to bottom would lead to successful --remote execution, which didn't happen.
Additional context to reproduce
1. create virtual env
2. git clone <https://github.com/flyteorg/flytesnacks>
3. cd flytesnacks/cookbook
4. pip install -r core/requirements.txt
5. install flytectl
6. flytectl demo start
7. pyflyte run --remote core/flyte_basics/hello_world.py my_wf <-- this fails
8. flytectl config init
9. edit ~/.flyte/config.yaml and change edpoint port from 30081 to 30080
10. pyflyte run --remote core/flyte_basics/hello_world.py my_wf <-- now it works
Steps 8 and 9 are missing in the documentation.
Screenshots
Result of step 8 in _Additional context to reproduce_:
pyflyte run --remote cookbook/core/flyte_basics/hello_world.py my_wf
{"asctime": "2023-01-13 13:48:16,864", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses\"\n\tdebug_error_string = \"{\"created\":\"@1673614096.864140274\",\"description\":\"Failed to pick subchannel\",\"file\":\"src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3260,\"referenced_errors\":[{\"created\":\"@1673614096.864139984\",\"description\":\"failed to connect to all addresses\",\"file\":\"src/core/lib/transport/error_utils.cc\",\"file_line\":167,\"grpc_status\":14}]}\"\n>, sleeping 200ms and retrying"}
{"asctime": "2023-01-13 13:48:17,064", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses\"\n\tdebug_error_string = \"{\"created\":\"@1673614097.064544493\",\"description\":\"Failed to pick subchannel\",\"file\":\"src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3260,\"referenced_errors\":[{\"created\":\"@1673614097.064543902\",\"description\":\"failed to connect to all addresses\",\"file\":\"src/core/lib/transport/error_utils.cc\",\"file_line\":167,\"grpc_status\":14}]}\"\n>, sleeping 400ms and retrying"}
Traceback (most recent call last):
File "/home/michal/miniconda3/envs/flyte/bin/pyflyte", line 8, in <module>
sys.exit(main())
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clis/sdk_in_container/run.py", line 539, in _run
remote_entity = remote.register_script(
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py", line 596, in register_script
upload_location, md5_bytes = fast_register_single_script(
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/tools/script_mode.py", line 113, in fast_register_single_script
upload_location = create_upload_location_fn(content_md5=md5)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/friendly.py", line 998, in get_upload_signed_url
return super(SynchronousFlyteClient, self).create_upload_location(
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/raw.py", line 41, in handler
return fn(*args, **kwargs)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/raw.py", line 854, in create_upload_location
return self._dataproxy_stub.CreateUploadLocation(create_upload_location_request, metadata=self._metadata)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1673614097.465232045","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3260,"referenced_errors":[{"created":"@1673614097.465231304","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyte