acoustic-carpenter-78188
09/12/2023, 1:05 AMpyflyte but doesn't work properly when running on the local demo cluster.
running:
> pyflyte run --remote example.py wf --n 500 --mean 42 --sigma 2
results in the error:
{"asctime": "2022-09-15 13:06:21,069", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"DNS resolution failed for localhost:30081: C-ares status is not ARES_SUCCESS qtype=A name=localhost is_balancer=0: DNS server returned answer with no data\"\n\tdebug_error_string = \"UNKNOWN:DNS resolution failed for localhost:30081: C-ares status is not ARES_SUCCESS qtype=A name=localhost is_balancer=0: DNS server returned answer with no data {created_time:\"2022-09-15T13:06:21.069160907+02:00\", grpc_status:14}\"\n>, sleeping 200ms and retrying"}
{"asctime": "2022-09-15 13:06:21,270", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"DNS resolution failed for localhost:30081: C-ares status is not ARES_SUCCESS qtype=A name=localhost is_balancer=0: DNS server returned answer with no data\"\n\tdebug_error_string = \"UNKNOWN:DNS resolution failed for localhost:30081: C-ares status is not ARES_SUCCESS qtype=A name=localhost is_balancer=0: DNS server returned answer with no data {created_time:\"2022-09-15T13:06:21.269916921+02:00\", grpc_status:14}\"\n>, sleeping 400ms and retrying"}
Traceback (most recent call last):
File "/home/tux/.local/bin/pyflyte", line 8, in <module>
sys.exit(main())
File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/tux/.local/lib/python3.10/site-packages/flytekit/clis/sdk_in_container/run.py", line 539, in _run
remote_entity = remote.register_script(
File "/home/tux/.local/lib/python3.10/site-packages/flytekit/remote/remote.py", line 596, in register_script
upload_location, md5_bytes = fast_register_single_script(
File "/home/tux/.local/lib/python3.10/site-packages/flytekit/tools/script_mode.py", line 113, in fast_register_single_script
upload_location = create_upload_location_fn(content_md5=md5)
File "/home/tux/.local/lib/python3.10/site-packages/flytekit/clients/friendly.py", line 998, in get_upload_signed_url
return super(SynchronousFlyteClient, self).create_upload_location(
File "/home/tux/.local/lib/python3.10/site-packages/flytekit/clients/raw.py", line 41, in handler
return fn(*args, **kwargs)
File "/home/tux/.local/lib/python3.10/site-packages/flytekit/clients/raw.py", line 854, in create_upload_location
return self._dataproxy_stub.CreateUploadLocation(create_upload_location_request, metadata=self._metadata)
File "/home/tux/.local/lib/python3.10/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/tux/.local/lib/python3.10/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "DNS resolution failed for localhost:30081: C-ares status is not ARES_SUCCESS qtype=A name=localhost is_balancer=0: DNS server returned answer with no data"
debug_error_string = "UNKNOWN:DNS resolution failed for localhost:30081: C-ares status is not ARES_SUCCESS qtype=A name=localhost is_balancer=0: DNS server returned answer with no data {created_time:"2022-09-15T13:06:21.670827636+02:00", grpc_status:14}"
Expected behavior
The example code should run on the local cluster I created using:
> flytectl demo start
Additional context to reproduce
No response
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteacoustic-carpenter-78188
09/12/2023, 1:05 AM