mammoth-mouse-1111
04/08/2025, 6:29 PMflyte-binary
on K8s. I pyflyte register
the eager task specified in the docs, and attempt to run it with pyflyte --remote run
. I then hit the following error almost immediately:
FlyteSystemUnavailableException: Flyte cluster is currently unavailable. Please make sure the cluster is up and running.
I've tried specifying the remote
section of the eager task in a variety of ways but no dice. I'd really appreciate some help with this 🙏mammoth-mouse-1111
04/08/2025, 6:30 PMgrpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30080: Failed to connect to remote host: connect: Connection refused (111)"
debug_error_string = "UNKNOWN:Error received from peer {grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30080: Failed to connect to remote host: connect: Connection refused (111)", grpc_status:14, created_time:"2025-04-08T18:28:27.626869553+00:00"}"
>
mammoth-mouse-1111
04/08/2025, 6:34 PMlocalhost:30080
?mammoth-mouse-1111
04/08/2025, 7:03 PMFlyteRemote
. It always keeps trying to use the default value.high-park-82026
mammoth-mouse-1111
04/09/2025, 2:47 PMFlyteRemote
for eager workflows.
I've tried the below code, along with various combinations of FlyteRemote.auto
, Config.auto
, specifying a config file instead of the endpoint, ...
No matter what I do, it always tries to use the default endpoint of localhost:30080
and fails since that isn't the right endpoint.
from flytekit import eager
from flytekit.remote import FlyteRemote
from flytekit.configuration import Config, PlatformConfig
from flytekit import task
remote = FlyteRemote(
# config=Config.for_endpoint(endpoint="dns:///calpha-service-flyte-grpc:8089", insecure=True),
config=Config(
platform=PlatformConfig(
endpoint="dns:///calpha-service-flyte-grpc:8089", insecure=True, insecure_skip_verify=True
)
),
default_project="flytesnacks",
default_domain="development",
)
@task
async def add_one(x: int) -> int:
import time
time.sleep(5)
return x + 1
@eager(remote=remote)
async def my_test() -> str:
y = await add_one(5)
return "hello" + str(y)
mammoth-mouse-1111
04/09/2025, 2:53 PMhigh-park-82026
@eager(remote=remote)
async def my_test() -> str:
import os
print (os.environ)
y = await add_one(5)
return "hello" + str(y)
Can you do this?
I think you may be right...mammoth-mouse-1111
04/10/2025, 4:16 PMmammoth-mouse-1111
04/10/2025, 4:28 PM30080
or localhost
or anything that suggests the remote is being overrode.
can't post all the envvars for security ofchigh-park-82026
Constructing default remote with no config and
high-park-82026
high-park-82026
FLYTE_PLATFORM_URL
it should workmammoth-mouse-1111
04/10/2025, 8:43 PMgrpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "DNS resolution failed for calpha-service-flyte-grpc:8089: C-ares status is not ARES_SUCCESS qtype=AAAA name=calpha-service-flyte-grpc is_balancer=0: Domain name not found"
debug_error_string = "UNKNOWN:Error received from peer {grpc_message:"DNS resolution failed for calpha-service-flyte-grpc:8089: C-ares status is not ARES_SUCCESS qtype=AAAA name=calpha-service-flyte-grpc is_balancer=0: Domain name not found", grpc_status:14, created_time:"2025-04-10T20:34:43.337184437+00:00"}"
>
mammoth-mouse-1111
04/10/2025, 8:49 PMmammoth-mouse-1111
04/10/2025, 8:54 PMFLYTE_PLATFORM_INSECURE=true
lets me connect and use eager tasks.
Thanks so much for all your help Haytham 🙏