careful-piano-35677
10/09/2023, 9:51 AMendpoint: "<http://minio.flyte.svc.cluster.local:9000>"
I slightly changed it to, endpoint: "<http://minio.flyte.svc.cluster.local:30084>"
which is the nodePort of that same minio
service specified in this part,
while accssing the node with th doing so, I've been facing error like
[ ] Registration <http://workflows.scoop_wf.wf|workflows.scoop_wf.wf> type WORKFLOW NoneFailed with Exception Code: SYSTEM:Unknown RPC Failed, with Status: StatusCode.INTERNAL details: failed to write marshaled workflow [resource_type:WORKFLOW project:"flytesnacks" domain:"development" name:"<http://workflows.scoop_wf.wf|workflows.scoop_wf.wf>" version:"v0" ] to storage <s3://my-s3-bucket/metadata/admin/flytesnacks/development/workflows.scoop_wf.wf/v0> with err Failed to write data [9119b] to path [metadata/admin/flytesnacks/development/workflows.scoop_wf.wf/v0].: PutObject, putting object: RequestError: send request failed caused by: Put "<http://minio.flyte.svc.cluster.local:30084/my-s3-bucket/metadata/admin/flytesnacks/development/workflows.scoop_wf.wf/v0>": dial tcp 10.43.158.135:30084: i/o timeout and base container: <s3://my-s3-bucket>
while it was able to register the tasks to the minio storage, but it keeps throwing error when registering the workflow, it keeps throwing the same error when I repeated the experiment for a few times, any help would be appreciated.tall-lock-23197
careful-piano-35677
10/10/2023, 5:39 AM/etc/hosts
file, since the minio service is NodePort
type and we can directly interact with minio
from that node ip and port instead of kubectl port-forward
, also since flyte is not interacting with the service, it is interacting directly with that IP (upto my understanding), that's why!
If there is any possible workaround, would be happy to change.tall-lock-23197
9000
) is an appropriate value to use.careful-piano-35677
10/10/2023, 6:31 AM9000
, just wanted to know if there is any workaround for that. Thanksaverage-finland-92144
10/17/2023, 6:41 PMminio
service using K8s Service Discovery so `minio.flyte.svc.cluster.local`is an entry known inside the cluster and handled by the coreDNS instance as part of the K8s cluster. Maybe for multi-node deployment we can try using a ClusterIP instead of NodePort