Hi, I have tasks that connects minio and download ...
# flyte-support
p
Hi, I have tasks that connects minio and download . This works perfectly fine when run local Same one fails when running in remote. Note that client.list_objects works on remote(pyflyte run --remote), but fails on iterating the object list( `for obj in objects`:)
Copy code
Message:

    MaxRetryError: HTTPConnectionPool(host='localhost', port=9000): Max retries exceeded with url: /flyte?location= (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x79443cd44d70>: Failed to establish a new connection: [Errno 111] Connection refused'))
Copy code
@task(container_image=image_spec)
def download_wod_ds()->FlyteDirectory:
# Set up Minio client
    client = Minio(
        "localhost:9000",
        access_key="BBsho4hEODTw0J2sv6uk",
        secret_key="rQn3WKDAhci6EnfwJf48XjpIpkbXInx0R1ciMNG7",
        secure=False
    )

    # Specify the bucket and folder to download
    bucket_name = "flyte"
    folder_name = "ns-wod-ds"

    # Create the download directory
    base_path = os.getcwd()
    new_directory_path = os.path.join(base_path, "wod-ds")
    os.makedirs(new_directory_path, exist_ok=True)
    print("made bucket")
    # List all objects in the folder
    objects = client.list_objects(bucket_name, prefix="flytesnacks/development/ns-wod-ds/training", recursive=True)
    print("ran list_objects")
    # Download each object to the local directory. FAIL'ed from here when run remote.
    for obj in objects:
        print("going thru object ist")
        object_name = obj.object_name
        download_path = os.path.join(new_directory_path, os.path.relpath(object_name, folder_name))
        download_dir_path = os.path.dirname(download_path)
        os.makedirs(download_dir_path, exist_ok=True)
        print("before downloading")
        try:
            client.fget_object(bucket_name, object_name, download_path)
            print(f"Downloaded {object_name} to {download_path}")
        except S3Error as exc:
            print(f"Error downloading {object_name}: {exc}")
    return FlyteDirectory(path=new_directory_path)
c
When your workflow runs remotely minio isn't available at localhost:9000, because when it runs remotely... its not running locally. What are you expecting to happen?
p
Yes, I realized that and worked after changing to FQDN. Thanks