Stephan Gref

    Stephan Gref

    4 months ago
    Scratching my head here a bit... Does anyone use flytekit + flytekitplugins-data-fsspec + fast register? It seems that fsspec does not handle the the default destination
    .
    (dot, like current directory) well and does not download the tar.gz at all. Reproduced this with this minimal example:
    import os
    
    from flytekit.tools.fast_registration import download_distribution as _download_distribution
    
    os.environ["AWS_ACCESS_KEY_ID"] = "key_id"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "secret_access_key"
    
    destination = "."
    
    _download_distribution(
        additional_distribution="<s3://bucket/fast-register.tar.gz>",
        destination=".")
    It works if I replace
    destination
    with is
    ./
    in this example. The issue is, that
    .
    is set by propeller (this is at least my understanding). Is there any way I can configure that?
    s

    Sören Brunk

    4 months ago
    I did try it a while a go but I ran into issues so I switched back. If I remember correctly it was in combination with fast register (regular download/upload worked fine) so it could be the same issue.
    Ketan (kumare3)

    Ketan (kumare3)

    4 months ago
    @Stephan Gref I do not think propeller sets the path
    the path is set by flytekit itself and should be safe to update