#1649 Add RemoteFileAccessProvider
Pull request opened by
pingsutw
TL;DR
Currently, flytekit remote will try to use s3fs to upload data, but it should use data proxy instead.
RemoteFileAccessProvider allows flytekit remote to upload the data (structure dataset, flyte file, etc) to the s3.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☐ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
https://flyte-org.slack.com/archives/CP2HDHKE1/p1684272325647759
Follow-up issue
NA
flyteorg/flytekit
Codecov: 70.60% (-0.63%) compared to faf02b1
Codecov: 13.95% of diff hit (target 71.22%)
GitHub Actions: build-plugins (3.11, flytekit-sqlalchemy)
GitHub Actions: build-plugins (3.11, flytekit-snowflake)
GitHub Actions: build-plugins (3.11, flytekit-polars)
GitHub Actions: build-plugins (3.11, flytekit-papermill)
GitHub Actions: build-plugins (3.11, flytekit-pandera)
GitHub Actions: build-plugins (3.11, flytekit-kf-tensorflow)
GitHub Actions: build-plugins (3.11, flytekit-kf-pytorch)
GitHub Actions: build-plugins (3.11, flytekit-kf-mpi)
GitHub Actions: build-plugins (3.11, flytekit-k8s-pod)
GitHub Actions: build-plugins (3.11, flytekit-hive)
GitHub Actions: build-plugins (3.11, flytekit-envd)
GitHub Actions: build-plugins (3.11, flytekit-duckdb)
GitHub Actions: build-plugins (3.11, flytekit-dolt)
GitHub Actions: build-plugins (3.11, flytekit-deck-standard)
GitHub Actions: build-plugins (3.11, flytekit-dbt)
GitHub Actions: build-plugins (3.11, flytekit-data-fsspec)
GitHub Actions: build-plugins (3.11, flytekit-dask)
GitHub Actions: build-plugins (3.11, flytekit-bigquery)
GitHub Actions: build-plugins (3.11, flytekit-aws-sagemaker)
GitHub Actions: build-plugins (3.11, flytekit-aws-batch)
GitHub Actions: build-plugins (3.11, flytekit-aws-athena)
GitHub Actions: build-plugins (3.10, flytekit-vaex)
✅ 6 other checks have passed
6/30 successful checks