#6617 Update flyte copilot/stow store list for multi-part blob support
Pull request opened by
pingsutw
## Tracking issue
NA
## Why are the changes needed?
Flyte Copilot is not handling multi part blobs correctly because paths are coming back as
s3://https://... This updates the flytestdlib stow_store to use the url.Path instead of url.String().
## What changes were proposed in this pull request?
## How was this patch tested?
from flytekit import task, workflow, ContainerTask, kwtypes
from flytekit.types.directory import FlyteDirectory
from flytekit.types.file import FlyteFile
simple_container = ContainerTask(
name="my_simple_container",
image="python:3.8-slim",
# image="public.ecr.aws/unionai-flyte/flytecopilot:805f69f5aa8821141da6c221776b612d2e6adeee",
input_data_dir="/var/inputs",
output_data_dir="/var/outputs",
inputs=kwtypes(example_dir=FlyteDirectory),
outputs={},
retries=2,
command=[
"ls", "/var/inputs",
],
)
@task
def create_folder_task() -> FlyteDirectory:
# Create a local directory and return it as a FlyteDirectory
import os
import tempfile
local_dir = tempfile.mkdtemp()
# Create a sample file in the directory
with open(os.path.join(local_dir, "sample.txt"), "w") as f:
f.write("This is a sample file.")
return FlyteDirectory(path=local_dir)
@workflow
def example_container_task_wf():
ff = create_folder_task()
simple_container(
example_dir=ff,
)
simple_container_file = ContainerTask(
name="my_simple_container",
image="python:3.8-slim",
# image="public.ecr.aws/unionai-flyte/flytecopilot:805f69f5aa8821141da6c221776b612d2e6adeee",
input_data_dir="/var/inputs",
output_data_dir="/var/outputs",
inputs=kwtypes(example_file=FlyteFile),
outputs={},
retries=2,
command=[
"ls", "inputs"
],
)
@task
def create_file_task() -> FlyteFile:
return FlyteFile(
file)
@workflow
def file_container_task_wf():
ff = create_file_task()
simple_container_file(
example_file=ff,
)
### Labels
Please add one or more of the following labels to categorize your PR:
•
added: For new features.
•
changed: For changes in existing functionality.
•
deprecated: For soon-to-be-removed features.
•
removed: For features being removed.
•
fixed: For any bug fixed.
•
security: In case of vulnerabilities
This is important to improve the readability of release notes.
### Setup process
### Screenshots
## Check all the applicable boxes
• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.
## Related PRs
unionai/flyte#813
## Docs link
https://bito.ai#summarystart|
### Summary by Bito
This pull request enhances Flyte Copilot's multi-part blob handling by updating the download logic in
download.go
for correct path formatting and improving URL processing in
stow_store.go
for better compatibility with various storage schemes.
flyteorg/flyte