Daniel Xenes
04/12/2023, 3:09 PMTraceback (most recent call last):
File "/usr/local/bin/pyflyte-fast-execute", line 8, in <module>
sys.exit(fast_execute_task_cmd())
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/flytekit/bin/entrypoint.py", line 497, in fast_execute_task_cmd
_download_distribution(additional_distribution, dest_dir)
File "/usr/local/lib/python3.9/site-packages/flytekit/tools/fast_registration.py", line 111, in download_distribution
FlyteContextManager.current_context().file_access.get_data(additional_distribution, os.path.join(destination, ""))
File "/usr/local/lib/python3.9/site-packages/flytekit/core/data_persistence.py", line 303, in get_data
raise FlyteAssertion(
flytekit.exceptions.user.FlyteAssertion: Failed to get data from <s3://my-s3-bucket/flytesnacks/development/TLGPVQIJ7UHT2DCEZO3ZBNK77U======/fast0a28e459bf705dea2398e8d5270d09d1.tar.gz> to /root/ (recursive=False).
Has anyone encountered something like this before? Seems like either the container or the workflow can’t access the tarFROM <http://ghcr.io/flyteorg/flytekit:py3.9-latest|ghcr.io/flyteorg/flytekit:py3.9-latest>
# Install Python dependencies
USER root
RUN pip install --no-cache-dir cloud-volume scikit-image
USER flytekit
Ketan (kumare3)
Daniel Xenes
04/12/2023, 3:13 PM@task(container_image="localhost:30000/em-membrane-image:latest",
cache=True, cache_version="1.0", limits=Resources(mem="4Gi"))
I then registered the workflow
pyflyte register -p flytesnacks sobel_membrane.py
And then executed the workflow through the GUI. I get this error in the first stepMatplotlib created a temporary config/cache directory at /tmp/matplotlib-yfc_gvjn because the default path (/home/flytekit/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
/usr/local/lib/python3.9/site-packages/botocore/utils.py:1720: FutureWarning: The S3RegionRedirector class has been deprecated for a new internal replacement. A future version of botocore may remove this class.
warnings.warn(
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/s3fs/core.py", line 112, in _error_wrapper
return await func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/aiobotocore/client.py", line 358, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/flytekit/core/data_persistence.py", line 301, in get_data
self.get(remote_path, to_path=local_path, recursive=is_multipart)
File "/usr/local/lib/python3.9/site-packages/flytekit/core/data_persistence.py", line 205, in get
return file_system.get(from_path, to_path, recursive=recursive)
File "/usr/local/lib/python3.9/site-packages/fsspec/asyn.py", line 115, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/fsspec/asyn.py", line 100, in sync
raise return_result
File "/usr/local/lib/python3.9/site-packages/fsspec/asyn.py", line 55, in _runner
result[0] = await coro
File "/usr/local/lib/python3.9/site-packages/fsspec/asyn.py", line 562, in _get
return await _run_coros_in_chunks(
File "/usr/local/lib/python3.9/site-packages/fsspec/asyn.py", line 246, in _run_coros_in_chunks
await asyncio.gather(*chunk, return_exceptions=return_exceptions),
File "/usr/local/lib/python3.9/asyncio/tasks.py", line 442, in wait_for
return await fut
File "/usr/local/lib/python3.9/site-packages/s3fs/core.py", line 1171, in _get_file
body, content_length = await _open_file(range=0)
File "/usr/local/lib/python3.9/site-packages/s3fs/core.py", line 1162, in _open_file
resp = await self._call_s3(
File "/usr/local/lib/python3.9/site-packages/s3fs/core.py", line 347, in _call_s3
return await _error_wrapper(
File "/usr/local/lib/python3.9/site-packages/s3fs/core.py", line 139, in _error_wrapper
raise err
PermissionError: Access Denied.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/pyflyte-fast-execute", line 8, in <module>
sys.exit(fast_execute_task_cmd())
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/flytekit/bin/entrypoint.py", line 497, in fast_execute_task_cmd
_download_distribution(additional_distribution, dest_dir)
File "/usr/local/lib/python3.9/site-packages/flytekit/tools/fast_registration.py", line 111, in download_distribution
FlyteContextManager.current_context().file_access.get_data(additional_distribution, os.path.join(destination, ""))
File "/usr/local/lib/python3.9/site-packages/flytekit/core/data_persistence.py", line 303, in get_data
raise FlyteAssertion(
flytekit.exceptions.user.FlyteAssertion: Failed to get data from <s3://my-s3-bucket/flytesnacks/development/TLGPVQIJ7UHT2DCEZO3ZBNK77U======/fast0a28e459bf705dea2398e8d5270d09d1.tar.gz> to /root/ (recursive=False).
Original exception: Access Denied.
Ketan (kumare3)
Daniel Xenes
04/13/2023, 6:09 PMadmin:
# For GRPC endpoints you might want to use dns:///flyte.myexample.com
endpoint: localhost:30080
authType: Pkce
insecure: true
console:
endpoint: <http://localhost:30080>
logger:
show-source: true
level: 0
I notice in this one though there is storage block for minioSamhita Alla
Ketan (kumare3)
Samhita Alla
import os
import flytekit
from flytekit import task, workflow
from flytekit.types.directory import FlyteDirectory
@task
def t1() -> FlyteDirectory:
working_dir = flytekit.current_context().working_directory
dir = os.path.join(working_dir, "flyte_directory")
os.makedirs(dir, exist_ok=True)
return FlyteDirectory(dir)
@task
def t2(ff: FlyteDirectory) -> str:
return ff.download()
@workflow
def wf() -> str:
return t2(ff=t1())
Ketan (kumare3)
Samhita Alla
Ketan (kumare3)