Hi everyone, I was able to run the blastX use case...
# ask-the-communityj
Jay Phan
02/28/2023, 3:23 PMHi everyone, I was able to run the blastX use case and NLP processor locally on my computer, but when I try to run them on the sandbox, it usually fails because its missing the package (gensim or blastx) needed, could someone help me out with this or guide me to the correct tutorial?
f
Fabio Grätz
02/28/2023, 3:51 PMYou could do the following:
1. Create a
DockerfileFabio Grätz
02/28/2023, 3:51 PM Copy code
FROM <http://ghcr.io/flyteorg/flytekit:py3.8-1.2.5|ghcr.io/flyteorg/flytekit:py3.8-1.2.5> # The image used by flyte tasks by default
RUN pip install <your package>Fabio Grätz
02/28/2023, 3:51 PMBuild this image and push it to a container registry, e.g. dockerhub.
Fabio Grätz
02/28/2023, 3:52 PMAnd then when executing your workflow, specify the image tag:
Copy code
pyflyte run --remote --image <your image tag>j
Jay Phan
02/28/2023, 3:52 PMThanks!
Jay Phan
02/28/2023, 10:21 PMHey I tried that and is getting this error, any idea how to fix it?
Copy code
Pod failed. No message received from kubernetes.
[fd644c3a4d2b441389e6-n0-0] terminated with exit code (1). Reason [Error]. Message:
/python3.8/asyncio/tasks.py", line 455, in wait_for
return await fut
File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 1134, in _get_file
body, content_length = await _open_file(range=0)
File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 1125, in _open_file
resp = await self._call_s3(
File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 339, in _call_s3
return await _error_wrapper(
File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 139, in _error_wrapper
raise err
PermissionError: Access Denied.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/pyflyte-fast-execute", line 8, in <module>
sys.exit(fast_execute_task_cmd())
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flytekit/bin/entrypoint.py", line 496, in fast_execute_task_cmd
_download_distribution(additional_distribution, dest_dir)
File "/usr/local/lib/python3.8/site-packages/flytekit/tools/fast_registration.py", line 111, in download_distribution
FlyteContextManager.current_context().file_access.get_data(additional_distribution, os.path.join(destination, ""))
File "/usr/local/lib/python3.8/site-packages/flytekit/core/data_persistence.py", line 456, in get_data
raise FlyteAssertion(
flytekit.exceptions.user.FlyteAssertion: Failed to get data from <s3://my-s3-bucket/flytesnacks/development/YDNUR2PPEXSI4ZZ4HQEXVN3UW4======/scriptmode.tar.gz> to /root/ (recursive=False).
Original exception: Access Denied.
.s
Samhita Alla
03/01/2023, 6:00 AM
@Jay Phan, how have you spin up the sandbox? Have you used
flytectl demo startj
Jay Phan
03/01/2023, 6:21 AMYes I did
Jay Phan
03/01/2023, 6:21 AMThis is my dockerfile for references:
Copy code
FROM <http://ghcr.io/flyteorg/flytekit:py3.8-1.2.5|ghcr.io/flyteorg/flytekit:py3.8-1.2.5>
RUN pip3 install awscli
RUN pip3 install flytekit
RUN pip3 install gensim
RUN pip3 install nltk
RUN pip3 install numpy
RUN pip3 install plotly
RUN pip3 install dataclasses_json
RUN pip3 install scikit_learn
RUN pip3 install typing
RUN pip3 install dataclassess
Samhita Alla
03/01/2023, 6:24 AM
Can you build your Docker image using https://github.com/flyteorg/flytesnacks/blob/master/cookbook/case_studies/ml_training/nlp_processing/Dockerfile?
157 Views