https://flyte.org logo
Join Slack
Powered by
New RunLLM response generated: >>>*# Question* Hi...
# runllm-flyte-trial
a
New RunLLM response generated: >>>*# Question* Hi teams How to disable the s3 when i don't have a s3 service. I follow hard way of onprem cluster to allow flytectl in order to interact with onprem k8s cluster in 1.30.2. 
Traceback (most recent call last):                                                                                                                                                                             │
│ File "/usr/local/bin/pyflyte-fast-execute", line 8, in <module> │ │ sys.exit(fast_execute_task_cmd()) │ │ File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in call │ │ return self.main(*args, *kwargs) │ │ File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main │ │ rv = self.invoke(ctx) │ │ File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke │ │ return ctx.invoke(self.callback, *ctx.params) │ │ File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke │ │ return __callback(*args, **kwargs) │ │ File "/usr/local/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 534, in fast_execute_task_cmd │
>>│ _download_distribution(additional_distribution, dest_dir) │
│ File "/usr/local/lib/python3.10/site-packages/flytekit/core/utils.py", line 306, in wrapper │ │ return func(*args, **kwargs) │ │ File "/usr/local/lib/python3.10/site-packages/flytekit/tools/fast_registration.py", line 144, in download_distribution │ │ FlyteContextManager.current_context().file_access.get_data(additional_distribution, os.path.join(destination, "")) │ │ File "/usr/local/lib/python3.10/site-packages/flytekit/core/data_persistence.py", line 529, in get_data │ │ raise FlyteAssertion( │ │ flytekit.exceptions.user.FlyteAssertion: USERAssertionError error=Failed to get data from s3://flyte/flytesnacks/development/WBA2M5HMNI2H72BDVM56ALRSKQ======/script_mode.tar.gz to ./ (recursive=False). │ │ │ │ Original exception: Unable to locate credentials │ │ Stream closed EOF for flytesnacks-development/f0a93fef32f4345f8b5a-n0-0 (f0a93fef32f4345f8b5a-n0-0) # Answer To disable S3 when you don't have an S3 service, you need to configure Flyte to use a different storage backend. You can use MinIO or other compatible storage services. Here are the steps to configure Flyte to use MinIO, based on the provided data sources: 1. Modify the Flyte Configuration: Update the Flyte configuration file to use MinIO instead of S3. The configuration file is typically located at 
~/.flyte/config.yaml
or 
~/.flyte/config-sandbox.yaml
. 2. Update the Storage Section: Modify the 
storage
section to point to your MinIO setup. Here is an example configuration:
Copy code
storage:
  connection:
    access-key: minio
    secret-key: minio123
    endpoint: <http://localhost:9000>
    auth-type: accesskey
    disable-ssl: true
  type: minio
>>```
container: my-flyte-bucket
Copy code
3. *Deploy MinIO*: Ensure that MinIO is deployed and accessible from your Flyte setup. You can follow the <https://docs.flyte.org/en/latest/deployment/sandbox.html#deploy-flyte-sandbox-to-a-cloud-kubernetes-cluster|Flyte documentation> for deploying MinIO.

4. *Set Environment Variables*: If you are running Flyte tasks locally, set the necessary environment variables for MinIO:
export AWS_ACCESS_KEY_ID=minio export AWS_SECRET_ACCESS_KEY=minio123 export AWS_ENDPOINT_URL=http://localhost:9000 ``` 5. Restart Flyte Components: Restart Flyte components to apply the new configuration. If you encounter any issues, refer to the Flyte community discussion for additional support. Would you like more details on any specific step? Sources: - Flyte Community Discussion - Flyte Documentation Slack Link https://flyte-org.slack.com/archives/CP2HDHKE1/p1719122518.133729 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
2 Views
Open in Slack
PreviousNext
Powered by