New RunLLM response generated: >>>*# Question* he...
# runllm-flyte-trial
j
New RunLLM response generated: >>>*# Question* hello Flyte community, I'm taking Flyte for a test drive in a bare metal k8s environment with a flyte-binary deployment. I've been able to remotely run some basic hello_world workflows when flyte is configured to use plain AWS S3, but I'm running into problems when I switch over to minIO. our credentials are restricted to target
s3://<my-minio-bucket>/<user-specific-prefix>/...
Copy code
% pyflyte run --remote hello_world.py my_wf                                                         
Running Execution on Remote.
FlyteValueException: USER:ValueError: error=Value error!  Received: 403. Request to send data 
https://<my-minio-endpoint>/<my-minio-bucket>/flytesnacks/development/JYCYKEBONJ7A22EHVR3LYNWZQE%3D%3D%3D%3D%3D%3D/fastd57844519531b1
161d2506c45b0fecd5.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=<minio-username>%2F20241204%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241204T
040812Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-md5%3Bhost%3Bx-amz-meta-flytecontentmd5&X-Amz-Signature=d5fa6dadb90f2e5a0d73f37630e98ae6b1994d4a9e4ac760387
38ecd6e11cef0 failed.
Response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access 
Denied.</Message><Key>flytesnacks/development/JYCYKEBONJ7A22EHVR3LYNWZQE======/fastd57844519531b1161d2506c45b0fecd5.tar.gz</Key><BucketName>my-minio-bucket</BucketName><Resource>/<my-minio-bucket>/flytesnacks/development/JYCYKEBONJ7A22EHVR3LYNWZQE======/fastd57844519531b1161d2506c45b0fecd5.tar
.gz</Resource><RequestId>180DDDCA8CF19855</RequestId><HostId>414fdc917451eb973643c2ecd05e6a7702dd399ed76ed0fbea4e978999873dac</HostId></Error>
is there any way to force Flyte to write to
s3://<my-minio-bucket/<user-specific-prefix>/...
? I haven't had any luck injecting the prefix that our minIO store requires us to use. # Answer To resolve the issue of Flyte not writing to the specified MinIO bucket with a user-specific prefix, you need to configure the
raw_output_data_config
in Flyte to include the desired prefix. According to GitHub Issue #4615, you can update the raw output data configuration using the
flytectl update workflow-execution-config
command. This allows you to specify an output location prefix, which should be set to your required MinIO path. Here's an example configuration:
Copy code
domain: development
project: flytesnacks
raw_output_data_config:
  output_location_prefix: s3://<my-minio-bucket>/<user-specific-prefix>
>>
Ensure that your Flyte configuration files are correctly set up to use this prefix. Would you like more details on configuring Flyte or troubleshooting this issue? Sources: - GitHub Issue #4615 Slack Link https://flyte.slack.com/archives/CP2HDHKE1/p1733286559.390079 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.