justin hallquist
03/15/2023, 3:35 PMflytekit.exceptions.user.FlyteAssertion: Failed to put data from /tmp/tmp030l19_7/script_mode.tar.gz to <http://localhost:30084/my-s3-bucket/a/b/PJPRTCSJDCALQRYI44IFNWNQ5M%3D%3D%3D%3D%3D%3D/scriptmode.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20230315%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230315T152958Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-md5%3Bhost&X-Amz-Signature=9b449916e2b9ca077d45255d05f86fec463c3e999e14c2c38a0075928906afa0> (recursive=False).
Original exception: HTTPConnectionPool(host='localhost', port=30084): Max retries exceeded with url: /my-s3-bucket/a/b/PJPRTCSJDCALQRYI44IFNWNQ5M%3D%3D%3D%3D%3D%3D/scriptmode.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20230315%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230315T152958Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-md5%3Bhost&X-Amz-Signature=9b449916e2b9ca077d45255d05f86fec463c3e999e14c2c38a0075928906afa0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f205d87edc0>: Failed to establish a new connection: [Errno 111] Connection refused'))
noticed it was using a signed url override (not sure why it's doing that all of the sudden) and pretty much anything I change/update in the charts just doesnt apply, for example:
storage:
# -- Sets the storage type. Supported values are sandbox, s3, gcs and custom.
type: sandbox
# -- bucketName defines the storage bucket flyte will use. Required for all types except for sandbox.
bucketName: my-s3-bucket
# -- settings for storage type s3
signedUrl:
stowConfigOverride:
endpoint: <http://minio.flyte.svc.cluster.local:9000>
anyone have any thoughts?Ketan (kumare3)
03/15/2023, 3:44 PMjustin hallquist
03/15/2023, 3:46 PMjeev
03/15/2023, 3:46 PMjustin hallquist
03/15/2023, 3:47 PMstowConfigOverride
to address thatjeev
03/15/2023, 3:47 PMjustin hallquist
03/15/2023, 3:48 PMjeev
03/15/2023, 3:48 PMjustin hallquist
03/15/2023, 3:49 PMjeev
03/15/2023, 3:49 PMjustin hallquist
03/15/2023, 3:49 PMjeev
03/15/2023, 3:50 PMjustin hallquist
03/15/2023, 3:57 PMflyte/flyte
chart that works great and we're setting this up for enterprise and would like the ability to have more configuration optionspyflyte run
is what's signing?kubectl port-forward
to the flyte-contour-envoy
service and running pyflyte run
jeev
03/15/2023, 4:07 PMjustin hallquist
03/15/2023, 4:07 PMdefault-env-vars:
- FLYTE_AWS_ENDPOINT:
- FLYTE_AWS_ACCESS_KEY_ID:
- FLYTE_AWS_SECRET_ACCESS_KEY:
need to be set
is that pulled from the workflow/task? I don't see a secret mount option in the k8s pluginjeev
03/15/2023, 4:18 PMjustin hallquist
03/15/2023, 4:20 PMpyflyte run
my understanding is the code is packaged and stored ... somewhere
i think im confused as I can't seem to get that storage to work with either minio or s3jeev
03/15/2023, 4:24 PMjustin hallquist
03/15/2023, 4:24 PMjeev
03/15/2023, 4:24 PMjustin hallquist
03/15/2023, 4:34 PMremoteData:
remoteData:
region: "us-gov-west-1"
scheme: "local"
signedUrls:
durationMinutes: 3
had no comments behind scheme
but it looks sus -- is that what needs to be swapped out?jeev
03/15/2023, 4:56 PMjustin hallquist
03/15/2023, 4:56 PMjeev
03/15/2023, 4:57 PMjustin hallquist
03/15/2023, 5:02 PM