flat-exabyte-79377
08/09/2023, 1:36 PM$ pyflyte run --project flytesnacks --domain development --remote basics/hello_world.py my_wf
Failed with Exception Code: USER:ValueError
Value error! Received: 400. Request to send data <azure blob storage URI> failed.
Has anyone seen this before? Is it flytekit
that tries to upload to blob storage or is this error coming from flyteadmin
?
I haven't set up any kind of authentication yet, so that could be related as well. flyteadmin
also doesn't show any logs even with a high log level
Thanks in advance!glamorous-carpet-83516
08/09/2023, 5:28 PMglamorous-carpet-83516
08/09/2023, 5:30 PMsignedUrl:
stowConfigOverride:
endpoint: <http://flyte:30084>
glamorous-carpet-83516
08/09/2023, 5:30 PMflat-exabyte-79377
08/09/2023, 5:30 PMflyteadmin:
?glamorous-carpet-83516
08/09/2023, 5:30 PMflat-exabyte-79377
08/09/2023, 5:30 PMflat-exabyte-79377
08/09/2023, 5:40 PMflat-exabyte-79377
08/09/2023, 5:41 PM<http://flyteadmin:30084>
glamorous-carpet-83516
08/09/2023, 5:52 PMflat-exabyte-79377
08/09/2023, 6:18 PMstorage:
type: custom
bucketName: flyte
custom:
container: "flyte"
enable-multicontainer: true
connection: {}
type: stow
stow:
kind: azure
config:
account: "${storage_account_name}"
key: "${storage_account_key}"
enableMultiContainer: false
The two variables are replaced by terraformflat-exabyte-79377
08/09/2023, 6:38 PMflytectl
that tries to upload something to blob storage? or does it talk to flyteadmin which then tries to upload it?glamorous-carpet-83516
08/09/2023, 7:18 PMglamorous-carpet-83516
08/09/2023, 7:18 PMflat-exabyte-79377
08/09/2023, 7:20 PMflyteadmin
logs 😕 I can't get flyteadmin to show more logs despite increasing the log levelflat-exabyte-79377
08/09/2023, 7:20 PMflat-exabyte-79377
08/09/2023, 7:50 PMflat-exabyte-79377
08/09/2023, 9:50 PMflat-exabyte-79377
08/09/2023, 9:51 PMglamorous-carpet-83516
08/09/2023, 9:52 PMflat-exabyte-79377
08/09/2023, 9:52 PMglamorous-carpet-83516
08/09/2023, 9:53 PMflat-exabyte-79377
08/09/2023, 9:54 PMflat-exabyte-79377
08/10/2023, 10:36 AM--verbose
I saw that my 400 was due to this error:
<?xml version="1.0" encoding="utf-8"?>
<Error>
<Code>MissingRequiredHeader</Code>
<Message>
An HTTP header that's mandatory for this request is not specified.
RequestId:ebd5dfbc-f01e-001d-5074-cb4f26000000
Time:2023-08-10T10:24:58.6553767Z
</Message>
<HeaderName>x-ms-blob-type</HeaderName>
</Error>
I manually patched the code to add x-ms-blob-type: BlockBlob
as a header and the file was successfully uploaded. However, I still got an exception: FlyteValueException: Value error! Received: 201. Request to send data
because this check is too strict.
Happy to make a PR to to at enable passing additional headers if that helpsflat-exabyte-79377
08/10/2023, 1:16 PMglamorous-carpet-83516
08/10/2023, 4:33 PMkind-kite-58745
08/10/2023, 4:51 PMcalm-zoo-68637
08/16/2023, 4:03 PMflat-exabyte-79377
08/16/2023, 4:04 PM