little-cricket-84530
03/25/2024, 4:27 PM[4/4] currentAttempt done. Last Error: USER::[1/1] currentAttempt done. Last Error: USER::[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[flyte-copilot-downloader] terminated with exit code (1). Reason [Error]. Message:
Type to use [iam, accesskey]. (default "iam")
--storage.connection.disable-ssl Disables SSL connection. Should only be used for development.
--storage.connection.endpoint string URL for storage client to connect to.
--storage.connection.region string Region to connect to. (default "us-east-1")
--storage.connection.secret-key string Secret to use when accesskey is set.
--storage.container string Initial container (in s3 a bucket) to create -if it doesn't exist-.'
--storage.defaultHttpClient.timeout string Sets time out on the http client. (default "0s")
--storage.enable-multicontainer If this is true, then the container argument is overlooked and redundant. This config will automatically open new connections to new containers/buckets as they are encountered
--storage.limits.maxDownloadMBs int Maximum allowed download size (in MBs) per call. (default 2)
--storage.stow.config stringToString Configuration for stow backend. Refer to github/graymeta/stow (default [])
--storage.stow.kind string Kind of Stow backend to use. Refer to github/graymeta/stow
--storage.type string Sets the type of storage to configure [s3/minio/local/mem/stow]. (default "s3")
--tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
--token string Bearer token for authentication to the API server
--user string The name of the kubeconfig user to use
--username string Username for basic authentication to the API server
-v, --v Level number for the log level verbosity
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
freezing-airport-6809
freezing-airport-6809
little-cricket-84530
03/25/2024, 4:43 PMlittle-cricket-84530
03/25/2024, 4:44 PMlittle-cricket-84530
03/25/2024, 4:47 PMproud-answer-87162
03/25/2024, 5:07 PMContainerTask
run, or only a seemingly-random subset of them?little-cricket-84530
03/25/2024, 5:08 PMlittle-cricket-84530
03/25/2024, 5:11 PMproud-answer-87162
03/25/2024, 5:11 PMflyte-copilot-downloader
container that was spawned from our ContainerTask
required credentials to the metadata bucket (to fetch an input FlyteFile). those credentials were not being propagated from the PodTemplate containerspec, so we had to manage them independently.proud-answer-87162
03/25/2024, 5:12 PMlittle-cricket-84530
03/25/2024, 5:12 PMproud-answer-87162
03/25/2024, 5:13 PMproud-answer-87162
03/25/2024, 5:16 PMsecret_requests
because it added a prefix that was not configurable. we ended up having to manage the credentials outside of the ContainerTask
definitionglamorous-carpet-83516
03/25/2024, 5:28 PMfreezing-airport-6809
damp-lion-88352
03/26/2024, 12:52 AMdid you see this error before?
little-cricket-84530
03/26/2024, 1:01 AMdamp-lion-88352
03/26/2024, 1:03 AMstrong-soccer-41351
03/27/2024, 12:16 AM{"json":{},"level":"error","msg":"Failed to Get credentials.","ts":"2024-03-26T18:14:27Z"}
{"json":{},"level":"error","msg":"Failed to Get credentials.","ts":"2024-03-26T18:14:32Z"}
{"json":{},"level":"error","msg":"Failed to Get credentials.","ts":"2024-03-26T18:14:37Z"}
i guess what i don't understand so far is what is printing this error since we do have access to our s3 bucket from within this container (i reproduced this issue and can ls the contents of the relevant bucket)strong-soccer-41351
03/27/2024, 12:17 AMstrong-soccer-41351
03/27/2024, 5:25 AMdamp-lion-88352
03/27/2024, 6:57 AMdamp-lion-88352
03/27/2024, 6:58 AM