Adam Mitchell
03/10/2023, 3:30 PMpyflyte run
, but it's failing when I try to set these two arguments in the task decorator: @task(_secret_requests_=[*AWS_SECRETS], _limits_=Resources(_mem_="1500Mi"))
and then run --remote
. Either argument works by itself, but when they're both present I get the following error:
Request rejected by the API, due to Invalid input.
Reason:
Input Request: {
"id": {
"resourceType": "TASK",
"project": "REDACTED",
"domain": "development",
"name": "cmi_orchestration.tasks.data_generation.get_data_gen_params",
"version": "FwNcirU-K58BewCeTAtVog=="
},
"spec": {
"template": {
"id": {
"resourceType": "TASK",
"project": "REDACTED",
"domain": "development",
"name": "tasks.data_generation.get_data_gen_params",
"version": "FwNcirU-K58BewCeTAtVog=="
},
"type": "python-task",
"metadata": {
"runtime": {
"type": "FLYTE_SDK",
"version": "1.4.1",
"flavor": "python"
},
"retries": {}
},
"interface": {
"inputs": {
"variables": {
"input_location": {
"type": {
"simple": "STRING"
},
"description": "input_location"
}
}
},
"outputs": {
"variables": {
"o0": {
"type": {
"simple": "STRING"
},
"description": "o0"
}
}
}
},
"container": {
"image": "localhost:32112/REDACTED",
"args": [
"pyflyte-fast-execute",
"--additional-distribution",
"<s3://my-s3-bucket/cmi/development/WM5JKLAFUOMF4F7AUZ72CWHATI======/scriptmode.tar.gz>",
"--dest-dir",
"/root",
"--",
"pyflyte-execute",
"--inputs",
"{{.input}}",
"--output-prefix",
"{{.outputPrefix}}",
"--raw-output-data-prefix",
"{{.rawOutputDataPrefix}}",
"--checkpoint-path",
"{{.checkpointOutputPrefix}}",
"--prev-checkpoint",
"{{.prevCheckpointPrefix}}",
"--resolver",
"flytekit.core.python_auto_container.default_task_resolver",
"--",
"task-module",
"tasks.data_generation",
"task-name",
"get_data_gen_params"
],
"resources": {
"limits": [
{
"name": "MEMORY",
"value": "1500Mi"
}
]
}
},
"securityContext": {
"secrets": [
{
"group": "aws-creds",
"key": "AWS_ACCESS_KEY_ID",
"mountRequirement": "ENV_VAR"
},
{
"group": "aws-creds",
"key": "AWS_SECRET_ACCESS_KEY",
"mountRequirement": "ENV_VAR"
},
{
"group": "aws-creds",
"key": "AWS_REGION",
"mountRequirement": "ENV_VAR"
}
]
}
},
"description": {
"longDescription": {
"format": "DESCRIPTION_FORMAT_RST"
}
}
}
}
Ketan (kumare3)
Adam Mitchell
03/10/2023, 4:10 PMKetan (kumare3)
Adam Mitchell
03/10/2023, 4:14 PMKetan (kumare3)