I've been seeing some interesting behavior on the ...
# ask-the-community
a
I've been seeing some interesting behavior on the task decorator when working with a local cluster through Docker Desktop kubernetes on an intel MacOS. I've got a workflow that runs perfectly fine with local
pyflyte run
, but it's failing when I try to set these two arguments in the task decorator:
@task(_secret_requests_=[*AWS_SECRETS], _limits_=Resources(_mem_="1500Mi"))
and then run
--remote
. Either argument works by itself, but when they're both present I get the following error:
Copy code
Request rejected by the API, due to Invalid input.
	Reason: 
	Input Request: {
  "id": {
    "resourceType": "TASK",
    "project": "REDACTED",
    "domain": "development",
    "name": "cmi_orchestration.tasks.data_generation.get_data_gen_params",
    "version": "FwNcirU-K58BewCeTAtVog=="
  },
  "spec": {
    "template": {
      "id": {
        "resourceType": "TASK",
        "project": "REDACTED",
        "domain": "development",
        "name": "tasks.data_generation.get_data_gen_params",
        "version": "FwNcirU-K58BewCeTAtVog=="
      },
      "type": "python-task",
      "metadata": {
        "runtime": {
          "type": "FLYTE_SDK",
          "version": "1.4.1",
          "flavor": "python"
        },
        "retries": {}
      },
      "interface": {
        "inputs": {
          "variables": {
            "input_location": {
              "type": {
                "simple": "STRING"
              },
              "description": "input_location"
            }
          }
        },
        "outputs": {
          "variables": {
            "o0": {
              "type": {
                "simple": "STRING"
              },
              "description": "o0"
            }
          }
        }
      },
      "container": {
        "image": "localhost:32112/REDACTED",
        "args": [
          "pyflyte-fast-execute",
          "--additional-distribution",
          "<s3://my-s3-bucket/cmi/development/WM5JKLAFUOMF4F7AUZ72CWHATI======/scriptmode.tar.gz>",
          "--dest-dir",
          "/root",
          "--",
          "pyflyte-execute",
          "--inputs",
          "{{.input}}",
          "--output-prefix",
          "{{.outputPrefix}}",
          "--raw-output-data-prefix",
          "{{.rawOutputDataPrefix}}",
          "--checkpoint-path",
          "{{.checkpointOutputPrefix}}",
          "--prev-checkpoint",
          "{{.prevCheckpointPrefix}}",
          "--resolver",
          "flytekit.core.python_auto_container.default_task_resolver",
          "--",
          "task-module",
          "tasks.data_generation",
          "task-name",
          "get_data_gen_params"
        ],
        "resources": {
          "limits": [
            {
              "name": "MEMORY",
              "value": "1500Mi"
            }
          ]
        }
      },
      "securityContext": {
        "secrets": [
          {
            "group": "aws-creds",
            "key": "AWS_ACCESS_KEY_ID",
            "mountRequirement": "ENV_VAR"
          },
          {
            "group": "aws-creds",
            "key": "AWS_SECRET_ACCESS_KEY",
            "mountRequirement": "ENV_VAR"
          },
          {
            "group": "aws-creds",
            "key": "AWS_REGION",
            "mountRequirement": "ENV_VAR"
          }
        ]
      }
    },
    "description": {
      "longDescription": {
        "format": "DESCRIPTION_FORMAT_RST"
      }
    }
  }
}
k
Need more info
a
I'll circle back, I think it might be due to some tweaks that were being made to the secrets manager in another part of our library. It would be useful if there was a schema available for what an API call like this should look like. I'm not clear on where it's actually being generated.
k
All the schema is documented- it’s protobuf
a
Would you be able to point me towards its location?
k
Docs page is not loading for me
Will In a bit
152 Views