https://flyte.org logo
Powered by
#announcements
Title
# announcements
j

JP Kosymna

02/17/2022, 7:13 PM
Hey all, I'm having trouble passing an empty list to a map_task. I think it might be a bug, hopefully there is a simple work around I could use for now. In the following example. Workflow 
test_fixed_array_map_task
completes successfully in a few seconds, while workflow 
test_empty_array_map_task
hangs indefinitely with the status QUEUED.
Copy code
from flytekit import map_task, task, workflow

@task
def say_hello(name: str) -> str:
    message = f"hello {name}"
    print(message)
    return message

@workflow
def test_empty_array_map_task() -> list[str]:
    names: list[str] = []
    messages = map_task(say_hello)(name=names)
    return messages

@workflow
def test_fixed_array_map_task() -> list[str]:
    names: list[str] = ['jp', 'ketan']
    messages = map_task(say_hello)(name=names)
    return messages
Let me know if there is any additional information I can provide. Thanks
task config
Copy code
{
  "config": {},
  "id": {
    "resourceType": 1,
    "project": "flytesnacks",
    "domain": "development",
    "name": "flyte_workflows.example.mapper_say_hello_3",
    "version": "v60"
  },
  "type": "container_array",
  "metadata": {
    "runtime": {
      "type": 1,
      "version": "0.26.1",
      "flavor": "python"
    },
    "retries": {}
  },
  "interface": {
    "inputs": {
      "variables": {
        "name": {
          "type": {
            "collectionType": {
              "simple": 3
            }
          },
          "description": "name"
        }
      }
    },
    "outputs": {
      "variables": {
        "o0": {
          "type": {
            "collectionType": {
              "simple": 3
            }
          },
          "description": "o0"
        }
      }
    }
  },
  "taskTypeVersion": 1,
  "container": {
    "command": [],
    "args": [
      "pyflyte-map-execute",
      "--inputs",
      "{{.input}}",
      "--output-prefix",
      "{{.outputPrefix}}",
      "--raw-output-data-prefix",
      "{{.rawOutputDataPrefix}}",
      "--resolver",
      "flytekit.core.python_auto_container.default_task_resolver",
      "--",
      "task-module",
      "flyte_workflows.example",
      "task-name",
      "say_hello"
    ],
    "env": [
      {
        "key": "FLYTE_INTERNAL_IMAGE",
        "value": "<http://518673686532.dkr.ecr.us-west-2.amazonaws.com/flyte/test:v60|518673686532.dkr.ecr.us-west-2.amazonaws.com/flyte/test:v60>"
      }
    ],
    "config": [],
    "ports": [],
    "image": "<http://518673686532.dkr.ecr.us-west-2.amazonaws.com/flyte/test:v60|518673686532.dkr.ecr.us-west-2.amazonaws.com/flyte/test:v60>",
    "resources": {
      "requests": [],
      "limits": []
    }
  }
}
y

Yee

02/17/2022, 7:27 PM
if you have easy access to propeller logs, would you mind doing a kubectl logs <propeller pod> | grep exec id?
j

JP Kosymna

02/17/2022, 7:35 PM
This line just repeats
Copy code
{
  "json": {
    "src": "controller.go:138"
  },
  "level": "info",
  "msg": "==> Enqueueing workflow [flytesnacks-development/l7mc2614ly]",
  "ts": "2022-02-17T19:33:52Z"
}
d

Dan Rammer (hamersaw)

02/17/2022, 8:14 PM
Hey @JP Kosymna, thanks for reporting this. I think it's definitely a bug and have a few ideas where it could be. Do you mind filing an issue? I'll pick it up right away and pile it into all the other map task fixes I'm working on.
j

JP Kosymna

02/17/2022, 8:18 PM
d

Dan Rammer (hamersaw)

02/17/2022, 8:18 PM
Thanks!
j

JP Kosymna

02/17/2022, 8:20 PM
This is blocking a job of mine. If you find any temporary fix please do post back here. Thanks
k

Ketan (kumare3)

02/17/2022, 8:28 PM
@JP Kosymna can you use one element array instead?
j

JP Kosymna

02/17/2022, 8:35 PM
In my workflow I have a few tasks that produce lists of data to operate on later in the flow. I don't know that data ahead of time and in some cases it can be none. I guess I could return a nonsense value in case there is no data. The tasks I'm mapping over could then ignore that value.
d

Dan Rammer (hamersaw)

02/17/2022, 8:54 PM
It looks like this is a specific check to ensure a map tasks size is > 0. I'll get working on a patch so that map tasks with no inputs transition immediately to success.
A nonsense value could certainly work, but obviously isn't a long-term solution. Once I have a PR it wouldn't be difficult to cut a propeller image from the patch and deploy that in your environment? I would be happy to help out with that. Otherwise, this will certainly be included in the next release.
j

JP Kosymna

02/17/2022, 9:32 PM
I already started the nonsense value solution so might stick with that. Is the next release on track for end of the month?
k

Ketan (kumare3)

02/17/2022, 10:44 PM
@Dan Rammer (hamersaw) is working hard on it, we are hopeful all these issues to be addressed. Extremely sorry for the issues
15 Views
Powered by