https://flyte.org logo
#ask-ai
Title
# ask-ai
s

Scott Blackwood

03/13/2024, 9:55 PM
@Glime I am trying to run a map_task in the local flyte demo cluster and getting the error
RuntimeExecutionError: failed during plugin execution, caused by: failed to execute handle for plugin [k8s-array]: panic when executing a plugin [k8s-array]. Stack: [goroutine 643 [running]:
. Any idea what is going wrong?
g

Glime

03/13/2024, 9:56 PM
Based on the context provided, it seems like you are encountering a known issue with the execution of map tasks that utilize the
k8s-array
plugin in Flyte. This issue manifests as a panic during the plugin's execution, as indicated by the
RuntimeExecutionError
and the detailed stack trace pointing to a panic occurrence. This problem has been reported in multiple instances on the Flyte discussion forum, indicating that it is not isolated to your environment. The panic could be related to several factors, including but not limited to, issues with handling large numbers of tasks (as seen in one of the context messages where the error occurred when fanning out 10,000 tasks), interactions with Kubernetes resources, or specific configurations of the
k8s-array
plugin. To troubleshoot and potentially resolve this issue, consider the following steps: 1. Review Plugin Configuration: Ensure that the
k8s-array
plugin is correctly configured in your workflow. Misconfigurations can lead to unexpected behavior. 2. Check for Updates: Verify if there are updates or patches for Flyte and the
k8s-array
plugin that address this issue. The Flyte community actively works on resolving bugs, and an update might contain the fix you need. 3. Resource Limits: If your map task involves a large number of tasks, consider if resource limits (e.g., memory and CPU) for the Kubernetes cluster are sufficient and if they need adjustment. 4. Community Support: Since this issue has been discussed in the Flyte community forums, engaging with the community by providing details of your error could yield specific advice or solutions. The community might have identified workarounds or fixes for the issue you are facing. If the problem persists after trying these steps, providing more detailed logs and configuration details when seeking help from the Flyte community or on GitHub issues could facilitate identifying a solution.