New RunLLM response generated: >>>*# Question* Flyte seems to not report stack trace of nested exceptions for recoverable user exceptions. We've been observing this in our training jobs, and I reproduced it minimally via the following:

from sys import exc_info
from flytekit.exceptions.user import FlyteRecoverableException
from flytekit.exceptions.scopes import FlyteScopedUserException
try:
	try:
		raise RuntimeError("a")
	except Exception as e:
		raise FlyteRecoverableException("b") from e
except Exception:
	ex = FlyteScopedUserException(*exc_info()) # this is how Flyte wraps up user exceptions

print(ex.verbose_message)

This produces:

Traceback (most recent call last):

   File "<stdin>", line 5, in <module>

Message:

  FlyteRecoverableException: USER:Recoverable: error=b, cause=a

User error.

See how there is only a single traceback. Compare to:

from flytekit.exceptions.user import FlyteRecoverableException
import traceback
try:
	try:
		raise RuntimeError("a")
	except Exception as e:
		raise FlyteRecoverableException("b") from e
except Exception:
	traceback.print_exc()

which correctly produces 2 stack traces:

Traceback (most recent call last):
 File "<stdin>", line 3, in <module>
RuntimeError: a

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "<stdin>", line 5, in <module>
flytekit.exceptions.user.FlyteRecoverableException: USER:Recoverable: error=b, cause=a

Seems like a problem with

FlyteScopedUserException

. # Answer The issue you're experiencing is related to how

FlyteScopedUserException

handles nested exceptions. According to the Flytekit source code,

FlyteScopedUserException

does not preserve the full stack trace of nested exceptions, which results in only a single traceback being printed. To address this, you can manually print the full stack trace using the

traceback

module, as shown in your second example. This approach ensures that both the original and the nested exceptions are fully reported. Would you like more details on how to handle this within Flyte's exception handling framework? Sources: - Flytekit source code Slack Link https://.slack.com/archives/CP2HDHKE1/p1728515674.311849 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.