<#3441 [BUG] Type Engine Resolution should properl...
# flytekit
c
#3441 [BUG] Type Engine Resolution should properly support inheritance Issue created by nicholasjng Describe the bug Hello Flyte team, I am trying to define a generic Python type wrapping a single resource (think data frame, string etc.) that is supposed to inherit serialization features from the builtin
FlyteFile
. Everything went well until I tried to define a custom
TypeTransformer
for it, which I wanted to behave exactly like `FlyteFile`'s type transformer with a few additions. Pseudocode: class Resource(Generic[T], FlyteFile): ... class ResourceTransformer(TypeTransformer[Resource]): def to_literal(ctx, python_val, python_type, expected): lit = TypeEngine().to_literal(ctx, python_val, FlyteFile, None) # <- construct a literal representation of the FlyteFile ... # now, do something with the literal like embedding a hash etc. return lit TypeEngine.register(ResourceTransformer(...)) The problem is that the current type resolution will now resolve
FlyteFile
as the appropriate transformer for a
Resource
object, as per this code: https://github.com/flyteorg/flytekit/blob/28da983bba36e243bc671f9ba1aa53a0791efd62/flytekit/core/type_engine.py#L756-L760 There is a TODO comment a little above that sort of acknowledges this problem, and proposes to use an ordered dict, presumably to walk it in reverse and resolve the
Resource
transformer before the
FlyteFile
transformer. This might be brittle though for multiple inheritance use cases like the one here, maybe even with a third resource type sprinkled in that gets registered way after the other two. What I propose instead is trying to walk down the `python_val`'s class hierarchy and greedily match the first type transformer available, roughly like so: # flytekit/core/type_engine.py # begins at the code block linked above # Step # To facilitate cases where users may specify one transformer for multiple types that all inherit from # parent. if hasattr(python_type, "__mro__"): for base_type in inspect.getmro(python_type): try: return cls._REGISTRY[base_type] except KeyError: continue For an inheritance chain like mine with MRO
Resource -> FlyteFile -> ... -> object
, this would lead to a match directly at the
Resource
level. What do you think? Expected behavior
TypeEngine().to_literal(ctx, python_val, python_type, expected)
calls the
ResourceTransformer
class to construct the literal, but instead it calls `FlyteFile`'s associated transformer. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? • Yes Have you read the Code of Conduct? • Yes flyteorg/flyte