acoustic-carpenter-78188
07/05/2023, 2:23 PMliteral.
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☐ Smoke tested
☑︎ Unit tests added
☐ Code documentation added
☑︎ Any pending items have an associated Issue (there aren't any I guess)
Complete description
This came up during internal tests, where it broke existing tests when I changed from a single Literal to a LiteralMap in a type transformer with an explicitly set hash value.
Minimal reproducer:
# test.py
from flytekit import Literal
from flytekit.core.local_cache import _recursive_hash_placement
from flytekit.models.literals import LiteralMap, Scalar, Primitive
lit = Literal(
map=LiteralMap(
literals={
"hello": Literal(scalar=Scalar(primitive=Primitive(string_value="hello"))),
}
),
hash="0xffff",
)
if __name__ == "__main__":
print(lit)
# --> prints <FlyteLiteral map { literals { key: "hello" value { scalar { primitive { string_value: "hello" } } } } } hash: "0xffff">
print(_recursive_hash_placement(lit))
# --> prints <FlyteLiteral map { literals { key: "hello" value { scalar { primitive { string_value: "hello" } } } } }>
This means that the _recursive_hash_placement function, which facilitates part of the hash key computation, silently scrubs hash values from input literal maps, which can cause involuntary cache hits if the values inside the map stay the same.
I would be happy for a discussion - maybe the repro can go in as a unit test in a suitable location.
Tracking Issue
N/A
Follow-up issue
N/A
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksacoustic-carpenter-78188
07/12/2023, 10:01 PM