#7075 fix: normalize msgpack Binary scalars for deterministic cache hashing Pull request opened by majiayu000 on <!date^1774213854^{date_short}|2026-03-22T21:10:54Z> Tracking issue Closes #6776 Why are the changes needed?

DictTransformer

uses msgpack for serialization, which does not enforce key ordering. When the same dict is serialized on different machines or Python versions, the key order can vary, producing different bytes and therefore different cache hashes. This causes unexpected cache misses for identical inputs. What changes were proposed in this pull request? In

hashify()

(

hashing.go

), added a normalization step for

Binary

scalars with the

msgpack

tag: unmarshal the msgpack bytes, then re-marshal to JSON (which sorts map keys alphabetically at every nesting level). The normalized JSON bytes replace the original value for hashing purposes only — stored data is unchanged. If normalization fails (e.g., non-standard msgpack), the original bytes are used as-is, preserving backward compatibility. Changes: • Added

normalizeMsgpackBytes()

helper in

hashing.go

• Called it from

hashify()

for Binary literals tagged

msgpack

• Added tests covering flat dicts, nested dicts, non-msgpack tags, and invalid input fallback How was this patch tested? Added the following tests in `hashing_test.go`: • `TestHashLiteralMap_MsgpackBinaryDeterministic`: verifies that two msgpack-encoded maps with identical content but different key orders produce the same hash, including nested structures • `TestHashify_NonMsgpackBinaryTag`: verifies Binary literals with non-msgpack tags are not modified • `TestNormalizeMsgpackBytes_InvalidInput`: verifies graceful fallback on invalid msgpack input All existing tests pass:

cd flyteplugins && go test ./go/tasks/pluginmachinery/catalog/ -v

Labels • fixed: Bug fix for non-deterministic cache hashing Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. flyteorg/flyte