<#3691 [BUG] The dictionary is not JSON serializab...
# flytekit
c
#3691 [BUG] The dictionary is not JSON serializable, and it isn't being pickled either Issue created by samhita-alla ### Describe the bug I'm having trouble serializing a complicated dictionary of type
Dict[str, Union[List[List[int]], np.ndarray]]
. The error message indicates that the Union type does not match with the List type.
Copy code
AssertionError: this should be a list and it is not: <class 'list'> vs union_type {\n  variants {\n    collection_type {\n      collection_type {\n        simple: INTEGER\n      }\n    }\n    structure {\n      tag: \"Typed List\"\n    }\n  }\n  variants {\n    blob {\n      format: \"NumpyArray\"\n    }\n    structure {\n      tag: \"Numpy Array\"\n    }\n  }\n}
When I set the type to
FlytePickle
to force flytekit to pickle the data, I get an error message saying that the dictionary should be specified as the type. raise AssertionError(\nAssertionError: this should be a Dictionary type and it is not: <class 'dict'> vs blob {\n format: \"PythonPickle\"\n} ### Expected behavior Since the dictionary isn't JSON serializable, it has to be pickled by default. ### Additional context to reproduce I don't have a reproducible code snippet. The data I'm trying to annotate looks as follows:
Copy code
{'stride': [[232751, 0, 0]], 'input_features': array([[[ 0.02830195, -0.06521094, -0.14700186, ..., -0.6676489 ,
         -0.6676489 , -0.6676489 ],
        [-0.18206358, -0.22672772, -0.35251796, ..., -0.6676489 ,
         -0.6676489 , -0.6676489 ],
        [-0.45926154, -0.4596578 , -0.60333264, ..., -0.6676489 ,
         -0.6676489 , -0.6676489 ],
        ...,
        [-0.6676489 , -0.6676489 , -0.6676489 , ..., -0.6676489 ,
         -0.6676489 , -0.6676489 ],
        [-0.6676489 , -0.6676489 , -0.6676489 , ..., -0.6676489 ,
         -0.6676489 , -0.6676489 ],
        [-0.6676489 , -0.6676489 , -0.6676489 , ..., -0.6676489 ,
         -0.6676489 , -0.6676489 ]]], dtype=float32)}
### Screenshots No response ### Are you sure this issue hasn't been raised already? • Yes ### Have you read the Code of Conduct? • Yes flyteorg/flyte