Mike Ossareh
06/24/2022, 8:30 PMtyping.List[typing.List[float]]
; it appears to fail to serialize. Is this a known issue / limitation?Transformer.get_transformer
is where the issue is cropping up; but I'm pretty unconvinced that my test case is correctly flexing this code path.@dataclass_json
@dataclass
class Container:
floats: typing.List[typing.List[float]]
ints: typing.List[typing.List[int]]
when passed through json_format.Parse()
and converted into literals it ends up looking like:
collection {
literals {
scalar {
generic {
fields {
key: "floats"
value {
list_value {
values {
list_value {
values {
number_value: 1.2
}
}
}
values {
list_value {
values {
number_value: 4.7
}
}
}
}
}
}
fields {
key: "ints"
value {
list_value {
values {
list_value {
values {
number_value: 1.0
}
}
}
values {
list_value {
values {
number_value: 4.0
}
}
}
}
}
}
}
}
}
}
Yee
Mike Ossareh
06/24/2022, 8:48 PMYee
Mike Ossareh
06/27/2022, 3:06 PMYee
import typing
from dataclasses import dataclass
from dataclasses_json import dataclass_json
from google.protobuf import json_format
from marshmallow_jsonschema import JSONSchema
from flytekit.core.context_manager import FlyteContextManager
from flytekit.core.type_engine import (
TypeEngine,
convert_json_schema_to_python_class,
)
from flytekit.models.literals import Literal, LiteralCollection, Scalar
@dataclass_json
@dataclass()
class Container:
floats: typing.List[typing.List[float]]
ints: typing.List[typing.List[int]]
def test_nested_list_of_float2():
ctx = FlyteContextManager.current_context()
data = Container(floats=[[1.2, 2.4, 3.5], [4.7, 5.8, 6.0]], ints=[[1, 2, 3], [4, 5, 6]])
literal_type = TypeEngine.to_literal_type(Container)
lit = TypeEngine.to_literal(ctx, data, python_type=Container, expected=literal_type)
print(lit)
pv = TypeEngine.to_python_value(ctx, lit, Container)
print(pv)
Mike Ossareh
06/28/2022, 3:48 PMYee
Mike Ossareh
06/28/2022, 6:05 PM