Samhita Alla
LiteralCollection
with multiple literal types? If so, how should the get_literal_type
method be defined since collection_type
can only accept a single LiteralType
? See https://github.com/flyteorg/flytekit/pull/1269/files PR for reference.Ketan (kumare3)
Samhita Alla
to_literal
method. What’s the preferred way to implement this?Ketan (kumare3)
Samhita Alla
Ketan (kumare3)
Samhita Alla
cryptic
11/01/2022, 11:40 AM@pytest.mark.parametrize(
"transformer, python_val",
[
(TensorFlowTensorTransformer(), result_tensor),
],
)
def test_get_literal_type(transformer, python_val):
tf = transformer
lt = tf.get_literal_type(python_val)
> assert lt == LiteralType(blob=BlobType(format=python_val.dtype.name, dimensionality=BlobType.BlobDimensionality.SINGLE))
E assert blob {\n format: "TensorflowTensor"\n}\n == blob {\n format: "float32"\n}\n
E + where blob {\n format: "float32"\n}\n = LiteralType(blob=format: "float32"\n)
E + where format: "float32"\n = BlobType(format='float32', dimensionality=0)
E + where 'float32' = tf.float32.name
E + where tf.float32 = <tf.Tensor: shape=(2, 2), dtype=float32, numpy=\narray([[22., 28.],\n [49., 64.]], dtype=float32)>.dtype
E + and 0 = <class 'flytekit.models.core.types.BlobType.BlobDimensionality'>.SINGLE
E + where <class 'flytekit.models.core.types.BlobType.BlobDimensionality'> = BlobType.BlobDimensionality
tests/flytekit/unit/extras/tensorflow/test_transformations.py:36: AssertionError
______________ test_to_python_value_and_literal[transformer0-float32-python_val0] ______________
Why am I seeing "TensorflowTensor" in this line?
assert blob {\n format: "TensorflowTensor"\n}\n == blob {\n format: "float32"\n}\n
python_val.dtype.name
Samhita Alla
cryptic
11/03/2022, 3:13 PMSamhita Alla
cryptic
11/04/2022, 5:44 AMformat=python_val.dtype.name
, test_tensor.py raises an error (the other test passes)
format=python_val.dtype.name,
E AttributeError: 'property' object has no attribute 'name'
format=python_val.dtype
test_transformations.py raises an error that it cannot accept anything other than unicode/bytes (test_tensor.py passes)
> return _types_pb2.BlobType(format=self.format, dimensionality=self.dimensionality)
E TypeError: tf.float32 has type DType, but expected one of: bytes, unicode
flytekit/models/core/types.py:69: TypeError
str(python_val.dtype)
to resolve the errors. Is it a valid approach?Samhita Alla
dtype.name
should exist.cryptic
11/04/2022, 6:01 AMtests/flytekit/unit/extras/tensorflow/test_tensor.py:8: in <module>
def create_rank1_tensor() -> tf.Tensor:
flytekit/core/task.py:212: in task
return wrapper(_task_function)
flytekit/core/task.py:195: in wrapper
task_instance = TaskPlugins.find_pythontask_plugin(type(task_config))(
flytekit/core/tracker.py:35: in __call__
o = super(InstanceTrackingMeta, cls).__call__(*args, **kwargs)
flytekit/core/python_function_task.py:121: in __init__
super().__init__(
flytekit/core/python_auto_container.py:74: in __init__
super().__init__(
flytekit/core/base_task.py:386: in __init__
interface=transform_interface_to_typed_interface(interface),
flytekit/core/interface.py:220: in transform_interface_to_typed_interface
outputs_map = transform_variable_map(interface.outputs, output_descriptions)
flytekit/core/interface.py:328: in transform_variable_map
res[k] = transform_type(v, descriptions.get(k, k))
flytekit/core/interface.py:346: in transform_type
return _interface_models.Variable(type=TypeEngine.to_literal_type(x), description=description)
flytekit/core/type_engine.py:710: in to_literal_type
res = transformer.get_literal_type(python_type)
flytekit/extras/tensorflow/tensor.py:26: in get_literal_type
format=python_val.dtype.name,
E AttributeError: 'property' object has no attribute 'name'
Samhita Alla
get_literal_type
method?cryptic
11/04/2022, 6:10 AMdef get_literal_type(self, python_val: tf.Tensor) -> LiteralType:
return LiteralType(
blob=_core_types.BlobType(
format=python_val.dtype.name,
dimensionality=_core_types.BlobType.BlobDimensionality.SINGLE,
)
)
str(python_val.dtype.name)
?Samhita Alla
get_literal_type
doesn’t accept python_val
. It needs to be t
that corresponds to the type info.t
to initialize format
. Will need to check.cryptic
11/04/2022, 11:19 AMI’m not sure if we can extract type info fromto initializet
. Will need to check.format
I don't think so. It is something that's associated with the items of
python_val
or tensor objectSamhita Alla
cryptic
11/04/2022, 12:53 PM