Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

<@U0265RTUJ5B> Would it be a proper way to serialize tensors using pickle?

I don’t think so. Is there no serialization technique proposed by TensorFlow?

`to_python_value()` present in the TypeTransformer generally has only three arguments, but when using `tf.io.parse_tensor` I've to additionally pass `python_val` but most of the invocations in various files present in flytekit pass only three arguments, so this is giving me a TypeError when running pytest for one of my tests

This is slightly older (<https://github.com/flyteorg/flytekit/blob/be865e5a97dc8f8095bd78ca3a73acb3ec5121b0/flytekit/extras/tensorflow/tensor/tensor_transformer.py|https://github.com/flyteorg/flytekit/blob/be865e5a97dc8f8095bd78ca3a73acb3ec5121b0/flytekit/extras/tensorflow/tensor/tensor_transformer.py>)
I've have made few more changes locally

I'll also be making the changes you'd proposed soon <@U01J90KBSU9>

Here's the changes i'd made : <https://flyte-org.slack.com/archives/C043P7NDKL6/p1666881279621609?thread_ts=1666848642.919639&amp;cid=C043P7NDKL6|https://flyte-org.slack.com/archives/C043P7NDKL6/p1666881279621609?thread_ts=1666848642.919639&amp;cid=C043P7NDKL6>

I’ve a few suggestions; let me get back to you in an hour.

the out_type is a mandatory argument for `tf.io.parse_tensor`

<@U01J90KBSU9> There seems to be no other way of doing this

1. `filename` should be `local_path` in `to_literal`
2. `tf.io.read_file("tensor_data", name = None)` should be `tf.io.read_file(local_path)`
3. `pathlib.Path(local_path).parent.mkdir(parents=True, exist_ok=True)` isn’t required because `tf.io.write_file` will always creae a directory recursively.
4. tf.io.parse_tensor(read_serial, out_type = python_val.dtype, name = None)
5. You can send the tensor `dtype` using `LiteralCollection`: <https://github.com/flyteorg/flytekit/blob/7c915bb3a9347db09cc3cd2de60c53ddc5718d36/tests/flytekit/unit/core/test_literals_resolver.py#L79-L84>. I took a stab at it: <https://gist.github.com/samhita-alla/013601cd8b3a86a4277cdda54fa51a00>.

Let me know if this still doesn’t work.