cryptic
10/18/2022, 3:46 AMtf.tensor
. The tf.io.serialize_tensor
returns a string, should we store it in tf.train.Feature
?
and further put this into tf.train.Example
and use SerializeToString()
method to serialize the proto? Is there a need to store it on to the disk? i'm not sure if storing on disk is possible or notKetan (kumare3)
cryptic
10/18/2022, 4:22 AMPyTorchTypeTransformer
actually doing? In the PyTorchTypeTransformer
there is a step where we are locally saving the tensor/module @Ketan (kumare3)Samhita Alla
torch.Tensor
as a Flyte type. It serializes and deserializes the tensor as the data passage happens among Flyte tasks.cryptic
10/18/2022, 12:28 PMKetan (kumare3)
Samhita Alla
Ketan (kumare3)
Samhita Alla
Ryan Nazareth
10/18/2022, 2:09 PMtf.train.Example
between tasks (this would automatically serialise and deserialise from tfrecord file.). but your data would need to be in tf.train.Example
message format.
I believe there is another issue and work being done on supporting passing tf.tensor
between taskscryptic
10/18/2022, 2:20 PM