<#1612 [Issue-3617] Enables FlyteFiles, FlyteDirec...
# flyte-github
a
#1612 [Issue-3617] Enables FlyteFiles, FlyteDirectors, and StructuredDatasets inputs in papermill plugin Pull request opened by peridotml TL;DR This is a first draft for allowing the papermill plugin to access more complex inputs. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Right now the papermill plugin can't use
FlyteFile
or
StructuredDataset
because of a Papermill limitation that everything is json serializable. My strategy is mostly copying
record_outputs
. These are the steps 1. Reserializing the inputs and save to local disk 2. Add a helper function
read_input
where users must give the input type (maybe there is a smart fix) 3.
read_input
loads the protos and uses the type to convert it to the correct format Basically I am using flyte to serialize the inputs locally (see comments), then you just need to do this.
Copy code
### Params
pi = (filled in by papermill)

### In a new cell
pi = read_input(pi, float)

Screenshot 2023-05-02 at 1 42 43 PM

Tracking Issue flyteorg/flyte#3617 Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit All checks have passed 30/30 successful checks