https://flyte.org
Join Slack
I'm looking for some more detailed documentation about flyte checkpoints from the doc it appears ``...
# flyte-support
c
I'm looking for some more detailed documentation about flyte checkpoints from the doc it appears
Copy code
current_context().checkpoint
returns a file like object - but no further details are provided specifically I'd like to know if there are integrations with frameworks like keras (the native keras checkpointing mechanism expects a directory, not a single file) and if not, is there a mechanism to write a new file instead of appending to a single file
f
Cc @boundless-pizza-95864 did you have an integration with huggingface
Yes you can write new files or byte streams
b
We're checkpointing directories (coming from the huggingface trainer in our case). You can use the save/restore methods on a 
Checkpoint
instance they work with directories
👍 1
c
Do you happen to have an example?
b
Here’s what we essentially do for saving. I haven’t used Keras for a while but if you can get the path of the current checkpoint in a callback, you should be able to do the same:
Copy code
cp: Checkpoint = flytekit.current_context().checkpoint
checkpoint_path = <your_local_checkpoint_path>
cp.save(checkpoint_path)
And then to resume:
Copy code
cp: Checkpoint = flytekit.current_context().checkpoint
if cp.prev_exists():
    checkpoint_path = cp.restore(<your_local_checkpoint_path>)
164 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by