Ketan (kumare3)
02/09/2023, 6:47 PMflytelineage
script
Problems
• This is not resilient, in case of failures does not do well.
• We would ideally love this to be a framework - that folks can add new plugins if needed. For example Amundsen (spotify has an internal catalog) etc
• Currently it simply listens to all events and keeps them in memory for a workflow and then replicates it. This is not needed, it could use flyte remote to get all the data only on receiving a terminal event.
• No one uses this in productionKetan (kumare3)
02/09/2023, 6:48 PMFredrik
02/10/2023, 6:15 AMVictor Gustavo da Silva Oliveira
02/16/2023, 8:24 PMVictor Gustavo da Silva Oliveira
03/06/2023, 4:39 PMVictor Gustavo da Silva Oliveira
03/08/2023, 6:07 PMVictor Gustavo da Silva Oliveira
03/08/2023, 6:07 PM