hey flyte team, we finally got around to investiga...
# flytekit
d
hey flyte team, we finally got around to investigating migration to structured dataset! We have a few questions about the changes, maybe it makes sense to hop on a call? We're also open to submitting PRs if things make sense cc @Yee (who's on vaca 🙂 ) @Govind Raghu • There are some api structural changes, we're curious if they're intentional (eg requiring Annotated, loss of
open().write()
) • We have an issue setting the bigquery project for perms for bq URIs, there's a couple approaches to fixing this we can consider • There doesn't appear to be any type checking or casting of types based on the schema. this could be pretty valuable
y
nah let’s set up a quick chat.
i can go over the thought process. also writing a blog on this soon so would like to make sure we have it right.
d
cool. monday?
y
for anyone else paying attention, we had a brief chat about this. One point to investigate that came up was around type-casting and strictness. https://github.com/flyteorg/flyte/issues/2794
Also https://github.com/flyteorg/flyte/issues/2795 has been made for moving the bigquery writer to use the lower level library. Dylan mind adding any code snippets you have to that ticket too please?
thanks!
actually dylan or @Govind Raghu would you guys be up for contributing that change? the bigquery one.
158 Views