Hi team I want to use map_tasks to process the data from a StructuredDataset.. It’s a large dataset so i’ll need to use

iter()

to access all of it and would like each map task to process 1 chunk. Can map task take a generator instead of a list as the input?

@Niels Bantilan @Kevin Su let’s chat about this?

let’s!

free in 30?

yep

So am assuming you will let me know what you decide? 🙂

oh you can join too… we’ll let you know

I can join.. in case you need details of my use case

i’ll post the link here when it happens

Thanks

can i join?

we’re already done

haha

@Niels Bantilan is writing up the issue