https://flyte.org logo
#ask-the-community
Title
# ask-the-community
r

Rupsha Chaudhuri

01/10/2023, 7:50 PM
Hi team I want to use map_tasks to process the data from a StructuredDataset.. It’s a large dataset so i’ll need to use
iter()
to access all of it and would like each map task to process 1 chunk. Can map task take a generator instead of a list as the input?
There was a related discussion here
y

Yee

01/10/2023, 9:00 PM
@Niels Bantilan @Kevin Su let’s chat about this?
n

Niels Bantilan

01/10/2023, 9:02 PM
let’s!
y

Yee

01/10/2023, 9:02 PM
free in 30?
n

Niels Bantilan

01/10/2023, 9:12 PM
yep
r

Rupsha Chaudhuri

01/10/2023, 9:16 PM
So am assuming you will let me know what you decide? 🙂
y

Yee

01/10/2023, 9:17 PM
oh you can join too… we’ll let you know
but it might be boring for you
r

Rupsha Chaudhuri

01/10/2023, 9:17 PM
I can join.. in case you need details of my use case
y

Yee

01/10/2023, 9:17 PM
i’ll post the link here when it happens
yup sounds good
r

Rupsha Chaudhuri

01/10/2023, 9:17 PM
Thanks
k

Ketan (kumare3)

01/10/2023, 10:25 PM
can i join?
y

Yee

01/10/2023, 10:25 PM
we’re already done
had two meetings
k

Ketan (kumare3)

01/10/2023, 10:25 PM
haha
y

Yee

01/10/2023, 10:25 PM
@Niels Bantilan is writing up the issue
n

Niels Bantilan

01/10/2023, 10:29 PM
3 Views