Hi i had query regarding how we can create a queue of the bacalhau tasks, so that in the workflow we can choose asynchronously which of the tasks are to be executed or others are to be pre-empted ? thanks

Is this a bacalhau question?

Cc @David Aronchick

Heya! Happy to help

Hi David (sorry i kind of mixed up the slack channels as initially i would've contacted bacalhau-general ) . i wanted to create queue of the various jobSpecs that are given on top of the bacalhau So i wanted to essentially is to implement a queue of the bacalhau jobs (json provided in the array) which create

BacalhauTask

objects and then puts them into queue, once we get the array of json_spec , we'll pop the task and then pass the correponding JobSpec to the given object and execute the job tasks. here is the gist of the pseudocode: https://gist.github.com/dhruvmalik007/17c1b644f799f4c2fe9130d217e9c67b . so basically being a automated version of the chained jobs: https://github.com/bacalhau-project/bacalhau/blob/main/integration/flyte/plugins/flytekit-bacalhau/examples/chained-jobs.py .

Oh interesting! So I guess this really is an airflow thing? Like would you want to create other types of jobs in a queue?

thanks @David Aronchick for response. so the fact that i am not well verse about Airflow but in my understanding i thought that flyte and Airflow are nearly same (DAG frameworks that allow scheduling the job) and that me and my collague decided to use flyte as the normal choice

Thanks! Can we move this to a mail? I don’t want this to get lost- aronchick@expanso.io

thanks and sounds good.