Thread
#feature-discussions
    Sohan Shah

    Sohan Shah

    5 months ago
    hey i have a request that would serve the Forecasting platform we’re building at Spotify on top of Flyte. We currently subscribe to a topic containing execution events to populate our UI with visibility on what forecasts have been run/where their outputs are located etc. That’s worked quite well! What Im wondering if a similar topic exists to capture the *Workflow* information created at registration time For context, our platform works as such:1. A model developer registers to Flyte a workflow that represents a forecast model that takes in several assumptions as CSVs at runtime 2. A separate financial analyst writes those assumptions and uploads them to our platform 3. the model can be launched from the platform with the appropriate assumptions by anyone My thought is that if we have an understanding of workflow dependencies at registration time, we can have the platform enforce at assumption upload time that all the correct assumptions are uploaded for each model (with the correct schemas) according to the Flyte version selected. Im sure there are other ways to do this (create a hook at registration time that queries the API for all workflows belonging to the registered version), but through a topic is ideal for us so just want to know feasibility here. Thanks!
    Varun Kulkarni

    Varun Kulkarni

    5 months ago
    following
    Ketan (kumare3)

    Ketan (kumare3)

    5 months ago
    can you explain a little more. do you mean, you want an event for registration?
    Sohan Shah

    Sohan Shah

    5 months ago
    yeah exactly, right after i register a version im able to query Flyte to inspect a workflow associated to that version. For example:
    <https://obol.spotify.net/api/v1/workflows/sp-one-model/development/onemodel.models.premium_revenue._flyte.premium_revenue_onemodel/sohans-sohanv12>
    The contents of this message allow us to make explicit on our platform what the inputs feeding into the premium_revenue_onemodel workflow on the sohans-sohanv12 version should look like. Was wondering the lift to have this data written to a topic as it’s created (at registration time), similar to how execution events are written right away
    cc @Sharon Gong
    Ketan (kumare3)

    Ketan (kumare3)

    5 months ago
    so there is no hook today at the point of registration
    and tbh, that was never in the original design as well
    so the lift might be a decent amount of engineering work
    I think since its just workflows, polling is not a bad idea for now
    Sohan Shah

    Sohan Shah

    5 months ago
    ah got it, yeah will do it by polling for now. There is a ton of value in understanding registration time lineage for us though so def interested if it ever becomes something that makes it to your roadmap
    Ketan (kumare3)

    Ketan (kumare3)

    5 months ago
    ohh
    do you want to file an issue
    we would love to completely understand the use case
    and ofcourse open to collaborating on this
    Sohan Shah

    Sohan Shah

    5 months ago
    yeah for sure 🙂 what’s best place to file issue?
    Ketan (kumare3)

    Ketan (kumare3)

    5 months ago
    Haha
    Sohan Shah

    Sohan Shah

    5 months ago
    lol duh🤦. will do!
    Ketan (kumare3)

    Ketan (kumare3)

    5 months ago
    Thank you