Hello, good morning! I was wondering if the team is planning to hold office hours today? My team has a quick question on max task nodes and figured it might be a good place to address it. Thanks!! @hallowed-mouse-14616 @tall-lock-23197

Hey @brief-leather-11096, we have office hours setup for Wednesdays, but we have been discussing revisiting the schedule to better trade-off our time devotion with user participation. If you have any thoughts on how we might improve availability, we would be happy to hear them!

Oh right, no worries! Yes, I think we can discuss here and if there's still questions we can attend the regular office hours tomorrow. We recently ran into the node limit for a workflow (it seems to be set to 100). We looked around and made this change to the Admin config map,

registration.yaml: |
    maxWorkflowNodes: 1500

but still hit the same error during registration. For context, the error looked like this during registration:

InvalidArgument desc = number of nodes in workflow
exceeds limit (1426 > 100)

Ok, it looks like FlyteAdmin isn't recognizing the configuration. I think this may need to be declared at another level, but double checking just to make sure. Something like:

registration.yaml: |
    registration:
      maxWorkflowNodes: 1500

Ah, I see! Let me try that variation in how we deployed the yaml. I do feel it would be helpful to know if errors are coming from the way we define values in the manifests. We're also trying to get GPUs on our platform but it's been a little hard to know if the problem is in the definitions, in our underlying kubernetes infra, or elsewhere. But of course, there's tradeoffs as you mentioned!