https://flyte.org logo
Join Slack
Powered by
So I've abandoned the `flytkit-java` SDK as it rea...
# flytekit-java
b
So I've abandoned the 
flytkit-java
SDK as it really doesn't provide much in terms of the FlyteAdmin service. I am wondering if there is a way to co-mingle the Scala type/binding support of the SDK with my gRPC interface. I'd have to provide some kind of shim there. Thoughts?
or do I abandon the SDK altogether
@alert-oil-1341 do you guys use any part of the SDK? If not, how do you manage input/outputs?
a
We just got very familiar w/ 
LiteralMap
🙂
b
If we're just passing JSON around, would we use the Blob support? or...
If you have an example you can share, that'd be awesome
f
@bumpy-match-83743 this is really bad, and we are extremely sorry. We at union have limited resources, and only spotify contributes to the java sdk.
Maybe we should indeed update the docs to drop support for it - cc @brief-window-55364?
b
I would love to see docs that support Java/Scala, but using gRPC bindings. I would hate for users not to leverage the platform because they’re on the JVM and think that it’s not viable. My intention is to contribute something like that once we get things up and running to the point where we’re comfortable with our cases using Flyte.
a
Yea, I'd be glad to help as well, just limited bandwidth during the summer. Since we've just taking the protos and plugged it into Spring Boot project(nothing fancy), we could at least do a guide describing that process
f
@bumpy-match-83743 so is the problem just the client?
You want a Grpc client on Java for Flyte? We recently helped someone use bug generated client with oauth2 and that worked
b
I will probably write up something more Scala specific. There’s enough subtlety on how you would go about it versus a Spring Boot example. I think both would be great.
I think the biggest challenge now is what to do about the data/parameter binding,
I currently have a sandbox environment that is able to trigger workflows via my gRPC binding that were registered via the SDK.
a
At some point when I first started looking into this, I thought that having an equivalent TypeEngine in java would have helped - https://github.com/flyteorg/flytekit/blob/f18668db137f2c927a5f533edcedf610898019fc/flytekit/core/type_engine.py
but I may have missed something in the sdk that provided this
b
Agree. With Scala being such a type-safe language, it'd definitely be nice to leverage the 
flytekit-scala
bindings, so I'm going to investigate that.
I just don't know how deep that cuts, i.e. how much of the SDK I need to leverage. @alert-oil-1341 How do you guys register your workflows?
a
We use python for all workflow development
We've built a custom event processor on top of the Flyte Admin API that allows us to define upstream/downstream dependencies in python and trigger those jobs off of the events. During the process, we intercept the outputs of one workflow, add in some metadata and then pass them as inputs to the next workflow(if required). We also have the ability to trigger Flyte from existing Spring Batch jobs and once again take outputs form those jobs and can use them as inputs to Flyte.
b
I see. In our case, the tasks/workflows/launch-plans would be defined in Scala. We do have some future plans to expose the Python environment,
Open in Slack
PreviousNext
Powered by