Hi, I have made some progress with rust-sdk. There has been a restructuring of the project so it does not depend on

python-plugin

using pyo3.

├── Cargo.toml
├── src
│   ├── condition.rs
│   ├── config
│   │   └── loader.rs
│   ├── config.rs
│   ├── context.rs
│   ├── controller.rs
│   ├── deploy.rs
│   ├── environment.rs
│   ├── error.rs
│   ├── hash.rs
│   ├── idl.rs
│   ├── image.rs
│   ├── interface.rs
│   ├── lib.rs
│   ├── runtime
│   │   ├── local.rs
│   │   ├── remote
│   │   │   ├── client
│   │   │   │   ├── auth.rs
│   │   │   │   └── cluster.rs
│   │   │   ├── client.rs
│   │   │   ├── run.rs
│   │   │   └── task.rs
│   │   └── remote.rs
│   ├── runtime.rs
│   ├── storage.rs
│   ├── submit.rs
│   ├── trace.rs
│   ├── types.rs
│   ├── worker
│   │   ├── contract.rs
│   │   └── execute.rs
│   └── worker.rs
└── tests
    ├── context_propagation.rs
    ├── devbox_deploy.rs
    ├── golden.rs
    ├── image_context_builds.rs
    ├── local_mode.rs
    └── storage_local.rs

I also plan to add a cli using clap to it, so people can register task using rust. Hopefully i can get some time over the weekend to test a few more things and paln to submit it by latest next weekend. Just so everyone one is aware: 1. Created flyteidl rust in flyte repo. 2. Changes

flyte-rs-controller

in flyte-sdk repo. (I seriously think you shoudl take it out from there and put it in its own repo). 3. Flyte-rs-sdk repo with the new idl. 4. Currently your -flyte-go-sdk cannot register new task. I think it will use the rs-controller as well. If all your SDK’s are using/plan-to-be-using it, take it out of flyte-sdk

@freezing-airport-6809 I want some suggestion: I am implementing

flyte run

in rust-sdk 1. There should be two modes for running a task : a) Binary (where the binary will be provided by the user, who would be running

cargo build/release

separately, and the binary along with if/any arguments will be run in the backend ) b) Module : The user will provide the

rust module

path which should have a

cargo.toml

and the build will happen as the part of the task execution in the backend. What do you suggest should be the default : like if the use just runs

flyte run

instead of `flyte run binary`/

flyte run rust-module

.

what is

Flyte-rs-sdk repo with the new idl.

this is the same sdk right

I am trying to almost mimic

python-sdk

where ever it makes sense

I prefer doing image. builds

and if you land on arm vs x86 it could fail

i love the idea but

This can depend on a flag the user provides : arm/amd/webasm etc

I think i will make the target env mandatory for user to provide

Oh, now i see your point. It is a distributed cluster. Then how will a compiled src work for anything. But mostly it will be and X86_64, Only cool kids/ wiz kids use arm for local dev

I just did a bit of reading on it. Cargo build will automatically choose the target for you target env. So if :

FROM rust:1-bookworm
WORKDIR /app
COPY . .
RUN cargo build --release

So the docker image will be pulled based on the target env, since we are not providing any

--platform

tag and cargo will build based on the same env as well. If we pass the

--platform

tag cargo builds as per the provided platform automatically. Currently your Image Class in python-sdk takes platform tags for images.

I think once i push the sdk, one can try and find out to be certain.

github.com/debasisdwivedy/flyte-sdk-rs/tree/upgrade_rust_sdk Above is a V0 working version of sdk in rust without python. You have to use my version of flyteidl (main-v2 and flyte-sdk-rs-controller) This is not yet ready to be merged. It is not yet ready to be merged (logging,testcase,docs) needs to be added. I have generated some test case (unit/intergration) and docs using LLM. Currently only

flyte run

is implemented , everything else has a placeholder. You can try running the example in the example folder:

flyte --config <PATH_TO_CONFIG_YAML> run ./hello-trace my_task --x 21 --label demo

flyte run

takes default a folder with src, but has 2 sub commands

rust-module    Run a rust module/folder containing Cargo.toml on a remote Flyte cluster.
  binary         Run rust binary file on a remote Flyte cluster.

I am super excited and want to test it

But i wanated to share so you guys can have a look and critique/review. I don’t want to move too far before a review