Hey all! In our quest to increase the number of plugins, we'd like to ask which plugins you want Flyte to be integrated with next. Here's what we are thinking, but feel free to let us know your ideas.

I would like to run sandbox locally on an offline machine. I'm encountering the issue above preventing me from restarting the sandbox. Ultimately I want to be able to run the sandbox offline. Is this supported out of the box or will I need to modify?

How do I configure default resource limits for tasks?

Anyone here having issues with running the sandbox environmnet + flytesnacks on a M1 Mac?

[Errors with Flyte task optionally returning None] I have a simple Flyte task which fetches weather data from an API like this

weather_data_op = NamedTuple(
    "WeatherDataOP",
    temperature=float,
    humidity=float,
    wind_speed=float
)

@task(cache=False, limits=Resources(cpu="1", mem="2Gi", ephemeral_storage="2Gi"))
def fetch_weather_data(latitude: float, longitude: float) -> weather_data_op:
    api_key = os.environ["WEATHER_API_KEY"]
    url = f"<http://api.openweathermap.org/data/2.5/weather?lat={latitude}&lon={longitude}&appid={api_key}>"
    res = requests.get(url)
    if not (res.status_code == 200):
        return None
    
    response = res.json()
    return weather_data_op(
        temperature=float(response['main']['temp']),
        humidity=float(response['main']['humidity']),
        wind_speed=float(response['wind']['speed']),
    )

Depending on the input longitude and latitude, the weather API might return None. When I execute this task for the inputs that return in None, Flyte will complain

╭─────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────╮
│ in <module>:4                                                                                                   │
│                                                                                                                 │
│ ❱ 4 fetch_weather_data(latitude=1237.0, longitude=-154.0)                                                       │
│                                                                                                                 │
│ /opt/bitnami/jupyterhub-singleuser/.local/lib/python3.8/site-packages/flytekit/core/base_task.py:304 in         │
│ __call__                                                                                                        │
│                                                                                                                 │
│ ❱ 304 │   │   return flyte_entity_call_handler(self, *args, **kwargs)  # type: ignore                           │
│                                                                                                                 │
│ /opt/bitnami/jupyterhub-singleuser/.local/lib/python3.8/site-packages/flytekit/core/promise.py:1114 in          │
│ flyte_entity_call_handler                                                                                       │
│                                                                                                                 │
│ ❱ 1114 │   │   │   result = cast(LocallyExecutable, entity).local_execute(child_ctx, **kwargs)                  │
│                                                                                                                 │
│ /opt/bitnami/jupyterhub-singleuser/.local/lib/python3.8/site-packages/flytekit/core/base_task.py:285 in         │
│ local_execute                                                                                                   │
│                                                                                                                 │
│ ❱ 285 │   │   │   outputs_literal_map = self.sandbox_execute(ctx, input_literal_map)                            │
│                                                                                                                 │
│ /opt/bitnami/jupyterhub-singleuser/.local/lib/python3.8/site-packages/flytekit/core/base_task.py:351 in         │
│ sandbox_execute                                                                                                 │
│                                                                                                                 │
│ ❱ 351 │   │   return self.dispatch_execute(ctx, input_literal_map)                                              │
│                                                                                                                 │
│ /opt/bitnami/jupyterhub-singleuser/.local/lib/python3.8/site-packages/flytekit/core/base_task.py:569 in         │
│ dispatch_execute                                                                                                │
│                                                                                                                 │
│ ❱ 569 │   │   │   │   │   expected_output_names[i]: native_outputs[i] for i, _ in enumerate(na                  │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: 'NoneType' object is not iterable

So I’m wondering what is a proper way to write a Flyte task that optionally returns None?

Hi flyte team. I am trying to configure s3 storage with iam authentication and I can’t figure out how to pass the role itself. It is assuming service account is setup with the right iam roles similar to eks irsa?

and a quick follow-up: when passing

secret_requests

to

ContainerTask

- how this is made available within container? I don't see

/etc/flyte/secrets

mounted

Any way to run the ShellTask on a specific image right now (besides using another image for the whole workflow)? To my understanding, is neither the task_config option nor the pod_template argument yet possible, right? Can I use ShellTasks right now only with the flyte image that runs by default on tasks?

Also it looks like flytekit will use the

AWS_ACCESS_KEY_ID

env var instead of the also existing

FLYTE_AWS_ACCESS_KEY_ID

env if the former exists in the container.... this made my task which needs to retrieve data from a different bucket with different creds (injected via secret) fail upon downloading/uploading flyte inputs/output. Will dig into it further next week...

I'm looking for some more detailed documentation about flyte checkpoints from the doc it appears

current_context().checkpoint

returns a file like object - but no further details are provided specifically I'd like to know if there are integrations with frameworks like keras (the native keras checkpointing mechanism expects a directory, not a single file) and if not, is there a mechanism to write a new file instead of appending to a single file