acoustic-carpenter-78188
05/16/2026, 12:33 AMstream_model parameter documentation and logic in both SGLangAppEnvironment and VLLMAppEnvironment to clarify that Flyte blob streaming is only enabled when a remote model_path is provided, and is ignored for Hugging Face-only paths.
• Changed the internal logic to set streaming-related environment variables and arguments only when both stream_model is True and model_path is set, preventing accidental streaming activation for Hugging Face-only models.
Testing improvements:
• Enhanced tests for both plugins to assert that streaming is disabled (FLYTE_MODEL_LOADER_STREAM_SAFETENSORS == "false") when only a Hugging Face path is used, and that no streaming-specific arguments are present in such cases.
vLLM custom loader registration and logging:
• Changed the registration of the FlyteModelLoader to occur only if both REMOTE_MODEL_PATH and STREAM_SAFETENSORS are set, preventing unnecessary loader registration.
• Improved logging in the vLLM loader shim to provide visibility into environment variable values and prefetching behavior.
flyteorg/flyte-sdkacoustic-carpenter-78188
05/18/2026, 8:29 PM