<#998 make fixes to python script cli> Pull reques...
# flyte-github
a
#998 make fixes to python script cli Pull request opened by dansola Summary Testing
flyte run python-script
end-to-end against a Union Cloud tenant surfaced five independent issues. This PR fixes all five. None of the changes alter the public API surface; they are all internal CLI / task runner fixes. Changes 1.
--image <uri>
was unusable
cli/_run_python_script.py
routed
--image
through
flyte.Image.from_ref_name(name)
, which only looks up named images registered in the config's image map. Passing a raw URI (the documented usage:
"Container image URI"
) raised
Image name '...' not found in config. Available: []
. Fix: branch on whether the value looks like a URI (contains
/
or
:
) and use
Image.from_base(image_uri)
in that case, falling back to
from_ref_name
for bare names. 2. No way to bundle anything besides the script itself
_run_python_script.py
called the runner with
_bundle_relative_paths=(script.name,)
. That's fine for self-contained scripts, but any script that imported a sibling helper or read a local config file failed in the pod (
ModuleNotFoundError
,
FileNotFoundError
) with no obvious escape hatch —
copy_style="custom"
was hard-coded and the user had no lever to widen the bundle. Fix: expose the existing
Environment.include
mechanism on the python-script command as
--include-files
(repeatable, accepts paths or globs). Relative entries are anchored at the script's directory via
env._declaring_file = str(script)
(the default stack-walk lands on CLI internals, which would resolve globs against the wrong directory). Absolute paths pass through unchanged. flyte run python-script train.py \ --include-files "*.py" \ --include-files "configs/settings.yaml" Same param is available in the Python API as
include_files=[...]
. Default behavior is unchanged — still just the script — so this is purely additive. 3. Every script registered under the same task identity The task name is computed as
env.name + "." + func.__name__
(
_task_environment.py:319
). Since the env name was hardcoded
"python_script"
and the function name is always
execute_script
, every script ended up registered as
python_script.execute_script
. In the UI this caused the same run IDs to appear under every script-stem grouping (the UI groups runs by
short_name
, but pulls the run list from the shared task identity). Fix: make env name script-specific:
f"python_script_{script.stem}"
. Each script now has a distinct backend task, and the UI groups runs correctly per script. 4. Default resources too heavy for a "hello world"-positioned feature Defaults were
--cpu 4 --memory 16Gi
, which do not fit on common smaller node shapes (e.g. t3a.xlarge) — users would see their first
flyte run python-script hello.py
sit in Pending indefinitely. Users who need more can still override. Fix:
--cpu 1 --memory 2Gi
. 5. No live log streaming The task used
subprocess.run(stdout=tempfile, stderr=tempfile)
inside the pod, capturing all output to tempfiles. Only the last 1000 bytes were returned as
PythonScriptOutput.stdout
after exit. Users could not observe a running script via the logs tab /
kubectl logs
— the script's output appeared nowhere until completion. Fix: switch to
subprocess.Popen
with
stdout=PIPE, stderr=STDOUT
, iterate lines as they arrive, echo each to
sys.stdout
(which the pod emits to its stdout stream → k8s → UI logs tab), and keep a
collections.deque(maxlen=80)
tail for the structured output. Also pass
python -u
for unbuffered interpreter so
print()
flushes line-by-line. flyteorg/flyte-sdk