Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-on-gcp
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
integrations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
show-and-tell
sig-large-models
torch-elastic
workflow-building-ui-proj
writing-w-sfloris
Title
r
Rahul Mehta
11/08/2022, 2:41 PMHas anyone noticed behavior where mypy doesn't catch invalid usages inside `@flytekit.task`/`@flytekit.dynamic`? We have a clear case w/ a missing positional argument that we'd typically expect the type-checker to catch, but it seems like the decorator may be causing some unintended behavior here?
k
Ketan (kumare3)
11/08/2022, 3:03 PMInteresting, this is something we probably as community need to work on is adding mypy complexity to flytekit
r
Rahul Mehta
11/08/2022, 11:50 PMhttps://github.com/python/mypy/issues/1551#issuecomment-253978622 this seems related
I noticed this issue elsewhere in our codebase (outside of Flyte), seems like there's something with how the `@task`/`@workflow` decorators work that's preventing mypy from performing proper validation
e
Eduardo Apolinario (eapolinario)
11/10/2022, 6:58 PM@Rahul Mehta, can you share an example of this invalid usage? What does
reveal_typer
Rahul Mehta
11/10/2022, 7:13 PMreveal_typeWe've encountered this on something as basic as this:
@flytekit.task()
def compute_stats(data: List[int]) -> Result:
"""Compute summary stats"""
return Result(
mean=statistics.mean(data),
median=statistics.median(data),
stdev=statistics.stdev(data),
)
@flytekit.task()
def generate_data(n_elements: int, random_min: int, random_max: int) -> List[int]:
"""Generate dummy data"""
return [random.randint(random_min, random_max) for _ in range(n_elements)]
@flytekit.workflow()
def simple_workflow(n_elements: float = 100, random_min: int = 0, random_max: int = 10) -> Result:
"""Generate random data & compute basic summary statistics"""
data = generate_data(n_elements=n_elements, random_min=random_min, random_max=random_max)
return compute_stats(data=data)n_elementsgenerate_dataintfloatThis only seems to be happening with arguments to functions that were decorated w/
@task@dynamice
Eduardo Apolinario (eapolinario)
11/10/2022, 7:43 PMGot it. Thanks. Just so I understand, this is specific to the default argument, right? In any case, this is a bug. I'll open a gh issue to track this. This repros the issue:
import typing
from typing_extensions import reveal_type
from flytekit import task, workflow
@task
def t(n: int) -> int:
return n
@workflow
def wf(n: float = 100) -> int:
return t(n=n)r
Rahul Mehta
11/10/2022, 9:14 PMThis isn't just specific to the default arg; thanks for tracking that
e
Eduardo Apolinario (eapolinario)
11/10/2022, 9:34 PMsure.
You want to tighten the feedback loop, right? Because you'd see this error when serializing the workflows+tasks
r
Rahul Mehta
11/10/2022, 9:35 PMRight, we'd ideally be able to catch these w/ our basic type checking (running that + the linters is a common part of our iteration loop). The complication in
@dynamic