GitHub
06/23/2023, 6:26 PMfrom flytekit.remote import FlyteRemote
from flytekit.configuration import Config
from flytekit import LaunchPlan
from flytekit.configuration import get_config_file
# FlyteRemote object is the main entrypoint to API
remote = FlyteRemote(
config=Config.auto(),
default_project="flytesnacks",
default_domain="development",
)
# Fetch launch plan
flyte_lp = remote.fetch_launch_plan(
name="core.flyte_basics.lp.go_greet", version="v0.3.203", project="flytesnacks", domain="development"
)
# Execute
execution = remote.execute(
flyte_lp, inputs={"day_of_week": "Sun", "number": 1}, execution_name="executionname", wait=True
flyteorg/flytesnacks
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/ray_example)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/pod)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kftensorflow)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfpytorch)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfmpi)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/k8s_spark)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/k8s_dask)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/gcp/bigquery)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/whylogs_examples)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/sql)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/papermilltasks)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/pandera_examples)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/onnx_examples)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/modin_examples)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/mlflow_example)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/greatexpectations)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/duckdb_examples)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/dolt)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/dbt_example)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/external_services/databricks)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/sagemaker_training)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/sagemaker_pytorch)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/batch)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/athena)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/core)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/spark_horovod)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/pima_diabetes)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/nlp_processing)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/mnist_classifier)
GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/house_price_prediction)
✅ No checks have passed
0/30 successful checksGitHub
06/23/2023, 7:02 PMregisterables
helper to importing the full list to the verbose syntax here: https://www.chartjs.org/docs/3.6.2/getting-started/integration.html
From some light reading it seems like registrablables
may be generally buggy, even in modern releases.
flyteorg/flyteconsole
Codecov: 40.32% of diff hit (target 65.80%)
GitHub Actions: Build & Push FlyteConsole Image
GitHub Actions: Get Release Tag
GitHub Actions: Generate Release
✅ 7 other checks have passed
7/11 successful checksGitHub
06/23/2023, 7:06 PM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by pmahindrakar-oss
<https://github.com/flyteorg/flytesnacks/commit/7f73fef88caec6f88789a64087675b32948ece84|7f73fef8>
- Using the auto config loader for flyteremote example (#1009)
flyteorg/flytesnacksGitHub
06/23/2023, 7:14 PM<https://github.com/flyteorg/community/tree/main|main>
by davidmirror-ops
<https://github.com/flyteorg/community/commit/4fca09bf9d381be6f51580ee8401c088a1f2f9a6|4fca09bf>
- Add minutes files
flyteorg/communityGitHub
06/23/2023, 8:01 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jsonporter
<https://github.com/flyteorg/flyteconsole/commit/e2fd2a48dd6d56aca73c90f3e18492b55f77df58|e2fd2a48>
- Fix project selector failing test (#780)
flyteorg/flyteconsoleGitHub
06/23/2023, 9:55 PMimage▾
image▾
image▾
image▾
image▾
image▾
GitHub
06/23/2023, 9:56 PMGitHub
06/24/2023, 3:15 AMGitHub
06/24/2023, 3:52 AMflytekit.Deck.persist()
to generate the html based on current collected metrics/information. So that user can see the real time deck even the task is running or has already failed. See the below example:
import flytekit
from flytekit import Resources, task, workflow
from flytekit.core.utils import timeit
@task(
disable_deck=False,
limits=Resources(mem="4Gi", cpu="1"),
)
def t1():
import time
for i in range(2):
# timeit measure the time used in the block and shown in time line deck. See <https://github.com/flyteorg/flytekit/pull/1581>.
# Or you can add information to your own deck. See <https://docs.flyte.org/projects/cookbook/en/latest/auto/core/flyte_basics/deck.html>.
with timeit(f"iteration {i}"):
time.sleep(50)
flytekit.Deck.persist()
@workflow
def wf():
t1()
if __name__ == "__main__":
wf()
截屏2023-06-23 22 39 08▾
截屏2023-06-23 22 39 23▾
GitHub
06/24/2023, 11:45 PMtyping.Iterator
in the type annotation
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
NA
Follow-up issue
NA
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
06/25/2023, 8:01 AMimage▾
image▾
GitHub
06/25/2023, 2:38 PMbrew install go
command which apparently does not bother exporting GOPATH
environment variable. So if we try to build Flyte code from the source, make compile
command fails with below error.
<http://github.com/flyteorg/flyte/cmd|github.com/flyteorg/flyte/cmd>
mv ./flyte /bin || echo "Skipped copying 'flyte' to /bin"
mv: fastcopy: open() failed (to): /bin/flyte: Operation not permitted
Skipped copying 'flyte' to /bin
Below line in Makefile
is using GOPATH
environment variable.
mv ./flyte ${GOPATH}/bin || echo "Skipped copying 'flyte' to ${GOPATH}/bin"
This is not a show stopper but we can simply check if this environment variable is set and handle the error accordingly.
What if we do not do this?
We have to set GOPATH
environment variable and then retry make compile
command.
Related component(s)
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/25/2023, 3:47 PMGitHub
06/25/2023, 11:37 PMGitHub
06/25/2023, 11:38 PMGitHub
06/25/2023, 11:39 PMGitHub
06/26/2023, 3:49 AMGitHub
06/26/2023, 4:08 AM<https://github.com/flyteorg/flytekit/tree/master|master>
by pingsutw
<https://github.com/flyteorg/flytekit/commit/d7bfa6e9b047678dda1c4baed53260bf5d6c138d|d7bfa6e9>
- Fix config of user facing execution parameters in spawning elastic tasks (#1677)
flyteorg/flytekitGitHub
06/26/2023, 11:38 AMScreen Shot 2023-06-26 at 14 19 37▾
Screen Shot 2023-06-26 at 14 20 39▾
Screen Shot 2023-06-26 at 14 19 49▾
GitHub
06/26/2023, 1:55 PMcycle
in order to keep backward compatibility, and also bigint
will practically make that unnecessary.
For reference https://www.postgresql.org/docs/current/sql-altersequence.html
Tracking Issue
Fixes flyteorg/flyte#3777
Follow-up issue
NA
flyteorg/flyteadmin
GitHub Actions: Goreleaser
GitHub Actions: Build & Push Flyteadmin Image
GitHub Actions: Build & Push Flytescheduler Image
GitHub Actions: Bump Version
Codecov: 0.00% of diff hit (target 58.60%)
Codecov: 58.58% (-0.03%) compared to 949f5a2
✅ 8 other checks have passed
8/14 successful checksGitHub
06/26/2023, 2:19 PMGitHub
06/26/2023, 3:58 PM<https://github.com/flyteorg/flyteadmin/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteadmin/commit/f44d7feedd3158941a409f0da3cc03aa47dc373b|f44d7fee>
- Bump propeller version (#580)
flyteorg/flyteadminGitHub
06/26/2023, 4:01 PMGitHub
06/26/2023, 4:01 PMGitHub
06/26/2023, 4:01 PMGitHub
06/26/2023, 4:37 PM<https://github.com/flyteorg/flytepropeller/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flytepropeller/commit/7b104d335d6fc41684217ce99781390cbd74cf84|7b104d33>
- propagating environment variables through launchplans (#576)
flyteorg/flytepropellerGitHub
06/26/2023, 4:38 PMGitHub
06/26/2023, 5:09 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/aedd721c9b4dbc70c458b99d9e575b3b173e5235|aedd721c>
- Union | optional return types supported (#1703)
flyteorg/flytekitGitHub
06/26/2023, 5:09 PM@task
def transform(encoder: Optional[OneHotEncoder] = None, ...)
def transform(encoder: OneHotEncoder = None, ...)
When caller do transform(encoder=None, ...), the task will work successfully with pyflyte run (locally).
However, if I bake the module containing the task estimate() into an image and refer to it in a pyflyte remote run, no mater what I do, I keep getting at the terminal (before pyflyte submit the code to remote flyte server):
flytekit/core/type_engine.py", line 1060, in to_literal
raise TypeError("Ambiguous choice of variant for union type")
...
flytekit/core/promise.py", line 937, in create_and_link_node
raise AssertionError(f"Failed to Bind variable {k} for function {entity.name}.") from e
AssertionError: Failed to Bind variable **_encoder_** for function dai_mle_models.onehotenc.simple_onehotenc. transform.
Expected behavior
The task should run remotely just like when None is supplied to pandas.DataFrame as task input value (which I have verified working).
e.g.
@task
def sometask(test_data: pd.DataFrame = None, ...)
Additional context to reproduce
No response
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/26/2023, 5:18 PM