GitHub
06/09/2023, 11:01 AMGitHub
06/09/2023, 2:03 PM<https://github.com/flyteorg/flytepropeller/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flytepropeller/commit/05e837b0810b0f09a205efa4d816d10667cdf923|05e837b0>
- Update flyteplugins to v1.0.67 (#575)
flyteorg/flytepropellerGitHub
06/09/2023, 2:10 PMimage_spec=ImageSpec(
base_image="some-base-image:latest
packages=["a-private-library==1.6.0"
pip_index="<https://private-pip-index/simple>"
)
and the generated build.envd file would look like this
# syntax=v1
def build():
...
config.pip_index(url = "<https://private-pip-index/simple>")
...
Describe alternatives you've considered
I think it's pretty straightforward.
Propose: Link/Inline OR Additional context
envd has built-in support for this, as shown in the main example in their readme
https://github.com/tensorchord/envd#create-an-envd-environment
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/09/2023, 2:40 PMGitHub
06/09/2023, 5:19 PMGitHub
06/09/2023, 5:30 PMGitHub
06/09/2023, 5:50 PMdef task_a() -> (int, int):
# ...
@task
def task_b(m: int) -> int:
# ...
@task
def task_c(n: int, l: int) -> int:
# ...
@workflow
def wf() -> int:
x, y = task_a()
z = task_b(m=x)
return task_c(n=y, l=z)
would create exactly what you mentioned. is that not the case? or do you want to UI to automatically remove the a->c line because it's redundant?
Felix Ruess
yeah, would be nice if the UI would not show the redundant lines...
not sure if they are needed in the backend...
Ketan (kumare3)
Why is it redundant
Ketan (kumare3)
I guess the lines are showing where data feeds from
Ketan (kumare3)
Cc
@Tim Sheiner
Dan Rammer (hamersaw)
IMO it would be more confusing to remove the line.
Felix Ruess
simple example where the extra lines are a bit confusing:
train_workflow▾
extraLines▾
GitHub
06/09/2023, 6:03 PMGitHub
06/09/2023, 6:08 PM<https://github.com/flyteorg/flyte/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flyte/commit/d95ee84333af84a327760062b602c4cd29b072bd|d95ee843>
- Update Flyte components (#3764)
flyteorg/flyteGitHub
06/09/2023, 7:04 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by pingsutw
<https://github.com/flyteorg/flytekit/commit/3370a96fe1df3b484b21f17a5cb919600333ac8c|3370a96f>
- feat: Add Auth0/audience support for ClientCredentials flow (#1639)
flyteorg/flytekitGitHub
06/09/2023, 10:18 PMGitHub
06/10/2023, 1:15 AMGitHub
06/10/2023, 9:13 AMGitHub
06/10/2023, 9:16 AM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by fg91
<https://github.com/flyteorg/flytesnacks/commit/21a4ce1e27d2ea99f16c33db3a9dce924745f014|21a4ce1e>
- Reference new kubeflow training operator instead of deprecated pytorch operator (#996)
flyteorg/flytesnacksGitHub
06/10/2023, 9:55 AMGitHub
06/10/2023, 8:18 PMimage▾
image▾
from flytekit import task, workflow
import torch
@task
def t1():
print("helloee")
@workflow
def wf():
t1()
if __name__ == '__main__':
wf()
Tracking Issue
NA
Follow-up issue
NA
flyteorg/flytekit
Codecov: 71.03% (-0.01%) compared to 8437023
✅ 29 other checks have passed
29/30 successful checksGitHub
06/10/2023, 8:53 PMAsyncAgentExecutorMixin
, and any agent task inherited from it can run locally.
import pandas as pd
from flytekit import kwtypes, StructuredDataset, workflow, task
from flytekitplugins.bigquery import BigQueryTask, BigQueryConfig
bigquery_doge_coin = BigQueryTask(
name=f"bigquery.doge_coin",
inputs=kwtypes(version=int),
query_template="SELECT * FROM `bigquery-public-data.crypto_dogecoin.transactions` WHERE version = @version LIMIT 10;",
output_structured_dataset_type=StructuredDataset,
task_config=BigQueryConfig(ProjectID="flyte-test-340607")
)
@task
def t1(sd: StructuredDataset):
print(sd.open(pd.DataFrame).all())
@workflow
def wf():
sd = bigquery_doge_coin(version=1)
bigquery_doge_coin(version=1)
t1(sd=sd)
if __name__ == '__main__':
wf()
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
NA
Follow-up issue
NA
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
06/11/2023, 6:45 AM<https://github.com/flyteorg/flyte/tree/master|master>
by samhita-alla
<https://github.com/flyteorg/flyte/commit/fb6b5c9be6cbd6acb2be5ebf8c84ba06e0dc95e0|fb6b5c9b>
- Replace installation instructions for pytorch operator with training operator (#3768)
flyteorg/flyteGitHub
06/11/2023, 9:45 AMvalues.yaml
contents are stored in a git repository and deployed using an external tool.
2. There are no existing configuration to pull configuration and/or secrets from existing configmaps or secrets; this makes it difficult inject secrets and configuration from a central vault (e.g. Hashicorp Vault).
3. Potentially long-lived secrets are being kept in ConfigMaps instead of Opaque Secrets; this is generally not idiomatic even if the security of the two option were roughly equivalent.
This is creating a significant barrier to me deploying a test instance in our environment (bare metal K8s).
Goal: What should the final outcome look like, ideally?
I would like to supply names of existing secrets to the Helm chart and/or Kustomization without directly exposing secrets in configuration files or Helm values. This will allow incremental improvement of my deployment configuration without needing to expose secrets directly as part of the deployment.
Describe alternatives you've considered
I've been working on a Kustomization with significant patching to move existing configmaps into secrets... however I am considering a completely parallel deployment because writing the patches takes more time than editing the original files.
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/11/2023, 2:40 PMBaseModel
.
Type
☐ Bug Fix
☐ Feature
☑︎ Plugin
Are all requirements met?
☐ Code completed: The transformer does not yet support all Flyte types in a pydantic model (e.g. data frames)
☑︎ Smoke tested
☑︎ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
The plugin allows passing pydantic models to and between tasks:
Type Example
from pydantic import BaseModel
import flytekitplugins.pydantic
class Config(BaseModel):
lr: float = 1e-3
batch_size: int = 32
@task
def train(cfg: Config):
...
Tracking Issue
flyteorg/flyte#2686
Follow-up issue
NA
flyteorg/flytekit
Codecov: 70.10% (-0.01%) compared to baf0b89
✅ 29 other checks have passed
29/30 successful checksGitHub
06/12/2023, 6:42 AMGitHub
06/12/2023, 9:21 AMGitHub
06/12/2023, 11:51 AMGitHub
06/12/2023, 12:25 PM<https://github.com/flyteorg/flytekit-java/tree/master|master>
by honnix
<https://github.com/flyteorg/flytekit-java/commit/81391ae64c4e7ff079ecdc3983a179c2dde792f7|81391ae6>
- Upgrade aws-java-sdk-bom (#230)
flyteorg/flytekit-javaGitHub
06/12/2023, 12:28 PMGitHub
06/12/2023, 12:45 PM<https://github.com/flyteorg/flytekit-java/tree/master|master>
by honnix
<https://github.com/flyteorg/flytekit-java/commit/89322ff969933970d83a7cfa9891e3683bdf9b6f|89322ff9>
- Remove unused maven build property (#231)
flyteorg/flytekit-javaGitHub
06/12/2023, 12:47 PM<https://github.com/flyteorg/flytekit-java/tree/master|master>
by github-actions[bot]
<https://github.com/flyteorg/flytekit-java/commit/f465f6a6c4755c0624221b08f0837b782b5b1d4d|f465f6a6>
- [maven-release-plugin] prepare release 0.4.19
flyteorg/flytekit-javaGitHub
06/12/2023, 12:47 PM<https://github.com/flyteorg/flytekit-java/tree/master|master>
by github-actions[bot]
<https://github.com/flyteorg/flytekit-java/commit/ac28797b0026dabc3e2c81f89df98b4a171a5b62|ac28797b>
- [maven-release-plugin] prepare for next development iteration
flyteorg/flytekit-javaGitHub
06/12/2023, 2:48 PMExecutionConfig
as defaults for all launchplans started from the execution.
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☑︎ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
fixes flyteorg/flyte#3770
Follow-up issue
NA
flyteorg/flytepropeller
GitHub Actions: Build & Push Flytepropeller Image
GitHub Actions: Goreleaser
GitHub Actions: Bump Version
✅ 11 other checks have passed
11/14 successful checksGitHub
06/12/2023, 3:02 PM<https://github.com/flyteorg/flytepropeller/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flytepropeller/commit/eadf07ba3ac2e54ee4aaf95a98c9941a5a005646|eadf07ba>
- Correctly validating error code on aborting terminal launchplans (#574)
flyteorg/flytepropeller