GitHub
01/09/2023, 1:22 PMGitHub
01/09/2023, 3:27 PMCaching was disabled for this execution.
I have also seen this warning for *_successful_ tasks (see screenshot) even though I unfortunately cannot come up with a minimal example to consistently reproduce this.
Expected behavior
This warning should not be shown.
Additional context to reproduce
from flytekit import task, workflow
@task(
cache=True,
cache_version="1.0",
)
def fail() -> None:
raise Exception()
@workflow
def workflow():
fail()
Screenshots
Screenshot 2023-01-09 at 16 24 40▾
Screenshot 2023-01-09 at 16 24 45▾
Screenshot 2023-01-09 at 17 20 56▾
Screenshot 2023-01-10 at 10 00 53▾
Screenshot 2023-01-10 at 12 05 56▾
Screenshot 2023-01-10 at 12 05 35▾
GitHub
01/09/2023, 4:54 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jsonporter
<https://github.com/flyteorg/flyteconsole/commit/cdafa9fcce52fd2a2ee3aa6b11b7e761ae62aa7b|cdafa9fc>
- chore(deps): bump minimatch from 3.0.4 to 3.1.2 (#659)
flyteorg/flyteconsoleGitHub
01/09/2023, 4:56 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jsonporter
<https://github.com/flyteorg/flyteconsole/commit/a9e1716f9906fd8530ce9b57dd02a0a42d9c93c1|a9e1716f>
- chore(deps): bump json5 from 1.0.1 to 1.0.2 (#658)
flyteorg/flyteconsoleGitHub
01/09/2023, 5:01 PMGitHub
01/09/2023, 5:01 PMGitHub
01/09/2023, 5:34 PMGitHub
01/09/2023, 6:30 PMGitHub
01/09/2023, 7:02 PMindex.rst
, conf.py
, and integration.rst
flyteorg/flytesnacks
✅ All checks have passed
3/3 successful checksGitHub
01/09/2023, 7:43 PMGitHub
01/09/2023, 7:47 PMGitHub
01/09/2023, 8:51 PMstructured_dataset.open(pd.Dataframe).iter()
Goal: What should the final outcome look like, ideally?
The end user should be able to specify the partition column when they output a structured dataset:
@task
def make_df() -> StructuredDataset:
df = pd.DataFrame.from_records([
{
"id": i,
"partition": (i % 10) + 1,
"name": "".join(
random.choices(string.ascii_uppercase + string.digits, k=10)
)
}
for i in range(1000)
])
return StructuredDataset(dataframe=df, partition_cols=["partition"]) # or ["partition1", "partition2"]
And then consume it like so:
@task
def use_df(dataset: StructuredDataset) -> pd.DataFrame:
output = []
for dd in dataset.open(pd.DataFrame).iter():
print(f"This is a partial dataframe")
print(dd.head(3))
output.append(dd)
return pd.concat(output)
Describe alternatives you've considered
The user needs to implement their own encoder/decoder for this use case.
Propose: Link/Inline OR Additional context
There is a working implementation of this here: https://github.com/flyteorg/flyte-demos/blob/main/flyte_demo/workflows/data_iter.py#L101
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/09/2023, 10:09 PMError: No such option: --image
. This PR puts the arguments in the correct order.
Signed-off-by: Kanyes Thaker kanyes.thaker@gmail.com
flyteorg/flytesnacksGitHub
01/09/2023, 10:12 PMSarama
api to read the config, and send the request to Kafka.
Goal: What should the final outcome look like, ideally?
CloudEvent Config
cloudevents:
enable: true
kafka:
brokers: <kafka_ip>
version: 1.2.1
security_ protocol: SASL_SSL
sasl_config: <config for SASL>
Describe alternatives you've considered
NA
Propose: Link/Inline OR Additional context
Slack Thread:
• https://flyte-org.slack.com/archives/CP2HDHKE1/p1667307468058299
• https://flyte-org.slack.com/archives/CP2HDHKE1/p1667412056307629
• https://flyte-org.slack.com/archives/CP2HDHKE1/p1670577684352189
• https://flyte-org.slack.com/archives/CP2HDHKE1/p1667412056307629
Authentication Methods Overview
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/09/2023, 10:25 PMGitHub
01/09/2023, 10:25 PM<https://github.com/flyteorg/flytetools/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytetools/commit/27211db21d30b8526e4dc06948bc5550efca4ec5|27211db2>
- Publish multi-architecture images (#70)
flyteorg/flytetoolsGitHub
01/09/2023, 10:25 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jeevb
<https://github.com/flyteorg/flyteconsole/commit/567afbe19783949eab9a46a90668c47ede654893|567afbe1>
- Publish multi-architecture image (#663)
flyteorg/flyteconsoleGitHub
01/09/2023, 10:32 PMGitHub
01/09/2023, 10:54 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jeevb
<https://github.com/flyteorg/flyteconsole/commit/516bbffd1e808d5b93f097ed962a26330f96b71f|516bbffd>
- fix: Dependency of release pipeline (#664)
flyteorg/flyteconsoleGitHub
01/09/2023, 10:57 PMGitHub
01/09/2023, 10:57 PMGitHub
01/09/2023, 10:57 PM<https://github.com/flyteorg/datacatalog/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/datacatalog/commit/faa86dbf56cce108f2c0b91f8fa2a99f67c1586f|faa86dbf>
- Publish multi-architecture image (#88)
flyteorg/datacatalogGitHub
01/09/2023, 11:08 PMGitHub
01/09/2023, 11:11 PMGitHub
01/09/2023, 11:15 PM<https://github.com/flyteorg/flytepropeller/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytepropeller/commit/0b5007b1c7157213deff1f9a4c65bec86e9f1b63|0b5007b1>
- Publish multi-architecture image (#514)
flyteorg/flytepropellerGitHub
01/09/2023, 11:45 PM<https://github.com/flyteorg/flyteadmin/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flyteadmin/commit/cb1224b0554347e62ea99e4495d0057f885f4fff|cb1224b0>
- Publish multi-architecture image (#510)
flyteorg/flyteadminGitHub
01/09/2023, 11:46 PMGitHub
01/09/2023, 11:56 PMGitHub
01/09/2023, 11:59 PMGitHub
01/10/2023, 12:24 AM<https://github.com/flyteorg/flyte/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flyte/commit/7b401bc20aad4b665daf466b862868c1ef27a0e9|7b401bc2>
- Update Flyte components (#3221)
flyteorg/flyte