Slackbot
12/21/2022, 2:18 PMZhiyi Li
12/21/2022, 2:18 PMDavid Cupp
12/21/2022, 4:13 PMDavid Cupp
12/21/2022, 6:51 PMname
parameter, or something in the ExecutionSpec
cause them to be deduplicated?Laura Lin
12/21/2022, 7:48 PMInstall an Amazon Loadbalancer Ingress Controller
or to do the SSL cert stuff again right?Klemens Kasseroller
12/22/2022, 10:53 AMfrom dataclasses import dataclass
from typing import Optional
from dataclasses_json import dataclass_json
from flytekit import workflow, task
from flytekit.types.file import FlyteFile
@dataclass_json
@dataclass
class TaskOutput:
file1: FlyteFile
file2: Optional[FlyteFile]
@task
def test_task_1() -> Optional[FlyteFile]:
with open("file11.txt", "w+") as f:
f.write("")
return FlyteFile("file11.txt", remote_path="<s3://test-bucket/tmp/file11.txt>")
@task
def test_task_2() -> TaskOutput:
with open("file21.txt", "w+") as f:
f.write("")
with open("file22.txt", "w+") as f:
f.write("")
return TaskOutput(
FlyteFile("file21.txt", remote_path="<s3://test-bucket/tmp/file21.txt>"),
FlyteFile("file22.txt", remote_path="<s3://test-bucket/tmp/file22.txt>")
)
@workflow
def test_workflow():
test_task_1()
test_task_2()
if __name__ == '__main__':
test_workflow()
Andreas Fehlner
12/22/2022, 1:40 PMhonnix
12/22/2022, 2:42 PMAndrew Korzhuev
12/22/2022, 3:12 PMflytekit.current_context().execution_id.domain
in particular. It’s a bit of a chore that now our libraries have to include 300+mb of deps of flytekit
to just get the runtime env.Evan Sadler
12/22/2022, 3:42 PMLeiqing
12/23/2022, 6:01 AMtags
field in TaskMetadata
https://docs.flyte.org/projects/flyteidl/en/latest/protos/docs/core/core.html#ref-flyteidl-core-taskmetadata
But we don’t find such field in the python representation: https://github.com/flyteorg/flytekit/blob/master/flytekit/models/task.py#L276-L308
Is there still a way to specify that?Pulkit Mishra
12/25/2022, 3:54 PMfrom typing import List
from flytekit import task, workflow, conditional
import base64
import requests
import cv2
@task
def _is_base64(data:bytes) -> bool:
try:
return base64.b64encode(base64.b64decode(data)) == data
except Exception:
return False
@task
def download(uri:str) -> bytes:
#code to download from url
return data
@task
def write_video(data:bytes) -> None:
if _is_base64(data=data):
data = base64.b64decode(data)
with open("input.mp4", "wb") as out_file:
out_file.write(data)
@workflow
def wf(data:str, uri:str, job_id:str) -> None:
write_video(data=conditional("input").if_(data == "1").then(download(uri=uri)).else_().then(data))
however I am running into errors and would really appreciate some help.varsha Parthasarathy
12/26/2022, 4:47 PMground_truth_workflow = flytekit.LaunchPlan.get_or_create(
name="ground_truth_workflow",
workflow=GroundTruthOfflinePCPWorkFlow,
)
Dynamic tasks calls a launch plan.
@flytekit.dynamic
def run:
# Figures out number of iterations
ground_truth_workflow(..)
for each iteration of “run” - is it possible to safely set a timeout ?seunggs
12/27/2022, 5:54 PMpython 3.10
and numpy
:seunggs
12/27/2022, 5:54 PM/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/types.py:323: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar. (This may have returned Python scalars in past versions.
_np.bool: SchemaType.SchemaColumn.SchemaColumnType.BOOLEAN, # type: ignore
Traceback (most recent call last):
File "/opt/venv/bin/pyflyte", line 5, in <module>
from flytekit.clis.sdk_in_container.pyflyte import main
File "/opt/venv/lib/python3.10/site-packages/flytekit/__init__.py", line 195, in <module>
from flytekit.types import directory, file, numpy, schema
File "/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/__init__.py", line 1, in <module>
from .types import (
File "/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/types.py", line 313, in <module>
class FlyteSchemaTransformer(TypeTransformer[FlyteSchema]):
File "/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/types.py", line 323, in FlyteSchemaTransformer
_np.bool: SchemaType.SchemaColumn.SchemaColumnType.BOOLEAN, # type: ignore
File "/opt/venv/lib/python3.10/site-packages/numpy/__init__.py", line 284, in __getattr__
raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'bool'. Did you mean: 'bool_'?
seunggs
12/27/2022, 5:58 PMnumpy>=1.23
Rupsha Chaudhuri
12/28/2022, 5:23 AMRupsha Chaudhuri
12/28/2022, 8:12 PMseunggs
12/29/2022, 4:11 AMpyflyte package
), flytectl register
fails with a strange error I don’t understand:seunggs
12/29/2022, 4:11 AMpanic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x138ddbb]
goroutine 1 [running]:
<http://github.com/flyteorg/flytectl/cmd/core.CommandContext.AdminClient(...)|github.com/flyteorg/flytectl/cmd/core.CommandContext.AdminClient(...)>
/home/runner/work/flytectl/flytectl/cmd/core/cmd_ctx.go:57
<http://github.com/flyteorg/flytectl/cmd/register.register({0x252bf70|github.com/flyteorg/flytectl/cmd/register.register({0x252bf70>, 0xc000116000}, {0x251e598?, 0xc0008d91a0?}, {0x0, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}, ...}, ...)
/home/runner/work/flytectl/flytectl/cmd/register/register_util.go:188 +0x37b
<http://github.com/flyteorg/flytectl/cmd/register.registerFile({0x252bf70|github.com/flyteorg/flytectl/cmd/register.registerFile({0x252bf70>, 0xc000116000}, {0xc000166230, _}, {_, _, _}, {0x0, {0x0, 0x0}, ...}, ...)
/home/runner/work/flytectl/flytectl/cmd/register/register_util.go:606 +0x95b
<http://github.com/flyteorg/flytectl/cmd/register.Register({0x252bf70|github.com/flyteorg/flytectl/cmd/register.Register({0x252bf70>, 0xc000116000}, {0xc0005d9b80?, 0x0?, 0x0?}, 0x36246c0, {0x0, {0x0, 0x0}, {0x0, ...}, ...})
/home/runner/work/flytectl/flytectl/cmd/register/files.go:160 +0x6ca
<http://github.com/flyteorg/flytectl/cmd/register.registerFromFilesFunc(|github.com/flyteorg/flytectl/cmd/register.registerFromFilesFunc(>{0x252bf70, 0xc000116000}, {0xc0005d9b80, 0x1, 0xa}, {0x0, {0x0, 0x0}, {0x0, 0x0}, ...})
/home/runner/work/flytectl/flytectl/cmd/register/files.go:118 +0xcd
<http://github.com/flyteorg/flytectl/cmd/core.generateCommandFunc.func1(0xc00064f900|github.com/flyteorg/flytectl/cmd/core.generateCommandFunc.func1(0xc00064f900>?, {0xc0005d9b80, 0x1, 0xa})
/home/runner/work/flytectl/flytectl/cmd/core/cmd.go:70 +0x93d
<http://github.com/spf13/cobra.(*Command).execute(0xc00064f900|github.com/spf13/cobra.(*Command).execute(0xc00064f900>, {0xc0005d9a40, 0xa, 0xa})
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:856 +0x67c
<http://github.com/spf13/cobra.(*Command).ExecuteC(0xc0008fbb80)|github.com/spf13/cobra.(*Command).ExecuteC(0xc0008fbb80)>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3b4
<http://github.com/spf13/cobra.(*Command).Execute(...)|github.com/spf13/cobra.(*Command).Execute(...)>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902
<http://github.com/flyteorg/flytectl/cmd.ExecuteCmd()|github.com/flyteorg/flytectl/cmd.ExecuteCmd()>
/home/runner/work/flytectl/flytectl/cmd/root.go:137 +0x1e
main.main()
/home/runner/work/flytectl/flytectl/main.go:12 +0x1d
seunggs
12/29/2022, 4:11 AMRupsha Chaudhuri
12/29/2022, 8:13 PMFredrik Lyford
12/30/2022, 6:55 AMMücahit
12/30/2022, 11:46 AMseunggs
01/01/2023, 4:08 AMseunggs
01/01/2023, 4:09 AMseunggs
01/01/2023, 4:09 AM{
"literals": {
"n_samples": {
"scalar": {
"primitive": {
"integer": "1000"
}
}
},
...
"n_estimator": {
"scalar": {
"primitive": {
"integer": "100"
}
}
}
}
}
seunggs
01/01/2023, 4:09 AMseunggs
01/01/2023, 4:09 AMinvalid input n_estimator
seunggs
01/01/2023, 4:11 AM