bored-laptop-29637
02/25/2022, 2:54 AM@dataclass_json
@dataclass
class Output:
manifest: FlyteFile
catalog: typing.Optional[FlyteFile]
run_results: typing.Optional[FlyteFile]
sources: typing.Optional[FlyteFile]
My first task makes this object and passes it to the next task but the next task errors with an error that there is no file where the manifest file is. Would love to know if maybe I'm doing something I shouldn't be doing or if there is something I need to do to get this to work.freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
bored-laptop-29637
02/25/2022, 3:32 PMbored-laptop-29637
02/25/2022, 3:40 PM@task(cache_version="1", cache=True)
def create_outputs_task(input_dir:FlyteDirectory)->Output:
manifest_path = os.path.join(input_dir,"manifest.json")
output = Output(
manifest=FlyteFile(manifest_path),
run_results=None,
catalog=None,
sources=None
)
open(manifest_path,"a").close()
return output
@task(cache_version="1", cache=True)
def consume_outputs_task(outputs:Output)->Artifact:
with open(output.manifest,"r") as f:
raw_json = json.load(f)
artifact = parse_artifact(raw_json)
return artifact
@workflow
def the_workflow(input_dir:FlyteDirectory)->Artifact
return consume_outputs_task(
outputs=create_outputs_task(
input_dir=input_dir
)
)
glamorous-carpet-83516
02/25/2022, 3:53 PMglamorous-carpet-83516
02/25/2022, 3:53 PMglamorous-carpet-83516
02/25/2022, 3:53 PMglamorous-carpet-83516
02/25/2022, 3:54 PMbored-laptop-29637
02/25/2022, 3:57 PMTraceback (most recent call last):
File "/root/.venv/lib/python3.8/site-packages/flytekit/common/exceptions/scopes.py", line 203, in user_entry_point
return wrapped(*args, **kwargs)
File "/root/.venv/lib/python3.8/site-packages/spotify_dbt_flytekit/tasks/parse_dbt_artifacts.py", line 22, in parse_dbt_artifacts_task
return parse_dbt_artifacts(outputs)
File "/root/.venv/lib/python3.8/site-packages/spotify_dbt_flytekit/tasks/parse_dbt_artifacts.py", line 10, in parse_dbt_artifacts
manifest = parse_manifest(outputs)
File "/root/.venv/lib/python3.8/site-packages/spotify_dbt_flytekit/tasks/parse_manifest.py", line 12, in parse_manifest
with open(outputs.manifest, "r") as f:
Message:
[Errno 2] No such file or directory: '/84/f95f818cb52bf410d9d6-n0-0/ae6a7be9d55ff1a7fe3014fea510a66b/manifest.json'
User error.
glamorous-carpet-83516
02/25/2022, 3:57 PMglamorous-carpet-83516
02/25/2022, 5:25 PMfreezing-airport-6809
glamorous-carpet-83516
02/25/2022, 5:32 PMfreezing-airport-6809