Hi, can Notebook task run as a AWS Map Task? I tri...
# ask-the-community
j
Hi, can Notebook task run as a AWS Map Task? I tried this:
Copy code
import os
import pathlib

from flytekitplugins.papermill import NotebookTask
from flytekitplugins.awsbatch import AWSBatchConfig

config = AWSBatchConfig(
    platformCapabilities="EC2",
)

backtest_wages = NotebookTask(
    name="test-notebook",
    notebook_path=os.path.join(
        pathlib.Path(__file__).parent.parent.absolute(), "notebooks/test-notebook.ipynb"
    ),
    render_deck=True,
    task_config=config
)
but got an error:
Copy code
logger.debug(f"Starting _dispatch_execute for {task_def.name}")
AttributeError: 'function' object has no attribute 'name'
And when I looked at the command:
Copy code
["pyflyte-execute","--inputs","<s3://fiolito-eks-cluster/metadata/propeller/liquidity-forecast-development-alg58fr86fmkgh9nqnf4/n0/data/inputs.pb>","--output-prefix","<s3://fiolito-eks-cluster/metadata/propeller/liquidity-forecast-development-alg58fr86fmkgh9nqnf4/n0/data/0/0>","--raw-output-data-prefix","<s3://fiolito-eks-cluster/99/alg58fr86fmkgh9nqnf4-n0-0>","--resolver","flytekit.core.python_auto_container.default_task_resolver","--","task-module","flytekitplugins.papermill.task","task-name","_dummy_task_func"]
It seems that the
_dummy_task_func
is wrong and it comes from: https://github.com/flyteorg/flytekit/blob/master/plugins/flytekit-papermill/flytekitplugins/papermill/task.py#L145
k
Hmm seems like a bug
j
🥲
k
Rather not a bug, but a missing feature, support for instance tasks
I remember I had looked at this - cc @Kevin Su do you remember
k
We just support using notebook in the map task, which version of flytekit are you using
j
I just tried 1.7.1b0, still didn’t work: is AWS Batch map task supported?
k
Should be
Hmm must be something with AWS batch, will take a look
j
Should I open an issue in github?
k
yes, please. thank you [flyte-bug]
j
Is there any update on this one?
k
leave a comment in the issue
j
Tried your method, seems no luck @Kevin Su
103 Views