Channels
#announcements
Title
z
Zach Palchick
01/26/2022, 10:20 PMis there documentation on how to use container tasks beyond https://docs.flyte.org/projects/cookbook/en/latest/auto/core/containerization/raw_container.html . I'm trying with the sandbox env but not having luck. I've heard that it depends on the SYS_PTRACE capability, which may depend on one's K8s setup
That might not even be the issue. I'm getting this screenshot when I try to run it
...with no logs
the task definition looks like this
Copy code
fastqc_raw = ContainerTask(
name="fastqc_container",
image="<http://docker.io/staphb/fastqc:latest|docker.io/staphb/fastqc:latest>",
command=[
"fastqc", "--version", ">", "/var/outputs/version"
],
outputs=kwtypes(version=str),
output_data_dir="/var/outputs",
)e
Eduardo Apolinario (eapolinario)
01/27/2022, 12:09 AM@Zach Palchick, where did you hear about the need for
SYS_PTRACE Copy code
command=[
"sh",
"-c",
"fastqc --version | tee /var/outputs/version",
],z
Zach Palchick
01/27/2022, 12:09 AMfrom Ketan, but I'll try what you have
same deal. No luck
e
Eduardo Apolinario (eapolinario)
01/27/2022, 12:25 AMinteresting. Is this the only
ContainerTaskz
Zach Palchick
01/27/2022, 12:25 AMyeah
e
Eduardo Apolinario (eapolinario)
01/27/2022, 12:31 AMweird, I just tested this workflow in a brand new sandbox:
Copy code
from flytekit.core.task import task
from flytekit.core.workflow import workflow
from flytekit import ContainerTask, kwtypes, workflow
fastqc_raw = ContainerTask(
name="fastqc_container",
image="<http://docker.io/staphb/fastqc:latest|docker.io/staphb/fastqc:latest>",
command=[
"sh",
"-c",
"fastqc --version | tee /var/outputs/version",
],
outputs=kwtypes(version=str),
output_data_dir="/var/outputs",
)
@workflow
def wf() -> str:
return fastqc_raw()z
Zach Palchick
01/27/2022, 12:32 AMhmm, that is weird. Let me try verbatim what you are doing. It would be nice if there was something else going on
👍 1
ohh...interesting
that did work
that definitely raises some questions
ok, I've made some good progress here. Is there a way to get the raw container input type to be a flytefile? I want the command I call in the container to operate on a file I generate in a previous task
This doesn't work (perhaps unexpectedly) because the fast_qc file I think is just the s3 file path that I pass in from a previous task
Copy code
fastqc_raw = ContainerTask(
name="fastqc_container",
image="<http://docker.io/staphb/fastqc:latest|docker.io/staphb/fastqc:latest>",
command=[
"sh",
"-c",
"fastqc -t 1 -q -o /var/outputs/ /var/inputs/fastqc_file"
],
input_data_dir="/var/inputs",
inputs=kwtypes(fastqc_file=FlyteFile),
outputs=kwtypes(out=FlyteDirectory),
output_data_dir="/var/outputs",
)
@workflow
def wf(path: FlyteFile) -> FlyteDirectory:
return fastqc_raw(fastqc_file=path)e
Eduardo Apolinario (eapolinario)
01/27/2022, 6:33 PMUnfortunately
FlyteFilefastqc👍 1
3 Views