Zach Palchick
01/26/2022, 10:20 PMfastqc_raw = ContainerTask(
name="fastqc_container",
image="<http://docker.io/staphb/fastqc:latest|docker.io/staphb/fastqc:latest>",
command=[
"fastqc", "--version", ">", "/var/outputs/version"
],
outputs=kwtypes(version=str),
output_data_dir="/var/outputs",
)
Eduardo Apolinario (eapolinario)
01/27/2022, 12:09 AMSYS_PTRACE
?
Can you replace the command you pass to the container with:
command=[
"sh",
"-c",
"fastqc --version | tee /var/outputs/version",
],
Zach Palchick
01/27/2022, 12:09 AMEduardo Apolinario (eapolinario)
01/27/2022, 12:25 AMContainerTask
in your workflow?Zach Palchick
01/27/2022, 12:25 AMEduardo Apolinario (eapolinario)
01/27/2022, 12:31 AMfrom flytekit.core.task import task
from flytekit.core.workflow import workflow
from flytekit import ContainerTask, kwtypes, workflow
fastqc_raw = ContainerTask(
name="fastqc_container",
image="<http://docker.io/staphb/fastqc:latest|docker.io/staphb/fastqc:latest>",
command=[
"sh",
"-c",
"fastqc --version | tee /var/outputs/version",
],
outputs=kwtypes(version=str),
output_data_dir="/var/outputs",
)
@workflow
def wf() -> str:
return fastqc_raw()
Zach Palchick
01/27/2022, 12:32 AMfastqc_raw = ContainerTask(
name="fastqc_container",
image="<http://docker.io/staphb/fastqc:latest|docker.io/staphb/fastqc:latest>",
command=[
"sh",
"-c",
"fastqc -t 1 -q -o /var/outputs/ /var/inputs/fastqc_file"
],
input_data_dir="/var/inputs",
inputs=kwtypes(fastqc_file=FlyteFile),
outputs=kwtypes(out=FlyteDirectory),
output_data_dir="/var/outputs",
)
@workflow
def wf(path: FlyteFile) -> FlyteDirectory:
return fastqc_raw(fastqc_file=path)
Eduardo Apolinario (eapolinario)
01/27/2022, 6:33 PMFlyteFile
is a flytekit concept, so you'd have to pipe that into your raw container task (and use python to download the file locally). Another option is to have a script to download the file from s3 into your container prior to running fastqc
.