Channels
#ask-the-community
Title
# ask-the-community
h
Hud
10/06/2023, 3:33 PMHey team is there a way to import my own modules in imagespec? or is can I only include pypi packages
l
L godlike
10/06/2023, 3:44 PMYou can use gitsha such as this.
(Just an overview)
Copy code
airflow_plugin = "git+<https://github.com/flyteorg/flytekit.git@62bd03b6eab075470e17932a268fc566f643fc8e#subdirectory=plugins/flytekit-airflow>"
image_spec = ImageSpec(base_image="pingsutw/flytekit:v1", packages=["apache-airflow", airflow_plugin], apt_packages=["git"], registry="pingsutw")j
Justin Boutwell
10/06/2023, 3:47 PMyou can also set
pip_indexImageSpech
Hud
10/06/2023, 4:05 PM@L godlike how do I get the git sha for a py file in a repo?
@Justin Boutwell do you have an example of how this is done?
j
Justin Boutwell
10/06/2023, 4:11 PM Copy code
image_spec = ImageSpec(
# need to aws codeartifact login first, we use direnv to set CODEARTIFACT_AUTH_TOKEN locally to pass into the image
pip_index="<https://aws>:{0}@DOMAIN-ACCOUNT.d.codeartifact.REGION.amazonaws.com/pypi/REPOSITORY/simple/".format(
os.environ["CODEARTIFACT_AUTH_TOKEN"]
),
requirements="requirements.txt",
# we push to ECR so docker login is required prior to running, I'd recommend <https://github.com/awslabs/amazon-ecr-credential-helper>
registry="<http://ACCOUNT.dkr.ecr.REGION.amazonaws.com|ACCOUNT.dkr.ecr.REGION.amazonaws.com>",
)l
L godlike
10/06/2023, 4:18 PMh
Hud
10/07/2023, 3:20 AM Copy code
#10 0.731 Collecting git+<https://github.com/blabla/bla.git@606c967f#subdirectory=flyteingest/plugins/mm-tokenizer>
#10 0.732 Cloning <https://github.com/blabla/bla.git> (to revision 606c967f) to /tmp/pip-req-build-by17ht1e
#10 0.737 Running command git clone --filter=blob:none --quiet <https://github.com/blabla/bla.git> /tmp/pip-req-build-by17ht1e
#10 1.505 fatal: could not read Username for '<https://github.com>': No such device or address
#10 1.520 error: subprocess-exited-with-errorpyflyte run --remote ..3 Views