Hey team is there a way to import my own modules i...
# ask-the-community
h
Hey team is there a way to import my own modules in imagespec? or is can I only include pypi packages
l
You can use gitsha such as this. (Just an overview)
Copy code
airflow_plugin = "git+<https://github.com/flyteorg/flytekit.git@62bd03b6eab075470e17932a268fc566f643fc8e#subdirectory=plugins/flytekit-airflow>"
image_spec = ImageSpec(base_image="pingsutw/flytekit:v1", packages=["apache-airflow", airflow_plugin], apt_packages=["git"], registry="pingsutw")
j
you can also set
pip_index
on
ImageSpec
, we use AWS CodeArtifact and it works well
h
@L godlike how do I get the git sha for a py file in a repo? @Justin Boutwell do you have an example of how this is done?
j
Copy code
image_spec = ImageSpec(
    # need to aws codeartifact login first, we use direnv to set CODEARTIFACT_AUTH_TOKEN locally to pass into the image
    pip_index="<https://aws>:{0}@DOMAIN-ACCOUNT.d.codeartifact.REGION.amazonaws.com/pypi/REPOSITORY/simple/".format(
        os.environ["CODEARTIFACT_AUTH_TOKEN"]
    ),
    requirements="requirements.txt",
    # we push to ECR so docker login is required prior to running, I'd recommend <https://github.com/awslabs/amazon-ecr-credential-helper>
    registry="<http://ACCOUNT.dkr.ecr.REGION.amazonaws.com|ACCOUNT.dkr.ecr.REGION.amazonaws.com>",
)
h
Copy code
#10 0.731 Collecting git+<https://github.com/blabla/bla.git@606c967f#subdirectory=flyteingest/plugins/mm-tokenizer>
#10 0.732   Cloning <https://github.com/blabla/bla.git> (to revision 606c967f) to /tmp/pip-req-build-by17ht1e
#10 0.737   Running command git clone --filter=blob:none --quiet <https://github.com/blabla/bla.git> /tmp/pip-req-build-by17ht1e
#10 1.505   fatal: could not read Username for '<https://github.com>': No such device or address
#10 1.520   error: subprocess-exited-with-error
@L godlike when running
pyflyte run --remote ..
getting a could not read Username error .. I've followed some of the solutions here https://stackoverflow.com/questions/22147574/github-fatal-could-not-read-username-for-https-github-com-no-such-file-o but cant seem to work it out Did you do something additional beyond this?