https://flyte.org logo
#ask-the-community
Title
# ask-the-community
g

Greg Linklater

11/23/2023, 1:28 PM
Hi; question regarding SQLAlchemyTask — I was able to connect to postgres without issue, however I tried to connect to a MSSQL database using
pyodbc
, however it appears this is not available on the generic image that is user for those kinds of tasks. This makes sense given that this also requires a custom driver from MS to be installed to be useful. Is it possible to swap out the docker image for
SQLAlchemyTask
? If so, how should I extend or build a new docker image for use with MSSQL?
Also I think we should just simply improve the entire sqlalchemy plugin, Flyte's fast registration system makes the plugin even simpler (the existing plugin was build on some legacy interface) - not wrong, just can be simplified. cc @Samhita Alla
s

Samhita Alla

11/24/2023, 4:27 AM
@Ketan (kumare3) could you elaborate more on how fast registration will make the plugin simpler?
k

Ketan (kumare3)

11/24/2023, 4:28 AM
You don’t need the template passing that we do today. We always have the code now when we fast register and simply installing the plugin with alchemy can solve the problem. Also we can do Similar to Spark where we can set the base image in imagespec
So the code is simply just execute
g

Greg Linklater

02/02/2024, 10:22 AM
@Ketan (kumare3) busy resurrecting a thread here but relevant to past conversation: I’m trying to override
container_image
on the existing
SQLAlchemyTask
to use the image specified for the rest of the workflow, however I am getting an error like this:
Copy code
couldn't parse image reference "{{.images.default.fqn}}:{{.images.default.tag}}": invalid reference format
The errors above are returned from Kubernetes so it looks like the
SQLAlchemyTask
is not resolving the templates. Any idea how I can get
SQLAlchemyTask
to run with the image passed in with
--image
without hard coding it? My config follows:
Copy code
SQLAlchemyTask(
    "example.hello_mssql.sql_task",
    query_template="""
        SELECT @@version version
    """,
    container_image="{{.images.default.fqn}}:{{.images.default.tag}}",
    output_schema_type=DataSchema,
    task_config=SQLAlchemyConfig(
        ...
    ),
    secret_requests=[*secrets.values()],
)
s

Samhita Alla

02/02/2024, 10:55 AM
placeholder string in the image isn't supported in sqlalchemytask. this needs to be wrapped in `get_registerable_container_image` method for that to work. not sure what the implications are.
g

Greg Linklater

02/02/2024, 11:49 AM
I guess hard-coded it is for now…
k

Ketan (kumare3)

02/02/2024, 4:30 PM
That’s a big
Let’s fix it, should be trivial
Just return and call super
s

Samhita Alla

02/02/2024, 5:40 PM
@Greg Linklater would you mind creating an issue? [flyte-core]
g

Greg Linklater

02/07/2024, 12:42 PM
Will do
4 Views