The other day I was having an issue where Meltano Analyse was giving an error (an SQLAlchemy UndefinedTable) in the UI whenever Meltano tried to run the SQL query. I checked the log output and saw that the search path wasn't being set to the relevant schema (

analytics

), even though the dialect was correctly defined as

postgres

i.e. it seems like this engine trigger wasn't being run https://github.com/meltano/meltano/blob/master/src/meltano/api/controllers/sql_helper.py#L85 I've now resolved the issue but it's still a bit of a mystery to me - I did some more digging and found that for some reason, I seemed to have ended up with versions of SQLAlchemy and flask-sqlalchemy which caused this issue. It only manifested itself when trying to use the Meltano Analyse UI though - all the normal extract/load steps worked fine. I'm running this project in Docker and I'm now pulling from the Meltano 1.79.0 image, so I'm not sure how I ended up with a container which contained these incompatible package versions, but pinning the versions in a

requirements.txt

file inside my project and doing

RUN pip install -r requirements.txt

in the Dockerfile fixed the problem. Specifically, the versions I had were

SQLAlchemy==1.4.23

and

flask-sqlalchemy==2.5.1

. Pinning these to

1.3.19

and

2.4.4

respectively was my fix. For info, this error came about whilst trying to upgrade a Meltano project from

1.69.0

to

1.79.0

. I know that Meltano has moved away from using pip in favour of poetry, so I've got a nagging feeling that my hacky workaround isn't the best approach (and it seems wasteful to install packages as part of the regular Docker install of Meltano and its own dependencies, then re-installing some specific versions using pip), but I haven't had time to look at poetry yet and just wanted to get the UI running again. If anyone has any suggestions about how to solve this properly I'd really like to hear any advice!

Thanks for posting this. It makes sense that this would only affect the UI, since the

flask-sqlalchemy

is the web adapter for sqlalchemy and is not needed for non-web-UI functions. Can I ask if you are installing meltano first or the

requirements.txt

first? If installing meltano directly in a clean environment, I would expect you to get the correct compatible versions. However, if you install into an existing environment where some of these are already installed, the likelihood of version conflicts against anything pre-installed becomes much higher.

Hi @aaronsteers! Thanks for this. To answer your question, I'm installing the requirements after pulling the meltano image. The top of my Dockerfile looks like this:

FROM meltano/meltano:v1.79.0

WORKDIR /project
COPY . .

# Install any additional requirements
RUN pip install -r requirements.txt