Hi, I'm using tap-mysql (from transferwise) and ta...
# troubleshooting
j
Hi, I'm using tap-mysql (from transferwise) and target-postgres (from meltanolabs) to move my data. When I was first setting things up the created target table name had the format
schema_name-table-name
. I'm not sure what changed, but at some point in my development environment it changed to creating tables named the same as the source table
table_name
(which I prefer). I'm now working on deploying Meltano, and in this new environment it's back to including the schema name in the target table name. How can I control this behavior?
1
r
singer-sdk
, which that
target-postgres
is built on, had a recent release (
v0.46.0
) that included this fix to use a stream name as the table name instead of splitting by hyphen if
default_target_schema
is specified (the linked issue has some discussion around this behaviour).
target-postgres
itself also had a recent release that bumped to this new SDK version. So if you are using
default_target_schema
and have reinstalled the target from scratch at some point since its
v0.4.1
release 3 days ago, you should be seeing
table_name
.
in this new environment it's back to including the schema name in the target table name
If you have the target pinned to a specific version in its
pip_url
(i.e. before
0.4.1
) for this environment in particular, then that might explain why you are seeing
schema_name-table-name
again. Or you might just have to
Copy code
meltano install --clean loader target-postgres
j
Ah, that does appear to be it. Thanks for the help! I was under the assumption that the version would be locked because of the
plugins/loaders/target-postgres--meltanolabs.lock
file, but looking at it now that I do have a newer version installed in my deployed environment (so I think I misunderstood what that file is locking). If I understand correctly, to pin plugin versions I should add the version to the
pip_url
in
meltano.yml
?
r
Yeah, it is a bit confusing: the
.lock
is a snapshot of the plugin definition, which doesn't extend to the installed package - that is controlled by the
pip_url
(by default for
target-postgres
, this is unpinned).
If I understand correctly, to pin plugin versions I should add the version to the
pip_url
in
meltano.yml
?
Yep!