Hi, I'm trying out meltano, currently using stitch...
# announcements
m
Hi, I'm trying out meltano, currently using stitch and pipelinewise. Some initial questions: no MySQL/postgres tap being included is a surprise. These are basic mature db's which I would expect to be in any etl product. Then I've tried running the singer MySQL tap as custom extractor and got the following issue: I cannot select a replication method. I've modified the installed python code to enforce logbased replication. It does sync, but it does not seem to save state. However a general issue that I:m seeing is that it does not seem possible to add properties to individual selected streams (e.g. replication key, replication method). This is important to do so for several integrations besides MySQL as well. Am I missing how to do it?
d
You're not missing anything:
meltano select
does not currently support setting any other catalog metadata properties than
selected
, and there's no way to provide a manually crafted
catalog.json
file instead of using the built-in automatic generation logic. It's because of that current lack of support that we don't claim
tap-mysql
support yet, but I definitely want to fix that. What do you think would be the most promising approach, a smarter
meltano select
(which corresponds to a smarter
select
array in
meltano.yml
), or the ability to manually generate/edit a catalog file and have Meltano use that?
@mathieu I've created two issues: https://gitlab.com/meltano/meltano/-/issues/2026 https://gitlab.com/meltano/meltano/-/issues/2027 Would you mind weighing in there so we can figure out what the next step will be to address these current limitations? 🙂