https://linen.dev logo
#plugins-general
Title
# plugins-general
r

ryan_bell

01/04/2024, 1:04 PM
Hey there, Long time user of Meltano here, but I've been running into some issues the last week or so with
tap-bing-ads
, which is a new plugin for the project I'm working in. I've got it all set up and authorized to the point where I can successfully pull in whatever data is relevant to me, which for our use case is the campaign and ads performance reports. However, I've discovered that whatever is controlling replication in this tap doesn't actually keep records distinct (or not in a way that I completely understand). Basically, if I pull in the last week of data for the two streams I mentioned (
campaign_performance_report
,
ads_performance_report
) all is well and good. But the next time I run the tap, instead of ignoring any of the records already in my Postgres DB, it creates copies of each one. So if there were 7 records before, now there are 14. Obviously this can be fixed with some SQL down the line, but I'd rather not have the DB filled with tons of duplicate records. Has anyone else encountered this issue while working with
tap-bing-ads
? I'm assuming that this is the important/relevant line here (see below), but even changing the list of primary keys to be what I think they should be for distinct records, I still get duplicates
Copy code
singer.write_schema('ads', ads_schema, ['Id'])
1