Meltano

has anyone ever seen this?
• running `meltano --environment=prod run tap-postgres target-bigquery`
• using incremental replication on two tables, both with a timestamp `updated_at` replication key
• for one of these tables (several million rows), the replication key _regresses_ to a temporally earlier value at the end of the run
• i’m using `pipelinewise-tap-postgres` 
• the second to last log, for the `pay_period_records` table it has the value `2022-01-24…` while on the last log message, it has the value `2020-05-11`

this results in subsequent `meltano run`s starting at the wrong point. i think i can manually edit the meltano db `state`  table to  be accurate but that’s obviously a hack

the problem appears to be
• this bug <https://github.com/transferwise/pipelinewise-tap-postgres/pull/209>
• no new release since the fix was merged

<@U06BXFG4CQ7> That's a surprising bug! Does that mean your problem is fixed now?

more or less, yeah -- just need to figure out how to either tell Meltano to use a specific commit from that repo, or wait for a new release

<@U06BXFG4CQ7> that was my implicit question :wink: Here you go for the most complicated example of a specific commit + more pip packages: <https://github.com/meltano/squared/blob/4f1508ee170a94a833e08904f0582afb9c728520/data/orchestrate/orchestrators.meltano.yml#L5>.

Don't bother waiting for a new release, they are quite slow with releasing (we're talking weeks here).

ah thank you so much! will give that a shot :raised_hands: