Hi, I made my own tap which I connected to a rest ...
# troubleshooting
g
Hi, I made my own tap which I connected to a rest API. I also set the replication method to INCREMENTAL. But it doesn't work. Every time i run it with the same state_id, and load it to tap-csv, it gets all the data. But I want it to only get new/updated data. Does anyone know if replication methods are supported for custom taps? Thanks in advance!
e
So, there's a few things you can try. In your custom tap, you have to make sure state is used to filter out old data. If you've built it with the SDK, that's done with https://sdk.meltano.com/en/latest/classes/singer_sdk.Stream.html#singer_sdk.Stream.get_starting_timestamp. If that's looking good, you also may need to make sure your custom plugin definition in
meltano.yml
declares the
state
capability: https://meltano.slack.com/archives/C01PKLU5D1R/p1655906375193099?thread_ts=1655906127.487989&cid=C01PKLU5D1R