Hi, I have query regarding INCREMENTAL approach
- name: tap-rest-api-msdk--github
inherit_from: tap-rest-api-msdk
config:
api_url:
https://api.github.com/
pagination_page_size: 50
pagination_results_limit: 250
streams:
- name: github_users
primary_keys:
- id
path: users
replication_key: id
is_sorted: True
metadata:
'*':
replication-method: INCREMENTAL
replication-key: id
When i run this job, i am also getting this at logs "github_users": {"replication_key": "id", "replication_key_value": 46}
But when i check postgres output columns like _sdc__sync_started_at, it always give current timestamp to all the rows.
So i am not sure whether incremental is working, or it is doing full refresh only