Stéphane Burwash
11/17/2023, 3:11 PMget_starting_timestamp
take into account the replication method?
• How can I ensure that the starting timestamp for my backfilling stream is always start_date
?
• Where in the code is the metadata from the catalog applied to the tap? Can the catalog be overwritten by the tap itself?
Thanks!
(Update)
When running with these settings:
metadata:
"*":
# replication_key: updated_at
replication_method: FULL_TABLE
Stream still seems to be registered as incremental
Beginning incremental sync of '<stream_name>'...
Is this normal?
Note: I do manually set the replication method in my tap:
replication_key = "updated_at"
replication_method = "INCREMENTAL"
With the expectation that I can overwrite this in the metadata