Hey! When I set replication-key: id in tap-mongodb...
# plugins-general
k
Hey! When I set replication-key: id in tap-mongodb, there is no problem, but it throws an error below when I use it for tap-salesforce. Any ideas?
a
The key part of that error message is the "invalid field" part. It is actually very common for SaaS sources that you can only use fields that are supported by the SaaS (and the tap authors) to be used as incremental extraction keys. In Salesforce case, the native Salesforce internal paradigm is that virtually all Salesforce objects have an internal and automatically managed column called SystemModstamp. You can see the reference to this column in the message above. So, for Salesforce you really shouldn't need to override those fields - automatic instrumental processing should come "for free" workout specifying an incremental key column. Contrasting that with database providers, the database itself rarely, if ever, knows what keys are valid for incremental relocation. Hence, you MongoDB, PostgresDb and all other relational databases, it's been common to pick a column and specify it explicitly using "replication-key" field.
Worth calling out that "replication key" corresponds to a timestamp or other column that tells the tap how to find the "changed" records since the last extraction. If you are looking for the primary key designation, that would in the be "key properties" field. However, everything above also is true about Salesforce handling of primary keys, in that the Salesforce unique/primary key properties are handled automatically and generally shouldn't need to be overridden.
Is this helpful?
k
@aaronsteers very helpful, thanks!!