Specifically: the recommended replication key

_id

does exist in the source database, but I'm finding the tap fails because _tap-mongodb/venv/lib/python3.11/site-packages/singer_sdk/helpers/state.py_ uses this key to index the latest record (

new_rk_value = to_json_compatible(latest_record[replication_key])

on line 221), which produces a KeyError because the

_id

field is missing — but it exists on

latest_record.document

in the form

ObjectId('value')

. I'm proposing that the documentation should recommend the replication key

object_id

, because from my observation that is the actual

'value'

obtained from

latest_record.document._id

. Thoughts? Disclaimer: I could be missing something here, I'm very new to Meltano.

thanks @Edgar Ramírez (Arch.dev)

@Rob Buelke you’re correct that there is an issue in the documentation. The docs say to use `_id`:

metadata:
  '*':
    replication-key: _id
    replication-method: LOG_BASED

I’ve just checked what we’re running in production. It should actually be set to the literal

replication_key

:

metadata:
  '*':
    replication-key: replication_key
    replication-method: LOG_BASED

In the tap-mongodb/meltano.yml file it is correct, but you’re right, that settings page on the hub needs to be updated. If you’d like to contribute that fix, it’d need to be a change made here.