Hello! I am working on my target and want to figur...
# singer-target-development
a
Hello! I am working on my target and want to figure out a better way to get the
replication_key
for the stream. Previously, I was using
MELTANO_EXTRACT__CATALOG
in the target to find the catalog file and parse it from there. It was a temporary solution that had the consequence of preventing invoking the target for testing and running meltano without the use of a catalog file. I see the SCHEMA message does send over the
bookmark_properties
which is functionally equivalent but the only way I can see to intercept that is to override the
Target.__process_schema_message_
method which I would prefer to not do. Has anyone been able to grab the
replication_key
in their target? If so, how were you able to do so?
e
Hi @Adam Wegscheid, we might just need to update the SDK to expose it. Help me understand your use case.
a
Hey @Edgar Ramírez (Arch.dev). Use case is to sync a transactional database table that lacks a natural key into another database. Without that natural key, we cannot use
key_properties
to merge and avoid duplicate records in the target table. Instead, we need the
replication_key
to delete records in the target table that have a timestamp that would overlap with the new data coming in.