<#2317 feat(extractors): add tap-diffgen (plinxore...
# hub-repo-activity
g
#2317 feat(extractors): add tap-diffgen (plinxore variant) Pull request opened by plinxore Summary Adds
tap-diffgen
(plinxore variant) to the extractors index.
tap-diffgen
is a generic Singer tap for the "diffgen" pattern: an application-level audit-log table that journals every `INSERT`/`UPDATE`/`DELETE` made to other tables in the same database (columns:
id
,
tablename
,
table_id
,
op
,
ts_created
). Instead of reading a target table directly, the tap replays that log since the last bookmark and, per event: • `i`/`u` (insert/update): repulls the affected row fresh from the target table and emits it with
is_deleted: false
. •
d
(delete): emits a tombstone -- just the primary key,
is_deleted: true
, every other business column
null
. This is what a plain timestamp-filtered incremental extraction can never represent on its own: a deleted row doesn't show up in a
WHERE updated_at > ...
query, so deletes silently vanish downstream. Supports PostgreSQL and MySQL. Each configured table also has a
tombstones_only
mode, for a table that already has a separate, conventional incremental tap and just needs the DELETE gap filled. Repo: https://github.com/plinxore/tap-diffgen PyPI: https://pypi.org/project/plinxore-tap-diffgen/ Test plan • Settings scraped from the real, published
plinxore-tap-diffgen==0.1.1
package's
--about --format json
output via
hub-utils add
, not hand-authored. •
uv run python utility_scripts/plugin_definitions/plugin_schema_validate.py
passes. •
uv run python utility_scripts/plugin_definitions/yaml_check_names.py
passes. •
uv run python utility_scripts/plugin_definitions/maintainers_validate.py
passes (the
plinxore
maintainer entry already exists from a prior PR). meltano/hub