I'm using

tap-postgres

and I'm trying to figure out why wildcard metadata extractors are overriding more specific metadata extractors; here's my `.yml`:

plugins:
  extractors:
    - name: tap-postgres--secondary-svc-db
      inherit_from: tap-postgres
      metadata:
        # secondary
        secondary-investment_account_owning_entity:
          replication-method: FULL_TABLE
        "secondary-*":
          table-key-properties: [ id ]
          replication-method: INCREMENTAL
          replication-key: modified_date
        # secondary_archive
        "secondary_archive-*_audit":
          replication-method: INCREMENTAL
          replication-key: audit_id
      select:
        # secondary
        - secondary-investment_account_owning_entity
        - secondary-investment_*.*
        - secondary-owning_*.*
        # secondary_archive
        - secondary_archive-investment*_audit.*
        - secondary_archive-owning*_audit.*

I'm receiving the following error message:

singer_sdk.exceptions.InvalidReplicationKeyException: Field 'modified_date' is not in schema for stream 'secondary-investment_account_owning_entity'

I would expect for

secondary-investment_account_owning_entity

that the

FULL_TABLE

replication method would be used and it would not try to inherit the incremental/modified_date of the less specific wildcard; at least this is the behavior I ascertained from the examples in the docs. Do wildcards override more specific settings?

Do wildcards override more specific settings?

I would think so. Can you try this?

metadata:
        "secondary-*":
          table-key-properties: [ id ]
          replication-method: INCREMENTAL
          replication-key: modified_date
        # secondary_archive
        "secondary_archive-*_audit":
          replication-method: INCREMENTAL
          replication-key: audit_id
        # secondary
        secondary-investment_account_owning_entity:
          replication-method: FULL_TABLE