Hey all, I’m working on a Oracle-tap project and my ODS is a little quirky when it comes to replication key column names. Instead of something simple like ‘last_updated’ it is instead ‘<tablename>_last_updated’. I’m wondering if there’s any way I can reduce the duplication the metadata declaration to help clean up the yaml file:

plugins:
  extractors:
  - name: tap-oracle-saturn
    inherit_from: tap-oracle
    config:
      filter_schemas: SATURN
      default_replication_method: INCREMENTAL
    select:
    - SATURN-TABLE1.*
    - SATURN-TABLE2.*
    metadata:
      SATURN-TABLE1
        replication-key: TABLE1_ACTIVITY_DATE
      SATURN-TABLE2:
        replication-key: TABLE2_ACTIVITY_DATE

I’ve limited the example to two, but in reality it will probably be between 50 and 100 when I’m done.

I tend to keep my meltano.yml file fairly vanilla because i may wish to ingest different tables in different environments. I tend to control what tables are ingested by what environment variables are set when the job is run. Example.

export TAP_ORACLE__SELECT='["STEVE_TABLE1.*","STEVE_TABLE2.*","STEVE_TABLE3.*"]'

export TAP_ORACLE__METADATA='{"STEVE_TABLE1": {"is-view":false, "replication-method": "FULL_TABLE", "table-key-properties": ["REP_KEY_1"]}}'

The man himself! Thanks Steve. I’ll take a look at Chamber and see what I can come up with!