Is there any reason an incremental load would be r...
# troubleshooting
c
Is there any reason an incremental load would be run as a full load? I run the below and a full replication is done every time
meltano --environment={env} elt tap-mssql target-snowflake --select {schema_name}-{table_name}.* --force --state-id {env}:tap-mssql-to-target-snowflake
...
Copy code
extractors:
  - name: tap-mssql
    inherit_from: tap-mssql
    variant: wintersrd
    pip_url: tap-mssql
    select:
    - schema-table.*
    metadata:
      schema-table:
        replication-method: INCREMENTAL
        replication-key: UpdateDate
t
In your meltano.yml are you actually specifying the replication-method and replication-key for each table you're replicating? The snipped shown here defines those values for a stream called "schema-table", which won't have any effect... unless of course your DB schema is named "schema" and the table is named "table". 😉
c
I do actually specify the schema name and table name xD
t
Just checking 😉
I do wonder a little if it's the
--force
option 🤔
j
Hey Connor, did you ever find a resolution for what you were experiencing? I'm having a similar problem.