Running into some issues using the

dbt-postgres

utility, when attempting incremental updates on a table. Maybe this is standard behavior, as I am fairly new here, but it seems to be changing the schema of the table. I am attempting to load data into a table managed outside of Meltano, is this not a good idea?

it seems to be changing the schema of the table

What kinds of changes is it making?

It was changing the order of the schema, and removing the primary key from

product_id

. As I have since learned, and correct me if I'm wrong...I shouldn't be using dbt to move data into an operational table that is managed elsewhere. I'm moving towards separating that logic into another job/pipeline where I can just extract the data from the 'mart' table and load it into the operational table. Any feedback on this approach?

jobs:
- name: load_and_transform_data
  tasks:
  - tap-spreadsheets-anywhere target-postgres dbt-postgres:run
- name: transfer_data
  tasks:
  - tap-postgres target-postgres

Is

transfer_data

operating in the same database?

Yes

That's gonna be quite inefficient. Might be worth exploring using a post-hook to insert from the dbt-managed table into the operational one

first job: raw_table -> stg_table -> mart_table Second job -> mart_table -> operational_table