been getting this alot lately so ill post again fo...
# singer-taps
k
been getting this alot lately so ill post again for vis. So i have a CDC tap as such
Copy code
- name: tap-reporting-db-source
    inherit_from: tap-postgres-pipelinewise
    namespace: tap_postgres_pipelinewise
    config:
      # <<: *reportingdb_credentials
    select:
    - public-external_balances.*
    - public-direct_deposits.*

    - public-transactions_history.user_id
    - public-transactions_history.currency
    - public-transactions_history.created_at
    - public-transactions_history.updated_at
when i do
meltano select tap-reporting-db-source --list
i get the pic attached. I dont see the
Copy code
public-transactions_history.transaction
field yet i see it in snowflake. i full loaded the table without the transaction column and then reverted to CDC where for the full load rows, the transaction is null but for new rows it appears. Not entirely sure why its doing this. Is there a config im missing or something i am doing wrong here?
shit i found this - https://github.com/transferwise/pipelinewise-tap-postgres/issues/200 which highlights the issues i am facing and doesnt seem to be resolved
👀 1
v
meltanolabs variant does do CDC, you could try it!
k
yes! was looking to migrate , would be next year for sure. for now just removing
refresh_streams_schema(conn_config, streams)
1
👍 1