Hi there, I’m running into a strange problem using tap-postgres and target-redshift. I’m saving state onto S3 and I can see through debug logs that the retrieval and saving of the state is working but when I run the job it doesn’t advance the LSN in postgres and records don’t seem to be coming through. Below is a snippet of my meltano.yml:

extractors:
  - name: tap-postgres
    variant: meltanolabs
    pip_url: meltanolabs-tap-postgres
    config:
      dates_as_string: false
      database: ${src_dbname}
      host: ${src_host}
      port: ${src_port}
      user: ${src_username}
      password: ${src_password}
      default_replication_method: LOG_BASED
      plugin: wal2json
      json_as_object: false
      max_record_count: 200000
      filter_schemas:
      - public
      replication_slot_name: meltano_tap_postg
    select:
      ... too many tables to list ...
    metadata:
      "*":
        replication_method: LOG_BASED
        replication_key: _sdc_lsn

I know that this is just restricted to log based replication as the full-table sync is working. When running debug logs I don’t see any error messages. Any ideas on what to check?

Hi @Justin Yang! What version of Meltano are you using, and would be able to share the sequence of commands that reproduce this odd behavior?

I’ve used 3.7.4, 3.7.8, and 3.80 version of meltano[s3] the meltano.yml snippet has the extractor, I also have:

jobs:
- name: tap-to-redshift-with-mapper
  tasks:
  - tap-postgres stringify-json-fields target-redshift

then its run with:

meltano run tap-to-redshift-with-mapper

I can tell that its specific to the extractor because I scrapped the target with a simple jsonl. (could be the mapper though looking at the logs I can only see state messages and nothing being emitted