Hello! I’m a new meltano user, trying to set up my first EL to take data from a postgres instance and push to S3 as parquet files. I believe I’ve followed the setup instructions to install and configure the extractor

tap-postgres

and loader

target-s3

. But when I run

meltano run tap-postgres target-s3

it fails with the error message about

batch_start_time

being missing from the context:

batch_start = self.context["batch_start_time"] cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-11-15T16:48:57.803500Z [info     ] KeyError: 'batch_start_time'   cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-11-15T16:48:57.972244Z [error    ] Loader failed
2023-11-15T16:48:57.972624Z [error    ] Block run completed.

As far as I can tell the postgres extractor works correctly (.json.gzip files are saved locally). I’d be grateful for any help with the possible causes of this error. I’m on a fresh install of meltano 3.1.0 (using pipx) on python 3.11.5 (also tried 3.10.12) and Ubuntu 22.04.3 (also same on mac Sonoma). This is my `meltano.yml`:

version: 1
default_environment: dev
project_id: f8f29104-1e08-4e22-bed2-88aa9137d411
environments:
- name: dev
  config:
    plugins:
      extractors:
      - name: tap-postgres
        config:
          batch_config:
            encoding:
              compression: gzip
              format: jsonl
            storage:
              root: tap-output
          replication_method: FULL_TABLE
        select:
        - public-my_table.*
      loaders:
      - name: target-s3
        config:
          append_date_to_filename: false
          prefix: meltano/dev
          partition_name_enabled: true
          include_process_date: true
          format:
            format_type: parquet
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-postgres
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-postgres.git>
  loaders:
  - name: target-s3
    variant: crowemi
    pip_url: git+<https://github.com/crowemi/target-s3.git>

I don't believe target-s3 implements batch mode, so you can't use batch mode from tap-postgres