Hi I'm using

tap-postgres

to get data from postgres DB and

target-s3-parquet

loader to load data into s3 in .parquet format

plugins:
      extractors:
      - name: tap-postgres
        config:
          host: test_host
          database: test_db
          port: 5432
          user: test_user
        select:
        - test_table.a

...

plugins:
  extractors:
  - name: tap-postgres
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-postgres.git>

  loaders:
  - name: target-s3-csv
    variant: transferwise
    pip_url: git+<https://github.com/transferwise/pipelinewise-target-s3-csv.git>
  
  - name: target-s3-parquet
    variant: gupy-io
    pip_url: git+<https://github.com/gupy-io/target-s3-parquet.git>
    config:
      s3_path: test_s3

When I run

meltano run tap-postgres target-s3-parquet

I'm getting the following error

Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Loader failed

When I try it

target-s3-csv

I can load it with no problem, which means the problem is not related with aws credentials and so on. Can someone help me here? Thank you

Try this https://meltano.slack.com/archives/C069CQNHDNF/p1707132641972759?thread_ts=1706937113.387929&cid=C069CQNHDNF

Perfect!

2024-03-21 15:53:23,249 | ERROR    | target-s3-parquet    | Config validation error: 'athena_database' is a required property

Not sure why 'athenta_database' is a required property, but ok 😞 Thanks a lot for the hint! 😄

You're welcome!

athena_database

seems to be used to construct the path: https://github.com/gupy-io/target-s3-parquet/blob/94b1b000a3bc49005cb85836f80a2b31bc8930aa/target_s3_parquet/sinks.py#L81