Hi all, I am new to Meltano and the singer-taps an...
# getting-started
a
Hi all, I am new to Meltano and the singer-taps and targets, but it really looks very interesting to me. Right now I am just trying to set up a very basic sample with a tap-postgres and a target-postgres on the same host, but can't make it work :-(. The target is trying to create a table with the tablename <source-schemaname>-<source-tablename>. 2022-01-09T155502.296717Z [info    ] LINE 1: CREATE TABLE target_postgres.tap_postgres-kategorie ("katid"... cmd_type=loader job_id=postgres-to-postgres name=target-postgres The configuration I defined is as follows:  - name: tap-postgres    variant: transferwise    pip_url: pipelinewise-tap-postgres    config:      host:      port: 53722      user: dbt_test      dbname: dbt_test      filter_schemas: tap_postgres      default_replication_method: FULL_TABLE  - name: target-postgres    variant: meltano    pip_url: git+https://github.com/meltano/target-postgres.git    config:      host:      port: 53722      user: dbt_test      dbname: dbt_test      schema: target_postgres What am I doing wrong? Thanks for your help Andreas
v
We need to see the error, posting all the logs would help
a
Do you need any further logs? Thanks.
v
Copy code
2022-01-09T15:55:02.295740Z [info     ] psycopg2.errors.SyntaxError: FEHLER:  Syntaxfehler bei »-« cmd_type=loader job_id=postgres-to-postgres name=target-postgres run_id=b97cfc7c-17a6-4e03-8763-06c7e087e90f stdio=stderr
2022-01-09T15:55:02.296717Z [info     ] LINE 1: CREATE TABLE target_postgres.tap_postgres-kategorie ("katid"... cmd_type=loader job_id=postgres-to-postgres name=target-postgres run_id=b97cfc7c-17a6-4e03-8763-06c7e087e90f stdio=stderr
2022-01-09T15:55:02.300690Z [info     ]                                                  ^ cmd_type=loader job_id=postgres-to-postgres name=target-postgres run_id=b97cfc7c-17a6-4e03-8763-06c7e087e90f stdio=stderr
2022-01-09T15:55:02.303087Z [info     ]                                cmd_type=loader job_id=postgres-to-postgres name=target-postgres run_id=b97cfc7c-17a6-4e03-8763-06c7e087e90f stdio=stderr
2022-01-09T15:55:02.331165Z [debug    ] Deleted configuration at /projects/.meltano/run/elt/postgres-to-postgres/b97cfc7c-17a6-4e03-8763-06c7e087e90f/target.ea19a042-f551-4689-b2f4-4d7ce76a264f.config.json
2022-01-09T15:55:02.336903Z [debug    ] Deleted configuration at /projects/.meltano/run/elt/postgres-to-postgres/b97cfc7c-17a6-4e03-8763-06c7e087e90f/tap.e531febb-02be-4b75-ba7d-6466d121815f.config.json
2022-01-09T15:55:02.337388Z [error    ] Loading failed                 code=1 job_id=postgres-to-postgres message=(see above) name=meltano run_id=b97cfc7c-17a6-4e03-8763-06c7e087e90f
2022-01-09T15:55:02.339173Z [debug    ] ELT could not be completed: Loader failed
Is your issue
imo target-postgres should use quoted identifiers so this should be a target-postgres issue as well, but to work around it don't use that
-
https://hub.meltano.com/targets/postgres points to https://github.com/transferwise/pipelinewise-target-postgres as well, not the target-postgres that you're using I believe, give that a shot maybe the problem with disappear?
a
Hi Derek, I installed the transferwise target and now it is working. Thanks for your help
e
Hi @visch should this Postgres target automatically de-nest? @jo_pearson and I are facing some issues de-nesting a tap that works fine with Redshift pipelinewise target
t
It’s generally up to the target whether to de-nest or not. It doesn’t seem like the transferwise one does this automatically. I’m personally not a fan of automatic unnesting, but I totally get folks want that!
e
Thanks @taylor! With the meltano target postgres, what's best practice to go about de-nesting?
t
dbt is generally the way to go if that’s an option for you
e
got it, the redshift target handles de-nesting well any idea why postgres target wouldn't?
t
different use cases and performance I would guess. postgres usually isn’t used for analytics use cases so have the full JSON might be better?