Hi can someone help me setting up a tap postgres extractor w Meltano #troubleshooting

Hi, can someone help me setting up a tap-postgres...

willi_langlitz

02/16/2023, 2:18 PM

Hi, can someone help me setting up a tap-postgres extractor with 2 different configurations? If I understood correctly, I should use the inheritance capability for this, right? So I would go with this: (I am using meltano/meltano:1.99.0-python3.7 docker image on windows)

Copy code

- name: tap-postgres-cfg1
  inherit_from: tap-postgres
  namespace: tap_postgres_willi
  pip_url: tap-postgres
  executable: tap-postgres
  capabilities:
  - discover
  - properties
  - state
  settings:
  - name: dbname
  - name: host
  - name: password
  - name: port
  - name: user
  config:
...

- name: tap-postgres-cfg2
  namespace: tap_postgres_willi
  inherit_from: tap-postgres
  pip_url: tap-postgres
  executable: tap-postgres
  capabilities:
  - discover
  - properties
  - state
  settings:
  - name: dbname
  - name: host
  - name: password
  - name: port
  - name: user
  config:
...

But when I try to run this I got the following error: ELT could not be completed: Cannot start extractor: Executable 'tap-postgres' could not be found. Extractor 'tap-postgres-cfg1' may not havebeen installed yet using

meltano install extractor tap-postgres-cfg1

, or the executable name may be incorrect.. I am refering to this: https://docs.meltano.com/guide/configuration#multiple-plugin-configurations with the second code snippet, where direct inheritance is used.

jan_soubusta

02/16/2023, 3:04 PM

Do you declare

tap-postgres

before these two? AFAIK you must. I did it for tap-github:

willi_langlitz

02/16/2023, 3:50 PM

Okay, thanks - that helped. But now it says that no property is selected. But when I change the tap-postgres-cfg1 to tap-postgres and remove the inheritance setting, then it is working properly and meltano is accepting my select setting.

willi_langlitz

02/16/2023, 4:46 PM

message has been deleted

jan_soubusta

02/17/2023, 12:07 PM

What exact statement do you execute?

willi_langlitz

02/17/2023, 1:07 PM

Copy code

docker run -v ${PWD}:/project -w /project meltano/meltano:v1.99.0-python3.7 elt tap-postgres-full-table target-postgres --transform=skip --job_id=tst-import

jan_soubusta

02/17/2023, 2:07 PM

From my experience this should work. Someone smarter than me must help here...

willi_langlitz

02/21/2023, 3:43 PM

Anyone who can help?

willi_langlitz

02/21/2023, 3:46 PM

And thanks @jan_soubusta

jan_soubusta

02/21/2023, 3:49 PM

I am now experiencing the same issue, but only on my localhost with latest meltano version. In the cloud, with v2.12 it works. Going to downgrade it locally and test it again in a while.

jan_soubusta

02/21/2023, 3:49 PM

Btw I am experiencing it iwht different tap - tap-github

Open in Slack

Previous Next