zafar
03/02/2022, 3:59 PMinherit_from
the same tap and deployed Meltano app in container and successfully ran a pipeline. After removing inherit_from
and configuring plugin separately (the same tap but with different name) in meltano.yml file, Meltano gives the following error. Reverting meltano.yml file to previous version doesn't help and gives the same error:
ELT could not be completed: Cannot start extractor: Executable 'tap-exactonline' could not be found. Extractor 'Client1' may not havebeen installed yet using `meltano install extractor Client1`, or the executable name may be incorrect.
'tap-exactonline' - main tap name. 'Client1' - inherited tap name. Does anyone know or have experienced this problem? Please can anyone help on this issue? How we can fix this?
Cc: @conno_kuytpat_nadolny
03/02/2022, 4:03 PMzafar
03/02/2022, 4:11 PMversion: 1
send_anonymous_usage_stats: false
project_id: 60140d5b-32f1-49b1-aa37-e7a48dd2ec43
plugins:
extractors:
- name: tap-exactonline
namespace: tap_exactonline
pip_url: <git+ssh://git@github.com/ebit-wise/tap-exactonline.git>
executable: tap-exactonline
capabilities:
- catalog
- discover
- state
settings:
- name: username
kind: string
- name: password
kind: password
- name: exactonline_url
kind: string
- name: redirect_url
kind: string
- name: client_id
kind: password
- name: client_secret
kind: password
- name: verification_secret_key
kind: password
- name: Client1
inherit_from: tap-exactonline
config:
username: <mailto:conno@ebitwise.nl|conno@ebitwise.nl>
password: $TAP_EXACTONLINE_PASSWORD_CLIENT1
exactonline_url: <http://start.exactonline.nl|start.exactonline.nl>
redirect_url: <https://www.ebitwise.nl/>
client_id: $TAP_EXACTONLINE_CLIENT_ID_CLIENT1
client_secret: $TAP_EXACTONLINE_CLIENT_SECRET_CLIENT1
verification_secret_key: $TAP_EXACTONLINE_VERIFICATION_SECRET_KEY_CLIENT1
- name: Client2
inherit_from: tap-exactonline
config:
username: <mailto:conno@ebitwise.nl|conno@ebitwise.nl>
password: $TAP_EXACTONLINE_USERNAME_CLIENT2
exactonline_url: <http://start.exactonline.nl|start.exactonline.nl>
redirect_url: <https://www.ebitwise.nl/>
client_id: $TAP_EXACTONLINE_CLIENT_ID_CLIENT2
client_secret: $TAP_EXACTONLINE_CLIENT_SECRET_CLIENT2
verification_secret_key: $TAP_EXACTONLINE_VERIFICATION_SECRET_KEY_CLIENT2
loaders:
- name: target-bigquery
variant: adswerve
pip_url: git+<https://github.com/adswerve/target-bigquery.git@0.11.3>
orchestrators:
- name: airflow
pip_url: psycopg2 apache-airflow==1.10.13 --constraint <https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.8.txt>
files:
- name: airflow
pip_url: git+<https://gitlab.com/meltano/files-airflow.git>
pat_nadolny
03/02/2022, 5:24 PMmeltano install extractors Client2 tap-exactonline --clean
to clear out and reinstallzafar
03/03/2022, 9:34 AMmeltano install --clean
command on my local machine and it removed old virtual environment and created new one for extractors/loaders and it worked for me. But when I added the same command in Dockerfile but it didn't help.
# Install all plugins into the `.meltano` directory
COPY ./meltano.yml .
RUN meltano install --clean
zafar
03/03/2022, 9:39 AMStep #0 - "build image": Installing 6 plugins...
Step #0 - "build image": Installing file bundle 'airflow'...
Step #0 - "build image": Installing orchestrator 'airflow'...
Step #0 - "build image": Installed file bundle 'airflow'
Step #0 - "build image": Run `meltano upgrade files` to update your project's 'airflow' files.
Step #0 - "build image": Installing loader 'target-bigquery'...
Step #0 - "build image": Installed loader 'target-bigquery'
Step #0 - "build image": Installing extractor 'Client2'...
Step #0 - "build image": Installed extractor 'Client2'
Step #0 - "build image": Installing extractor 'Client1'...
Step #0 - "build image": Installed extractor 'Client1'
Step #0 - "build image": Installing extractor 'tap-exactonline'...
Step #0 - "build image": Installed orchestrator 'airflow'
Step #0 - "build image": Installed extractor 'tap-exactonline'
Step #0 - "build image": Installed 6/6 plugins
pat_nadolny
03/03/2022, 2:46 PM--no-cache
or deleting the existing image off your local before building?zafar
03/07/2022, 6:15 PM