I am trying to test out Meltano 2 4 1 GUI s `dbt duckdb` but Meltano #troubleshooting

I am trying to test out Meltano 2.4.1 GUI's `dbt-d...

emcp

10/17/2022, 3:50 PM

I am trying to test out Meltano 2.4.1 GUI's

dbt-duckdb

.. but it seems confused I've successfully done

meltano add transform dbt-duckdb

.. checked the file area and removed the old legacy

dbt-postgresql

files that I found there.. BUT still seeing meltano looking for the original

dbt

.. any ideas?

emcp

10/17/2022, 3:51 PM

Copy code

2022-10-17T15:44:56.755088Z [error    ] Plugin 'Transformer 'dbt' not found.
Use of the legacy 'dbt' Transformer is deprecated in favor of new adapter specific implementations (e.g. 'dbt-snowflake') compatible with the 'meltano run ...' command.
<https://docs.meltano.com/guide/transformation>
To continue using the legacy 'dbt' Transformer, add it to your Project using 'meltano add transformer dbt'.' is not known to Meltano 
Traceback (most recent call last):
  File "/home/ubuntu/.local/pipx/venvs/meltano/lib/python3.10/site-packages/meltano/core/project_plugins_service.py", line 303, in get_plugin
    plugin = next(
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/.local/pipx/venvs/meltano/lib/python3.10/site-packages/meltano/core/elt_context.py", line 409, in plugin_context
    plugin = self.plugins_service.get_plugin(plugin_ref)
  File "/home/ubuntu/.local/pipx/venvs/meltano/lib/python3.10/site-packages/meltano/core/project_plugins_service.py", line 311, in get_plugin
    raise PluginNotFoundError(plugin_ref) from stop
meltano.core.plugin.error.PluginNotFoundError: Transformer 'dbt' is not known to Meltano

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/.local/pipx/venvs/meltano/lib/python3.10/site-packages/meltano/core/logging/output_logger.py", line 201, in redirect_logging
    yield
  File "/home/ubuntu/.local/pipx/venvs/meltano/lib/python3.10/site-packages/meltano/cli/elt.py", line 255, in _redirect_output
    yield
  File "/home/ubuntu/.local/pipx/venvs/meltano/lib/python3.10/site-packages/meltano/cli/elt.py", line 269, in _run_elt
    elt_context = context_builder.context()
  File "/home/ubuntu/.local/pipx/venvs/meltano/lib/python3.10/site-packages/meltano/core/elt_context.py", line 480, in context
    transformer = self.plugin_context(
  File "/home/ubuntu/.local/pipx/venvs/meltano/lib/python3.10/site-packages/meltano/core/elt_context.py", line 412, in plugin_context
    raise PluginNotFoundError(
meltano.core.plugin.error.PluginNotFoundError: Plugin 'Transformer 'dbt' not found.
Use of the legacy 'dbt' Transformer is deprecated in favor of new adapter specific implementations (e.g. 'dbt-snowflake') compatible with the 'meltano run ...' command.
<https://docs.meltano.com/guide/transformation>
To continue using the legacy 'dbt' Transformer, add it to your Project using 'meltano add transformer dbt'.' is not known to Meltano

emcp

10/17/2022, 3:52 PM

the

meltano.yaml

Copy code

transformers:
  - name: dbt-duckdb
    variant: jwills
    pip_url: dbt-core~=1.2.0 dbt-duckdb~=1.2.0

no other transformers exist in the yaml

emcp

10/17/2022, 3:52 PM

Copy code

schedules:
- name: ibkr-to-postgres-tickers
  interval: '@once'
  extractor: tap-ibkr-tickers
  loader: target-parquet
  transform: run

taylor

10/17/2022, 3:53 PM

The GUI only uses

meltano elt

which requires the older full

dbt

plugin. The GUI does not support

meltano run

which is what the adapter specific plugins are meant for.

emcp

10/17/2022, 3:53 PM

ah .. is it again that the default

transform/dbt_project.yml

is dropping in the "old" path (I seem to recall this last time I started migrating to v2.x

emcp

10/17/2022, 3:54 PM

ahhh ... is the GUI perhaps.. showing me something I shouldn't be shown post v2.0.0+ ?

emcp

10/17/2022, 3:55 PM

it supports only the legacy DBT then... okay

emcp

10/17/2022, 4:03 PM

I wish there was a way to keep the GUI and the CLI alongside one another.. or at least a way to put the CLI in the GUI itself

emcp

10/17/2022, 4:04 PM

I imagine that is a huge security challenge

taylor

10/17/2022, 4:05 PM

we’ll get there 😄

emcp

10/17/2022, 4:14 PM

i successfully (I think) configured in the CLI instead .. really really liking the

--interactive

emcp

10/17/2022, 4:14 PM

for the config

Open in Slack

Previous Next