`meltano invoke dbt` Noob questions incoming Meltano #troubleshooting

`meltano invoke dbt` - Noob questions incoming!

visch

05/13/2021, 9:11 PM

meltano invoke dbt

- Noob questions incoming!

visch

05/13/2021, 9:12 PM

Copy code

(.venv) visch@visch-ubuntu:~/git/oracle2mssql/oracle2mssql$ meltano invoke dbt run
Running with dbt=0.16.1
Encountered an error while reading the project:
  ERROR: Runtime Error
  Could not find profile named 'mssql'
Encountered an error:
Runtime Error
  Could not run dbt

visch

05/13/2021, 9:12 PM

I have dbt running on my dbtvenv by just running dbt run. But Meltano has expectations. Started from https://meltano.slack.com/archives/C01TCRBBJD7/p1620939154167100?thread_ts=1620932204.163100&cid=C01TCRBBJD7

visch

05/13/2021, 9:13 PM

I haven't dove into Meltano's docs much as the page says it may be old and I'm still learning DBT so I thought that just doing native DBT would be better for now as invoke didn't want to work for me out of the box

douwe_maan

05/13/2021, 9:14 PM

@visch Can you run with

meltano --log-level=debug

so we can see the full dbt command that's executed?

visch

05/13/2021, 9:16 PM

Feature request idea from this, --log-level=debug --support or something to auto redact my env variables. I"ll subtmit something. Back on task

visch

05/13/2021, 9:16 PM

Copy code

[2021-05-13 17:14:58,203] [10992|MainThread|root] [DEBUG] Creating engine <meltano.core.project.Project object at 0x7fa355a1e700>@sqlite:////home/visch/git/oracle2mssql/oracle2mssql/.meltano/meltano.db
[2021-05-13 17:14:58,250] [10992|MainThread|urllib3.connectionpool] [DEBUG] Starting new HTTPS connection (1): <http://www.meltano.com:443|www.meltano.com:443>
[2021-05-13 17:14:58,467] [10992|MainThread|urllib3.connectionpool] [DEBUG] <https://www.meltano.com:443> "GET /discovery.yml?project_id=a79fe068-6d4d-4fcf-a94a-f05ba65df8c9 HTTP/1.1" 200 103053
[2021-05-13 17:14:58,897] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_LOAD__DIALECT' is missing from the environment.
[2021-05-13 17:14:58,897] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_LOAD__TARGET_SCHEMA' is missing from the environment.
[2021-05-13 17:14:58,897] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_TRANSFORM__PACKAGE_NAME' is missing from the environment.
[2021-05-13 17:14:58,897] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_EXTRACTOR_NAMESPACE' is missing from the environment.
[2021-05-13 17:14:58,899] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_LOAD__DIALECT' is missing from the environment.
[2021-05-13 17:14:58,899] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_LOAD__TARGET_SCHEMA' is missing from the environment.
[2021-05-13 17:14:58,899] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_TRANSFORM__PACKAGE_NAME' is missing from the environment.
[2021-05-13 17:14:58,899] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_EXTRACTOR_NAMESPACE' is missing from the environment.
[2021-05-13 17:14:58,900] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_LOAD__DIALECT' is missing from the environment.
[2021-05-13 17:14:58,901] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_LOAD__TARGET_SCHEMA' is missing from the environment.
[2021-05-13 17:14:58,901] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_TRANSFORM__PACKAGE_NAME' is missing from the environment.
[2021-05-13 17:14:58,901] [10992|MainThread|meltano.core.utils] [DEBUG] Variable '$MELTANO_EXTRACTOR_NAMESPACE' is missing from the environment.
[2021-05-13 17:14:58,901] [10992|MainThread|root] [DEBUG] Invoking: ['/home/visch/git/oracle2mssql/oracle2mssql/.meltano/transformers/dbt/venv/bin/dbt', 'run']
[2021-05-13 17:14:58,902] [10992|MainThread|root] [DEBUG] Redacted
Running with dbt=0.16.1
Encountered an error while reading the project:
  ERROR: Runtime Error
  Could not find profile named 'mssql'
Encountered an error:
Runtime Error
  Could not run dbt
[2021-05-13 17:14:59,375] [10992|MainThread|urllib3.connectionpool] [DEBUG] Starting new HTTPS connection (1): <http://www.google-analytics.com:443|www.google-analytics.com:443>
[2021-05-13 17:14:59,516] [10992|MainThread|urllib3.connectionpool] [DEBUG] <https://www.google-analytics.com:443> "POST /collect HTTP/1.1" 200 35

douwe_maan

05/13/2021, 9:16 PM

I like it, issue pls? 😄

douwe_maan

05/13/2021, 9:17 PM

Ah, that env was actually relevant 😄 Can you share the

DBT_

prefixed vars?

visch

05/14/2021, 2:00 AM

'MELTANO_TRANSFORMER_NAME': 'dbt', 'MELTANO_TRANSFORMER_NAMESPACE': 'dbt', 'MELTANO_TRANSFORMER_VARIANT': 'original', 'DBT_PROJECT_DIR': '/home/visch/git/oracle2mssql/oracle2mssql/transform', 'MELTANO_TRANSFORM_PROJECT_DIR': '/home/visch/git/oracle2mssql/oracle2mssql/transform', 'DBT_PROFILES_DIR': '/home/visch/git/oracle2mssql/oracle2mssql/transform/profile', 'MELTANO_TRANSFORM_PROFILES_DIR': '/home/visch/git/oracle2mssql/oracle2mssql/transform/profile', 'DBT_TARGET_SCHEMA': 'analytics', 'MELTANO_TRANSFORM_TARGET_SCHEMA': 'analytics', 'DBT_MODELS': ' my_meltano_project', 'MELTANO_TRANSFORM_MODELS': ' my_meltano_project'

douwe_maan

05/14/2021, 1:33 PM

Odd, I'm not seeing

mssql

anywhere in those values, so I'm not sure where the

Could not find profile named 'mssql'

error is coming from. Do you have that string anywhere in your dbt project, in

dbt_project.yml

for example?

Open in Slack

Previous Next