getting error connecting to snowflake . meltano is...
# plugins-general
a
getting error connecting to snowflake . meltano is looking fro azureblobs but our snowflake host is on aws
# meltano elt tap-google-analytics target-snowflake --transform run [2020-05-19 184149,414|I|MainThread|alembic.runtime.migration] Context impl SQLiteImpl. [2020-05-19 184149,416|I|MainThread|alembic.runtime.migration] Will assume non-transactional DDL. System database up-to-date. Running extract & load... No state was found, complete import. Traceback (most recent call last): File "/project/.meltano/loaders/target-snowflake/venv/bin/target-snowflake", line 11, in <module> load_entry_point('target-snowflake==0.1.0', 'console_scripts', 'target-snowflake')() File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 487, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2728, in load_entry_point return ep.load() File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2346, in load return self.resolve() File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2352, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/target_snowflake/__init__.py", line 10, in <module> from snowflake.connector.errors import ProgrammingError File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/snowflake/connector/__init__.py", line 21, in <module> from .connection import SnowflakeConnection File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/snowflake/connector/connection.py", line 40, in <module> from .cursor import SnowflakeCursor, LOG_MAX_QUERY_LENGTH File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 29, in <module> from .file_transfer_agent import (SnowflakeFileTransferAgent) File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/snowflake/connector/file_transfer_agent.py", line 20, in <module> from .azure_util import SnowflakeAzureUtil File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.6/site-packages/snowflake/connector/azure_util.py", line 9, in <module> from azure.storage.blob import BlockBlobService ImportError: cannot import name 'BlockBlobService' INFO URL being requested: GET https://www.googleapis.com/discovery/v1/apis/analyticsreporting/v4/rest INFO URL being requested: GET https://www.googleapis.com/discovery/v1/apis/analytics/v3/rest INFO URL being requested: GET https://www.googleapis.com/analytics/v3/metadata/ga/columns?alt=json INFO Attempting refresh to obtain initial access_token INFO Refreshing access_token INFO Syncing stream: website_overview INFO URL being requested: POST https://analyticsreporting.googleapis.com/v4/reports:batchGet?alt=json CRITICAL [Errno 32] Broken pipe Traceback (most recent call last): File "/project/.meltano/extractors/tap-google-analytics/venv/bin/tap-google-analytics", line 11, in <module> load_entry_point('tap-google-analytics==0.1.1', 'console_scripts', 'tap-google-analytics')() File "/project/.meltano/extractors/tap-google-analytics/venv/lib/python3.6/site-packages/singer/utils.py", line 229, in wrapped return fnc(*args, **kwargs) File "/project/.meltano/extractors/tap-google-analytics/venv/lib/python3.6/site-packages/tap_google_analytics/__init__.py", line 205, in main sync(args.config, args.state, catalog) File "/project/.meltano/extractors/tap-google-analytics/venv/lib/python3.6/site-packages/tap_google_analytics/__init__.py", line 84, in sync singer.write_schema(stream_id, stream_schema, key_properties) File "/project/.meltano/extractors/tap-google-anal…
d
@atul_kakade
target-snowflake
depends on
snowflake-connector-python
, which depends on
azure-storage-blob
, even if you don't actually intend to use it with Azure. The error indicates that an incorrect version of
azure-storage-blob
may have been installed, or that
target-snowflake
is looking for the
azure
module in Meltano's main venv rather than its own dedicated venv. If the latter is the case, you're looking at this bug, which was fixed in Meltano v1.32.1, released last Friday. Are you on that version? If you already are, did you
meltano init
and
meltano add target-snowflake
after installing that version, or before? If before, try running
meltano install
in your project to fix the
target-snowflake
installation, and then try
meltano elt
again.
If that doesn't work, let me know, and we can continue investigating why an incorrect version of
azure-storage-blob
may be getting installed.
a
I am running the most recent docker image which I pulled today morning .it is version 1.32.1 . after doing an init and install for target-snowflake . I have tried elt and it failed as well. I did “invoke” statement to test it and it failed as well
d
@atul_kakade Thanks, that helps! It looks like we're running into this issue: https://github.com/snowflakedb/snowflake-connector-python/issues/225 I've fixed it in https://gitlab.com/meltano/target-snowflake/-/merge_requests/29. Can you run
meltano install
in your project to install the latest version of
target-snowflake
, and try again?
a
thanks the blob error went away .
d
@atul_kakade Great! Let me know if you run into any other issues 🙂