Hello guys, can please someone help me with tap-pendo error? It seems like it is because of python version, but I tried to run it in venv Conda with python 3.7.0, still gets same error.

2023-06-19T12:43:47.218881Z [error    ] Cannot start plugin tap-pendo: Catalog discovery failed: command ['/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/bin/tap-pendo', '--config', '/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/run/tap-pendo/tap.3b2813ad-7ecf-4548-b5eb-8636bce99254.config.json', '--discover'] returned 1 with stderr:
   File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/bin/tap-pendo", line 5
    meltanofrom tap_pendo import main
                ^^^^^^^^^
SyntaxError: invalid syntax

2023-06-19T12:43:47.219413Z [error    ] Block run completed.           block_type=ExtractLoadBlocks err=RunnerError('Cannot start plugin tap-pendo: Catalog discovery failed: command [\'/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/bin/tap-pendo\', \'--config\', \'/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/run/tap-pendo/tap.3b2813ad-7ecf-4548-b5eb-8636bce99254.config.json\', \'--discover\'] returned 1 with stderr:\n   File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/bin/tap-pendo", line 5\n    meltanofrom tap_pendo import main\n                ^^^^^^^^^\nSyntaxError: invalid syntax\n') exit_codes={} set_number=0 success=False
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Cannot start plugin tap-pendo: Catalog discovery failed: command ['/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/bin/tap-pendo', '--config', '/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/run/tap-pendo/tap.3b2813ad-7ecf-4548-b5eb-8636bce99254.config.json', '--discover'] returned 1 with stderr:
   File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/bin/tap-pendo", line 5
    meltanofrom tap_pendo import main
                ^^^^^^^^^
SyntaxError: invalid syntax

@alexander_andreyanov from your stack trace and screenshot it looks like maybe you might have edited the installed python module by accident and caused a syntax error. I would try running

meltano install extractor tap-pendo --clean

and run again. If it errors again you can try running with

meltano invoke

to isolate the tap from the target and get better error output

@pat_nadolny, thank you, just done

meltano install extractor tap-pendo --clean

- same error.

meltano invoke tap-pendo:

2023-06-19T15:56:15.237839Z [info     ] Environment 'dev' is active
Traceback (most recent call last):
  File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/bin/tap-pendo", line 5, in <module>
    from tap_pendo import main
  File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/lib/python3.10/site-packages/tap_pen
do/__init__.py", line 10, in <module>                                                                                                    from tap_pendo.discover import discover_streams
  File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/lib/python3.10/site-packages/tap_pen
do/discover.py", line 6, in <module>                                                                                                     from tap_pendo.streams import STREAMS
  File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/lib/python3.10/site-packages/tap_pen
do/streams.py", line 12, in <module>                                                                                                     import humps
  File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/lib/python3.10/site-packages/humps/_
_init__.py", line 13, in <module>                                                                                                        from humps.main import camelize
  File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/extractors/tap-pendo/venv/lib/python3.10/site-packages/humps/m
ain.py", line 7, in <module>                                                                                                             from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/local/python/3.10.4/lib/python3.10/collections/__init__.py)

@alexander_andreyanov ah ok thats a different error, it looks like youre using python 3.10 and according to the tap badges (which could be totally outdated) it only support 3.7 🤔 https://github.com/singer-io/tap-pendo. The pyhumps package they use is not on a version that support 3.10. I was able to replicate your error in my local test project and using a fork with the pyhumps package updated I get past your error and onto

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url

which seems like a good sign but I dont have real credentials to tests. To test this out with your config you can update your pip url in your meltano.yml to my fork and run

meltano install extractor tap-pendo --clean

to get a fresh install

pip_url: git+<https://github.com/pnadolny13/tap-pendo.git>

@pat_nadolny hm, I see now, but I don’t understand why it trying to use python 3.10, if I ran it in the Conda venv with Python 3.7.0? I’m full newbie with github at all, so I may not know smth, but in my understanding if I’m using terminal from Conda venv 3.7.0 - meltano should use it instead of default one, no?

I would think that it should use your terminal version but I'm honestly not very familiar with Conda. You can try running python --version to confirm your version is 3.7.0 then run

meltano install extractor tap-pendo --clean

again

@pat_nadolny great, your fork

pip_url: git+<https://github.com/pnadolny13/tap-pendo.git>

is working! thank you! Question with python version is still opened, but ok. Unfortunately when I’m trying to run tap-pendo target-snowflake - extracting data from pendo - no issues, but when it comes to loading data to snowflake - it returns me an errors:

2023-06-19T20:47:09.862714Z [info     ] INFO START Syncing: pages      cmd_type=elb consumer=False name=tap-pendo producer=True stdio=stderr string_id=tap-pendo
2023-06-19T20:47:09.864016Z [info     ] INFO Stream pages: Starting sync cmd_type=elb consumer=False name=tap-pendo producer=True stdio=stderr string_id=tap-pendo
2023-06-19T20:47:11.633521Z [info     ] INFO POST <https://app.pendo.io/api/v1/aggregation> None cmd_type=elb consumer=False name=tap-pendo producer=True stdio=stderr string_id=tap-pendo
2023-06-19T20:47:11.873550Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.874082Z [info     ]   File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/loaders/target-snowflake/venv/bin/target-snowflake", line 8, in <module> cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.874505Z [info     ]     sys.exit(main())           cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.874857Z [info     ]   File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/loaders/target-snowflake/venv/lib/python3.10/site-packages/target_snowflake/__init__.py", line 526, in main cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.875179Z [info     ]     persist_lines(config, singer_messages, table_cache, file_format_type) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.875483Z [info     ]   File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/loaders/target-snowflake/venv/lib/python3.10/site-packages/target_snowflake/__init__.py", line 143, in persist_lines cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.875790Z [info     ]     stream_utils.adjust_timestamps_in_record(o['record'], schemas[stream]) cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.876089Z [info     ]   File "/workspaces/coginiti_meltano_cloud_test/coginiti_test/.meltano/loaders/target-snowflake/venv/lib/python3.10/site-packages/target_snowflake/stream_utils.py", line 69, in adjust_timestamps_in_record cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.876854Z [info     ]     if 'string' in schema['properties'][key]['type'] and \ cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:11.877350Z [info     ] KeyError: 'type'               cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-19T20:47:12.217597Z [error    ] Loader failed
2023-06-19T20:47:12.218297Z [error    ] Block run completed.           block_type=ExtractLoadBlocks err=RunnerError('Loader failed') exit_codes={<PluginType.LOADERS: 'loaders'>: 1} set_number=0 success=False
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Loader failed

If I’m looking into mentioned in the output files - I see problems in 3 files: target-snowflake / init.py / stream_utils.py. All in pics attached.

I dont see anything in the screenshots to be worried about. You can set

default_target_schema

in your config if you want but it gets auto populated by the tap name when run with

elt

or

run

(not with

invoke

). I suspect that the data coming from the tap has issues. The target is looking for the type and it looks like it doesnt exist. You can try running

meltano invoke tap-pendo > output.json

to inspect the data that the tap is outputting then pipe it back into the target like

cat output.json | meltano invoke target-snowflake

. Additionally you can try a simpler target like target-jsonl which will also validate the data from the tap is good in order to isolate the tap from the target

@pat_nadolny thank you! will try!