Hi All, is there anyone have idea about this? my ...
# plugins-general
m
Hi All, is there anyone have idea about this? my tap is bing-ads and target is redshift
a
If you scroll up from there, do you get any more specific error message? The only thing I see of note from your log is “Extractor failed” with a “}” symbol.
m
nope just this message
@aaronsteers do you have any idea?
r
@mohsin I would set your log level to debug by doing this:
Copy code
meltano --log-level=debug elt ...
m
```meltano | INFO Incremental state has been updated at 2021-03-18 202345.698878. meltano | DEBUG Incremental state: {'bookmarks': {'accounts': {'last_record': '2021-02-23T212900.960000+00:00'}, '141559394_ad_performance_report': {'request_id': '1238132885391'}, '141559394_keyword_performance_report': {'request_id': '1238132885442'}}} meltano | DEBUG Deleted configuration at /home/ubuntu/dev-meltano/bing-to-postgres/meltano-bing-topostgres/.meltano/run/elt/2021-03-18T202303--tap-bing-ads--target-redshift/bff5d897-cf63-4ce1-ac38-68d9005a9e8b/target.config.json meltano | DEBUG Deleted configuration at /home/ubuntu/dev-meltano/bing-to-postgres/meltano-bing-topostgres/.meltano/run/elt/2021-03-18T202303--tap-bing-ads--target-redshift/bff5d897-cf63-4ce1-ac38-68d9005a9e8b/tap.config.json meltano | ERROR Extraction failed (1): } meltano | DEBUG ELT could not be completed: Extractor failed Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 236, in _run_elt await _run_extract_load(elt_context, output_logger) File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 274, in _run_extract_load await singer_runner.run( File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/core/runner/singer.py", line 253, in run await self.invoke( File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/core/runner/singer.py", line 227, in invoke raise RunnerError("Extractor failed", {PluginType.EXTRACTORS: tap_code}) meltano.core.runner.RunnerError: Extractor failed The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 224, in _redirect_output yield File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 245, in _run_elt raise CliError(f"ELT could not be completed: {err}") from err meltano.cli.utils.CliError: ELT could not be completed: Extractor failed meltano | ELT could not be completed: Extractor failed [2021-03-18 202345,815] [31668|MainThread|meltano.cli.utils] [DEBUG] ELT could not be completed: Extractor failed Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 236, in _run_elt await _run_extract_load(elt_context, output_logger) File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 274, in _run_extract_load await singer_runner.run( File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/core/runner/singer.py", line 253, in run await self.invoke( File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/core/runner/singer.py", line 227, in invoke raise RunnerError("Extractor failed", {PluginType.EXTRACTORS: tap_code}) meltano.core.runner.RunnerError: Extractor failed The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/__init__.py", line 43, in main cli(obj={"project": None}) File "/usr/lib/python3/dist-packages/click/core.py", line 764, in call return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/params.py", line 23, in decorate return func(*args, **kwargs) File "/home/ubuntu/.local/lib/py…
@ricky_renner here is my logs
r
Hmmm still not much there... looks like it's failing at the
bing-taps
but that's all I can gather. Have you gotten this working before?
a
Yeah, nothing from the logs pop out to me either - it looks like the tap is failing without much information on the root of the error. If you are able to execute the tap directly, that also might be worthwhile for debugging purposes. Do you mind opening an issue for us in the Meltano project, along with these full logs? Issues · meltano / Meltano · GitLab
If you familiar with the tap and able to execute it directly, outside of meltano, that would be another manner of testing with the hopes of receiving a clear error message.
And it’s still a week away, but we just started #C01QS0RV78D if you want to bring this to our weekly session for direct debugging assist.
m
i am familiar with this tap, i’ll create an issue
@edward_ryan FYI
d
Is there any additional error output from the tap (prefixed with
tap-bing-ads
) before these initial
meltano
lines?
m
nope
@douwe_maan here is my complete logs ```target-redshift (out) | {"bookmarks": {"accounts": {"last_record": "2021-02-23T212900.960000+00:00"}, "141559394_ad_extension_detail_report": {"request_id": null, "date": "2021-03-19T000000+00:00"}, "141559394_ad_performance_report": {"request_id": "1238134853920"}, "141559394_age_gender_audience_report": {"request_id": "1238134853947"}, "141559394_audience_performance_report": {"request_id": "1238134853965"}, "141559394_keyword_performance_report": {"request_id": "1238134853997"}}} meltano | INFO Incremental state has been updated at 2021-03-19 220602.556205. meltano | DEBUG Incremental state: {'bookmarks': {'accounts': {'last_record': '2021-02-23T212900.960000+00:00'}, '141559394_ad_extension_detail_report': {'request_id': None, 'date': '2021-03-19T000000+00:00'}, '141559394_ad_performance_report': {'request_id': '1238134853920'}, '141559394_age_gender_audience_report': {'request_id': '1238134853947'}, '141559394_audience_performance_report': {'request_id': '1238134853965'}, '141559394_keyword_performance_report': {'request_id': '1238134853997'}}} meltano | DEBUG Deleted configuration at /home/ubuntu/dev-meltano/bing-to-postgres/meltan-bing-to-redshift/bing-ads-to-redshift/.meltano/run/elt/2021-03-19T220443--tap-bing-ads--target-redshift/4aec8e37-5ffb-4dbf-80b0-1085cf559bee/target.config.json meltano | DEBUG Deleted configuration at /home/ubuntu/dev-meltano/bing-to-postgres/meltan-bing-to-redshift/bing-ads-to-redshift/.meltano/run/elt/2021-03-19T220443--tap-bing-ads--target-redshift/4aec8e37-5ffb-4dbf-80b0-1085cf559bee/tap.config.json meltano | ERROR Extraction failed (1): } meltano | DEBUG ELT could not be completed: Extractor failed Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 236, in _run_elt await _run_extract_load(elt_context, output_logger) File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 274, in _run_extract_load await singer_runner.run( File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/core/runner/singer.py", line 253, in run await self.invoke( File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/core/runner/singer.py", line 227, in invoke raise RunnerError("Extractor failed", {PluginType.EXTRACTORS: tap_code}) meltano.core.runner.RunnerError: Extractor failed The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 224, in _redirect_output yield File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 245, in _run_elt raise CliError(f"ELT could not be completed: {err}") from err meltano.cli.utils.CliError: ELT could not be completed: Extractor failed meltano | ELT could not be completed: Extractor failed [2021-03-19 220602,667] [38353|MainThread|meltano.cli.utils] [DEBUG] ELT could not be completed: Extractor failed Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 236, in _run_elt await _run_extract_load(elt_context, output_logger) File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cli/elt.py", line 274, in _run_extract_load await singer_runner.run( File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/core/runner/singer.py", line 253, in run await self.invoke( File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/core/runner/singer.py", line 227, in invoke raise RunnerError("Extractor failed", {PluginType.EXTRACTORS: tap_code}) meltano.core.runner.RunnerError: Extractor failed The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/meltano/cl…
@aaronsteers share the complete logs please have a look if is it helpful?
successfully able to run the tap, the issue with campaign_performance_report stream id, @aaronsteers please look into it.
d
@mohsin Does this mean you got it working? What was the issue with the
campaign_performance_report
stream?
m
i don’t know what is the issue actually is, but face error as i shared above in the logs..
d
I'm not seeing an error in those logs, it says "Extract & load complete!" at the bottom suggesting everything went right!
m
check the above screenshot
the first one and the logs
d
The
meltano               | ERROR Extraction failed (1): }
line in the failing output suggests that the last line output by the tap was
}
. In the full failing output for
meltano --log-level=debug elt ...
, do you see any lines prefixed with
tap-bing-ads
or
tap-bing-ads (out)
that may help us figure out what's going wrong there?