https://linen.dev logo
Join Slack
Powered by
[tap-airbyte] Guys, I'm having a problem using tap...
# troubleshooting
c
[tap-airbyte] Guys, I'm having a problem using tap-airbyte-wrapper, just to test it out for now, I've already tried using tap-apify and tap-github, but I'm getting the same error, below is the error and the settings for tap-github Repository: https://github.com/meltanolabs/tap-airbyte-wrapper Example config.json 
{
"airbyte_spec": {
"image": "airbyte/source-github:latest"
},
"airbyte_config": {
"credentials": {
"access_token": "gagasgsagasgaasgagas"
},
"repositories": "z3z1ma/*"
}
}
Command: 
meltano invoke tap-airbyte --config  config-github.json
error: 
Traceback (most recent call last):
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/bin/tap-airbyte", line 10, in <module>
sys.exit(TapAirbyte.cli())
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/tap_airbyte/tap.py", line 269, in cli
tap: TapAirbyte = cls(  # type: ignore
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/tap_airbyte/tap.py", line 314, in __init__
super().__init__(*args, **kwargs)
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/singer_sdk/tap_base.py", line 97, in __init__
self.mapper.register_raw_streams_from_catalog(self.catalog)
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/singer_sdk/tap_base.py", line 159, in catalog
self._catalog = self.input_catalog or self._singer_catalog
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/singer_sdk/tap_base.py", line 251, in _singer_catalog
for stream in self.streams.values()
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/singer_sdk/tap_base.py", line 122, in streams
for stream in self.load_streams():
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-airbyte/venv/lib/python3.10/site-packages/singer_sdk/tap_base.py", line 283, in load_streams
for stream in self.discover_streams():
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/tap_airbyte/tap.py", line 688, in discover_streams
for stream in self.airbyte_catalog["streams"]:
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/tap_airbyte/tap.py", line 581, in airbyte_catalog
raise AirbyteException(
tap_airbyte.tap.AirbyteException: Discover failed with return code 125: docker: invalid reference format.
See 'docker run --help'.
Can you help me with this problem?
p
Hey @cicero_moura! Have you tried 
meltano add extractor tap-github --variant airbyte
? We've done the work to preconfigure the wrapper and expose the appropriate settings for each airbyte source. You shouldnt need to call tap-airbyte-wrapper directly unless you're editing the wrapper logic (which should be relatively rare)
Similarly if you use 
meltano config tap-github set airbyte_config credentials.access_token xyz
to set your configs then run 
meltano invoke tap-github
without passing your 
--config
directly because meltano can handle it for you.
c
Great! Thank you very much for your support @pat_nadolny, I installed tap-github as instructed and configured it and it worked!
but I have a problem with the tap that I really want to use, which is apify I did the installation 
meltano add extractor tap-apify --variant airbyte
and the configuration of the datasetId 
meltano config tap-apify set airbyte_config.datasetId zzzzz
But I am having the following error: 
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-apify/venv/lib/python3.10/site-packages/singer_sdk/streams/core.py", line 1088, in sync
for _ in self._sync_records(context=context):
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-apify/venv/lib/python3.10/site-packages/singer_sdk/streams/core.py", line 1020, in _sync_records
self._write_record_message(record)
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-apify/venv/lib/python3.10/site-packages/tap_airbyte/tap.py", line 723, in _write_record_message
for record_message in self._generate_record_messages(record):
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-apify/venv/lib/python3.10/site-packages/singer_sdk/streams/core.py", line 789, in _generate_record_messages
record = conform_record_data_types(
File "/home/dadosfera/Documentos/Dadosfera-repo/singer-connector/tap-airbyte-wrapper/.meltano/extractors/tap-apify/venv/lib/python3.10/site-packages/singer_sdk/helpers/_typing.py", line 283, in conform_record_data_types
if property_name not in schema["properties"]:
KeyError: 'properties'
{"type":"STATE","value":{}}
2023-03-10 09:24:57,940 Synced 1 streams in 3.01 seconds.
I think it's because of the data schema, is there a way to set this schema if that's the case?
p
hmm is that the output when you run 
meltano invoke tap-apify
?
2 Views
Open in Slack
PreviousNext
Powered by