https://linen.dev logo
Join Slack
Powered by
Hi Team, I am new to meltano and having an issue w...
# troubleshooting
s
Hi Team, I am new to meltano and having an issue while executing “meltano elt tap-twitter target-jsonl --catalog .meltano/extractors/tap-twitter/venv/catalog.json” Can anyone please help me with the error. Appreciate your help.
a
Hi @srinivasula_reddy_yennapusa. To remove the warning, you would need to add 
catalog
in the 
capabilities
section under your extractor configuration in 
meltano.yml
.
Copy code
plugins:
  extractors:
  - name: tap-twitter
    capabilities:
    - catalog
    - discover
    - state
Can you please try updating your configuration and see if that helps? Thanks!
s
Thanks for your reply. I have already added these capabilities. Along with this warning, I am getting “INFO No properties were selected” because of which data load is not happening.
Hi @atharh, can you please guide me in resolving this issue.
a
@srinivasula_reddy_yennapusa You can refer to the Select entities and attributes to extract section in the Getting Started guide to see how you can list the entities supported by 
tap-twitter
and select them for extraction.
s
@atharh, even after the selection I am getting the same issue. [warning ] A catalog file was found, but it will be ignored as the extractor does not advertise the 
catalog
or 
properties
capability. “INFO No properties were selected”
@atharh, after correctly adding capabilities (as you mentioned) the warning has gone but the error “INFO No properties were selected” is still there when I run “meltano elt tap-twitter target-jsonl”.
p
@srinivasula_reddy_yennapusa if you run 
meltano select tap-twitter --list
do you see selected attributes?
s
@pat_nadolny, I guess no
p
@srinivasula_reddy_yennapusa try following the getting started guide link that Athar sent earlier (https://docs.meltano.com/getting-started#select-entities-and-attributes-to-extract) for making selections now that your have your catalog/properties fixed
a
Can you please share the tap configuration from your 
meltano.yml
file?
s
@pat_nadolny, I have selected “blocks” entity , and this is showing only under enabled patters but not under selected attributes.
should I make any changes to catalog.json aswell?
p
oh wait thats weird, it should have listed attributes under the 
Selected attributes
in your first screenshot what tap variant are you using? can you share you meltano.yml (cc @atharh you were way ahead of me 😝)?
s
@pat_nadolny , sharing the .yml file after removing credentials
meltano.yml
Not sure of the tap variant , I have installed via :
Copy code
meltano add --custom extractor tap-twitter
p
@srinivasula_reddy_yennapusa thanks for sharing, are you using custom catalogs and state files? Meltano usually handles the creation of those based on the settings in your meltano.yml and state in the system database. I think right now youre overriding the default behavior by setting 
catalog: .meltano/extractors/tap-twitter/venv/catalog.json
so its not auto discovering the available streams
s
@pat_nadolny, After removing catalog and state from meltano.yml file as you mentioned selected attributes got populated. Thanks. but when I run “meltano elt tap-twitter target-jsonl” I am getting the same earlier issue.
p
ok progress! What does 
meltano select tap-twitter --list
tell you now?
s
Selected attributes are getting populated
p
Hmm thats weird - @atharh @edgar_ramirez_mondragon any ideas?
@srinivasula_reddy_yennapusa can you try removing 
properties
from your capabilities list in meltano.yml?
s
Same error “INFO No properties were selected” even after removing properties from meltano.yml
a
I am wondering if the 
INFO
log is even an issue. @srinivasula_reddy_yennapusa The selected attributes are getting populated, right? Is there still some missing data?
s
yes, all the Selected attributes are getting populated. But concern is when I run “meltano elt tap-twitter target-jsonl” I am getting this issue and load is not happening to output folder.
e
@srinivasula_reddy_yennapusa can you remove the 
catalog
capability from the tap and retry?
(it may be preventing meltano from calling the tap with 
--properties
)
s
hi @edgar_ramirez_mondragon, thanks , I have removed catalog and added properties to the capabilities. And this issue is gone “INFO No properties were selected”. But the data load is not happening, when I run “meltano elt tap-twitter target-jsonl” to the output folder. Please find below the updated .yml file.
meltano_v2.yml
message has been deleted
e
it looks like you also removed your 
select
value? Try selecting everything with 
meltano select tap-twitter "*" "*"
s
Even after adding meltano select tap-twitter “*” “*” , data load is not happening, when I run “meltano elt tap-twitter target-jsonl” to the output folder.
e
what's the output of 
meltano select tap-twitter --list --all
?
s
we are getting some automatic Selected attributes.
Hi @edgar_ramirez_mondragon, Now if I execute “meltano select tap-twitter --list --all”, we are getting all the Selected attributes as automatic.
And if I run “meltano config tap-twitter test” , I am getting “Plugin configuration is invalid No RECORD message received”
message has been deleted
can you please help me with this.
Hi @edgar_ramirez_mondragon and @pat_nadolny, can you please help me on this issue.
e
hi @srinivasula_reddy_yennapusa. what's the output of 
meltano invoke tap-twitter
?
s
Hi @edgar_ramirez_mondragon, attached is the output for meltano invoke tap-twitter
Also If I run meltano config tap-twitter test, I am getting “Plugin configuration is invalid No RECORD message received” as mentioned above.
message has been deleted
Also sharing the meltano.yml file for your reference (removed the credentials)
meltano.yml
e
@srinivasula_reddy_yennapusa 
meltano.yml
looks good 🤔. Can you run 
meltano --log-level=debug config test
s
Hi @edgar_ramirez_mondragon, Please find attached error log for the above command. FYI : ‘MELTANO_EXTRACTOR_VARIANT’: ‘original’
errorlog.txt
Hi @edgar_ramirez_mondragon, can you please check the above log and let me know in case there is any issue.
Open in Slack
PreviousNext
Powered by