https://linen.dev logo
Join Slack
Powered by
Hello there, I've been trying to set the configura...
# troubleshooting
e
Hello there, I've been trying to set the configuration of the extractor 
tap-facebook-pages
(variant voxmedia) with no success. I followed the variant and meltano's doc. What I do (I'm using docker): 
meltano config tap-facebook-pages set pages [{'id': 'abc', 'access_token': 'def'}]
Copy code
/venv/lib/python3.9/site-packages/meltano/core/settings_service.py:471: RuntimeWarning: Unknown setting "pages.[{'id':.'abc',.'access_token':"
  warnings.warn(f"Unknown setting {name!r}", RuntimeWarning)
Extractor 'tap-facebook-pages' setting 'pages.[{'id':.'abc',.'access_token':' was set in `meltano.yml`: "'def'}]"
When I tun the test 
meltano config tap-facebook-pages test
I get:
Copy code
[...]
, line 158, in discover_streams
    stream_class.page_access_tokens = {
  File "/projects/.meltano/extractors/tap-facebook-pages/venv/lib/python3.9/site-packages/tap_facebook_pages/tap.py", line 159, in <dictcomp>
    page["id"]: page["access_token"] for page in self.config["pages"]
TypeError: string indices must be integers
Clearly, I'm doing my config very badly. Someone could please help me with the right config ? Do you know an example in the web using this extractor ? Thanks, any help is much appreciated.
v
Can you share your 
meltano.yml
setting objects / arrays is tough because of your shell needing characters to be escaped. If you look at 
meltano.yml
directly it will probably be easy to see what happened
e
Sure:
Copy code
version: 1
default_environment: dev
project_id: 41fcbec6-d8a1-4c7c-83ae-5aaba7cd660f
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-facebook-pages
    variant: voxmedia
    pip_url: git+<https://github.com/voxmedia/tap-facebook-pages.git>
    config:
      startdate: 01/01/2020
      pages:
        "[{'id':":
          "'abc',":
            "'access_token':": "'def'}]"
oh I removed something I put by hand, what the command sets is there
v
Copy code
"[{'id':":
          "'abc',":
            "'access_token':": "'def'}]"
Is the issue for sure You want it to be somethiung like
Copy code
config:
  pages:
  - id: abc
    acces_token: "def"
Then to test try 
meltano invoke --dump=config tap-facebook-pages
You want it to be a json array so it'd looking something like
Copy code
"pages": [{"id":"abc", "access_token": "def"}]
Array/Objects are tougher unfortunately! You can make it work via the cli but you have to have the right escape characters
e
Oh, ok I see, I got this: (it's clearly wrong)
Copy code
{
  "user_token": "EAARmCHrVUxEBALjvCzPuAcXhbLbEB91IjaqeYcFVgZAhrb0vhYl6u1HE6SS0QnZCyLeW0XI2jJqbBDwuIR9nc6P0ByUOaWtlyMcOZCrBo1lZCCyIvFZCej0AaN4OFdbr0X0M1UvdZAPVVBl2ZBAetsgU0W8U8sRjQKYZBjYuftRC0uoOPdzivIPBvD22f0ZBQZCrrQGW6lQwiIrrN4ddrCLdW9",
  "startdate": "01/01/2020",
  "pages": {
    "[{'id':": {
      "'abc',": {
        "'access_token':": "'def'}]"
      }
    }
  }
}
v
Now you know how to debug it so work with the yaml to get the output you're after
e
Yes, it was quite helpful to know what the yaml is expecting in there. Thanks a lot !
v
Yeah if you have ideas how to make it easier for users that would be awesome!
e
Hello @visch still there? when running 
config tap-facebook-pages test
Meltano is asking for GOOGLE_APPLICATION_CREDENTIALS:
Copy code
Plugin configuration is invalid
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see <https://cloud.google.com/docs/authentication/getting-started>
So I got an API key, I put it in the .env and got:
Copy code
Plugin configuration is invalid
google.auth.exceptions.DefaultCredentialsError: File AIzaSyCdOacK5PwyDBxczt74wV84VOCKh6AQkKw was not found.
I understand that meltano is expecting a json, but what must be inside? any help much appreciated
Open in Slack
PreviousNext
Powered by