I am still not able to configure tap-github in a w...
# troubleshooting
j
I am still not able to configure tap-github in a way that would fully work properly. Recently, pull_requests.user is loaded with empty JSON. It means that I cannot JOIN pull_requests with users, so many analytics use cases do not work. Anyone struggles with this issue? Any clue how to workaround/fix it?
d
Hey, I've been using tap-github a bunch recently, with a bunch of different targets, but never seen the null users issue. Just wondering if your problem is with a specific target?
j
Both target-postgres and target-snowflake are affected. meltano.yml:
Copy code
extractors:
    - name: tap-github
      variant: meltanolabs
      pip_url: git+<https://github.com/MeltanoLabs/tap-github.git>
    - name: tap-github-repo
      inherit_from: tap-github
      config:
        repositories:
          - gooddata/gooddata-python-sdk
        start_date: '2022-01-01'
      select:
        ....
        - pull_requests.user
Will try to debug it with target-jsonl, but AFAIK even the JSON contains empty dict for pull_requests.user.
d
I've run tap-github against the default target-postgres and target-snowflake today and my sync worked fine. Syncing from a public repo with the
public_repo
scope in my auth_token, but I do have push permissions to the repo. (I saw some issue about not being able to get certain streams unless you have push permissions, but these caused 403s). I don't inherit, and do not have any selects in my meltano config though. Can quickly try the select on my end and see if it makes a difference, will let ya know
So I ran it with 3 different selects against transferwise
target-postgres
, the first:
Copy code
select:
  - pull_requests.*
This synced all data, including users. Then I ran it with:
Copy code
select:
  - pull_requests.user
And the user column (only thing synced) was empty json. Then finally I tried:
Copy code
select: 
  - pull_requests.user*
This synced the user column with data. Not sure if this is the intended behaviour, but hope this works for you.
j
Works for me, now everything is possible, huge thanks! 🙂
a
By the way. @daniel_walker is working towards a dbt hub package for GitHub. We would love to include your models in it! This is to be part of an example project as a “medium complexity meltano example”.