tap-github - I decided to crawl repositories, whic...
# plugins-general
j
tap-github - I decided to crawl repositories, which are not under an organization but under personal account. I tried:
Copy code
- name: tap-github-user
    inherit_from: tap-github
    config:
      user_usernames:
        - jaceksan
    select:
      # Repositories
      - repositories.id
      - repositories.html_url
      - repositories.name
      - repositories.stargazers_count
      - repositories.watchers_count
      - repositories.created_at
      - repositories.updated_at
    stream_maps:
      __alias__: user_repositories
But it does crawl anything. Is it possible to crawl <https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#list-repositories-for-a-user%7C/&lt;username&gt;/repos API> with tap-github?
e
Yeah, that seems to be possible if you use the
organizations
setting: https://github.com/MeltanoLabs/tap-github/blob/b7a331dc9ac916ef423ade0fb3c3e5203ee3c1bd/tap_github/repository_streams.py#L46-L55 that'd be
organizations: [jaceksan]
in your case I think.
j
It does not work, it does not find
jaceksan
as organization, because it is not organization but personal account. GitHub describes the difference in their public DOC. That is why you have to list repositories of users(personal accounts) from different API(mentioned above) than repositories in organizations.
But this stream does not contain an implementation for UserRepositories
So, would the solution be to extend this user stream by repositories? I can implement it...
e
Ah you're right,
/users/{username}/repos
is what you need. Do send us a PR if you can 😁!