Soumaya Mauthoor
07/30/2024, 12:52 PM2024-07-30T12:48:52.245405Z [info ] raise RuntimeError( cmd_type=elb consumer=False job_name=dev:tap-github-to-target-jsonl name=tap-github producer=True run_id=90985761-8ce2-458f-9c8e-a88752e7bd16 stdio=stderr string_id=tap-github
2024-07-30T12:48:52.245457Z [info ] RuntimeError: All GitHub tokens have hit their rate limit. Stopping here. cmd_type=elb consumer=False job_name=dev:tap-github-to-target-jsonl name=tap-github producer=True run_id=90985761-8ce2-458f-9c8e-a88752e7bd16 stdio=stderr string_id=tap-github
2024-07-30T12:48:52.322575Z [info ] Incremental state has been updated at 2024-07-30 12:48:52.322564.
2024-07-30T12:48:52.329718Z [error ] Extractor failed
2024-07-30T12:48:52.329919Z [error ] Block run completed. block_type=ExtractLoadBlocks err=RunnerError('Extractor failed') exit_codes={<PluginType.EXTRACTORS: 'extractors'>: 1} set_number=0 success=False
I’ve checked my GitHub token and I can print commits without any problem using PyGitHub:
from github import Github
# Authentication is defined via github.Auth
from github import Auth
from dotenv import load_dotenv
import os
load_dotenv()
GH_TOKEN = os.environ['TAP_GITHUB_AUTH_TOKEN']
print(GH_TOKEN)
# using an access token
auth = Auth.Token(GH_TOKEN)
# First create a Github instance:
# Public Web Github
g = Github(auth=auth)
g.get_user().login
# %%
repo = g.get_repo("SoumayaMauthoorMOJ/test")
commit = repo.get_commit(sha="b6a5cb8bc0d3be339de8c6f5f36183aa7dc055c5")
print(commit.commit.author.date)
Any idea what I’m doing wrong?Soumaya Mauthoor
07/30/2024, 12:54 PMversion: 1
default_environment: dev
project_id: 9deeee6d-844d-449e-b36c-2be9c3f9fa64
environments:
- name: dev
- name: staging
- name: prod
plugins:
extractors:
- name: tap-github
variant: meltanolabs
pip_url: git+<https://github.com/MeltanoLabs/tap-github.git>
config:
start_date: '2024-05-01'
repositories:
- SoumayaMauthoorMOJ/test
select:
- commits.url
- commits.sha
- commits.commit_timestamp
loaders:
- name: target-jsonl
variant: andyh1203
pip_url: target-jsonl
Andy Carter
07/30/2024, 12:58 PMAndy Carter
07/30/2024, 1:00 PMSoumaya Mauthoor
07/30/2024, 1:29 PMmeltano config tap-github test
gives me this error:
Plugin configuration is invalid
RuntimeError: All GitHub tokens have hit their rate limit. Stopping here.
Soumaya Mauthoor
07/30/2024, 1:34 PM.env
file was in the root of my repo, not in the meltano project