I am trying to run the singer-io `tap-github` and ...
# troubleshooting
s
I am trying to run the singer-io
tap-github
and struggling to submit an authorized request. I can access private data via Postman and curl with
Copy code
curl --request GET '<https://api.github.com/repos/immuta/my-private-repo/commits|https://api.github.com/repos/immuta/bodata/commits>' \
--header 'authorization: token <mytoken>'
. But I get a "not authorized" when adding the header via requests / the tap. what's more is that it looks like requests is adding its own
Basic
authorization to the request payload instead of honoring the passed-in token. Has anyone seen this before?
Copy code
import requests

resp = requests.get(
    "<https://api.github.com/repos/immuta/my-private-repo/commits>",
    headers={"Authorization": "token foo"},
)

print(resp.request.headers)

> {'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Authorization': 'Basic L1VzZXJzL25vdG91Y2gtaG90c3BhcmU6L1VzZX5vdG91Y2gtaG90c3BhcmU='}
The basic authorization toekn, decoded, is
my-mac-user:my-mac-user
🤨 Where is that coming from??
a
You said this is the
singer-io
fork, right? No, I haven't seen anything like that.
s
but i can't figure out what's going on with
requests
itself, because it's just not respecting the headers i'm passing in...
or maybe it is, but when it saves / prints it out from
response.request.headers
it's just sanitizing it. at any rate, i can't auth to work and its making me try not to cry
l
@stephen_bailey I've not tried accessing private repos, as my focus on this project is purely opensource. That said, I've had quite a few hiccups with the
singer-io
tap, and so decided to bite the bullet and move over to the
MeltanoLabs
one instead. I am adding in the streams that I need, but overall the foundations are A LOT stronger.
s
Yes! That’s what I love about the SDK. No need to put on your adventure hat when going into a new repo
Ok, i figured it out — the Python requests package will use the netrc file in a users home directory if no “auth “ keyword is explicitly passed. I set up the netrc on my laptop and on our deployment environment, which explains why the auth process was being overridden for both versions of the tap: https://docs.python-requests.org/en/master/user/authentication/
e
interesting!
s
thats one word for it 🙃