https://linen.dev logo
Join Slack
Powered by
After playing a bit with `tap-gitlab` it looks to ...
# plugins-general
t
After playing a bit with 
tap-gitlab
it looks to me, that it doesn’t synchronize all commits, but only those of the default branch. Looking at the gitlab api seems to confirm this idea, since the commits endpoint defaults to the default branch if none is given. Is there a way to configure 
tap-gitlab
to pull ALL commits, and not only those of a specific branch?
I just looked through the issues in tap-gitlab and found this: https://github.com/MeltanoLabs/tap-gitlab/issues/61 There was even a PR up for it, but it looks abandoned now. The reason why this wasn’t merged yet was a unresolved discussion on if adding ALL commits would be a breaking change to existing pipelines, and therefore an additional optional parameter was preferred. This got never implemented.
e
Yup, the contributor unfortunately became unresponsive, but the PR was close to being ready to merge if you’d be interested in contributing with that as a starting point 🙂
t
@edgar_ramirez_mondragon I looked into the PR and actually just saw one change, which is a one-line change to the gitignore file. Am I looking in the wrong place? https://github.com/MeltanoLabs/tap-gitlab/pull/63/files I would be willing to contribute, but I’ll have to get my testing setup in shape for that. So, no clue how fast I can get something done, but I guess I’m the one who’s waiting for the feature anyways 😉
e
Oh I see the user force-pushed at some point so maybe that’s why some of the work was lost 😥.
I would be willing to contribute, but I’ll have to get my testing setup in shape for that.
Yeah, no worries. Take your time 🙂
And lmk if I can help
t
will do 👍
@edgar_ramirez_mondragon I’ve got my setup running, but I’m facing a weird issue: when I run target_gitlab against our company gitlab server, it doesn’t recognize some of the groups. for those that are recognized I get the wrong group IDs. I managed to find the graphQL query that is constructed and ran it via curl, but when I do this, I get correct results. What is the best way to figure out, what request is actually sent, because the request sent by tap-gitlab must be different than the one I used. I tried to dig into the Singer SDK already, but didn’t succeed yet
OK, I figured it out. It is not described in the readme, but setting 
api_url
is NOT enough. Additionally, 
graphql_api_url_base
needs to be set as well. Otherwise, some requests go via REST to the newly defined gitlab server, whereas the graphql requests still go to 
<http://gitlab.com|gitlab.com>
made a first PR to adapt the README, but tests fail with 
E           singer_sdk.exceptions.FatalAPIError: 401 Client Error: Unauthorized for path: /api/v4/groups/13315624/labels
since I only touched the README, this sounds like it was there before?
One more question: I just noticed that my meltano installation works with 
tap-gitlab
0.9.15, but when I’m in the 
tap-gitlab
repo, I work on 2.0.0 . So should I • make my meltano installation work with 
tap-gitlab 2.0.0
? can I do that via 
meltano add
? • should I work on the all commits topic against the 
legacy-stable
branch instead of 
main
?
@edgar_ramirez_mondragon Hi again. my PR is merged by now, so just the last question remains: can I get the 2.0.0 version of tap-gitlab via 
meltano add
?
in that case I would continue my work on the 2.0.0
e
Yeah, 2.0 work lands in the 
main
branch so you should be able add with 
meltano add tap-gitlab
then changing the 
pip_url
in 
meltano.yml
to point to that branch
Copy code
pip_url: git+<https://github.com/MeltanoLabs/tap-gitlab.git@main>
t
ah cool, that sounds good! I guess we should put a section on that into the readme as well
e
yeah, that probably makes sense in both the tap-gitlab readme here and in the Hub page under the 
next_steps
field: https://github.com/meltano/hub/blob/30be2403c5d87a762a259014f8cc6cb6d48fc16c/_data/meltano/extractors/tap-gitlab/meltanolabs.yml#L120
2 Views
Open in Slack
PreviousNext
Powered by