Hi all I ve noticed a peculiar error when trying to run `tap Meltano #plugins-general

Hi all, I've noticed a peculiar error when trying ...

daniel

11/28/2023, 3:50 PM

Hi all, I've noticed a peculiar error when trying to run

tap-gitlab

using the SDK version (

main

branch) from https://github.com/MeltanoLabs/tap-gitlab/tree/main. When running

poetry run tap-gitlab --config config.json

I see:

singer_sdk.exceptions.FatalAPIError: 404 Client Error: Not Found for path: /api/v4/projects/%7Bproject_id%7D

, and the path URL decodes to

/api/v4/projects/{project_id}

, so the

project_id

is not being replaced within the URL. Any ideas why this is happening? cc @edgar_ramirez_mondragon since you've been working on this tap.

Andy Carter

11/28/2023, 4:54 PM

What does the contents of your

config.json

look like? Removing any identifiable information! If you're running the tap directly, rather than via meltano, the environmental variable injection may work differently

daniel

11/28/2023, 4:58 PM

Copy code

{
  "api_url": "<https://gitlab.example.com/api/v4>",
  "graphql_api_url_base": "<https://gitlab.example.com/api>",
  "private_token": "redacted",
  "projects": "some_project",
  "start_date": "2023-10-01T00:00:00Z",
  "ultimate_license": true,
  "fetch_merge_request_commits": false,
  "fetch_pipelines_extended": false,
  "fetch_group_variables": false,
  "fetch_project_variables": false
}

edgar_ramirez_mondragon

11/28/2023, 5:21 PM

I wouldn't say that

project_id

is not being replaced within the URL, but that rather that we only print the unexpanded path if the response object does not have the attribute: https://github.com/meltano/sdk/blob/039c06f8b6c07a3cb33028759ca65e806226a6ad/singer_sdk/streams/rest.py#L206-L218

daniel

11/28/2023, 6:20 PM

@edgar_ramirez_mondragon Related to this tap, I'm also trying to use the

legacy-stable

branch, but it seems to use the project name in URLs, which is not right. Example log messages from Meltano:

Copy code

{
  ...
  "event": "INFO GET <https://gitlab.example.some/api/v4/projects/some_project_name?statistics=1>",
  ...
}
{
  ...
  "event": "INFO Skipping request to <https://gitlab.example.some/api/v4/projects/some_project_name?statistics=1>",
  ...
}
{
  ...
  "event": "INFO Reason: 404 - b'{\"message\":\"404 Project Not Found\"}'",
  ...
}

This is unexpected as the URL should be of the form

<https://gitlab.example.some/api/v4/projects/{project_id}?statistics=1>

, which actually works. Any insight into why it's doing this? Did this work at some point but now doesn't due to API changes?

edgar_ramirez_mondragon

11/28/2023, 6:21 PM

Did this work at some point but now doesn't due to API changes?

Yeah, that's the most likely reason

daniel

11/28/2023, 6:27 PM

Is anyone using this tap successfully that you're aware of? I suppose we will need to use the

main

branch, but would you say it's production ready?

daniel

11/28/2023, 7:16 PM

FYI if we use the project ID in the config rather than the project name it works on

legacy-stable

, even though the example config looks like this:

Copy code

{
  "api_url": "<https://gitlab.com>",
  "private_token": "your-access-token",
  "groups": "myorg mygroup",
  "projects": "myorg/repo-a myorg/repo-b",
  "start_date": "2018-01-01T00:00:00Z",
  "ultimate_license": true,
  "fetch_merge_request_commits": false,
  "fetch_pipelines_extended": false,
  "fetch_group_variables": false,
  "fetch_project_variables": false
}

edgar_ramirez_mondragon

11/28/2023, 10:41 PM

Gotcha, it's probably worth updating the readme then. Thanks @daniel!

Open in Slack

Previous Next