Hello. I am trying out Meltano (1.36.1) and I did

meltano ui setup localhost.localdomain

so that I can use my own Gitlab instance instead of gitlab.com. I am running on localhost. The issue I am facing is that the UI doesn't load if I try to setup access control. The command line doesn't show errors, but the browser shows me a 404. I also added a user with

meltano user add admin admin --role admin

to see if that helps, but it didn't. Any tips on how to proceed? If I trash

ui.cfg

and just run

meltano ui

without trying to secure it, the UI loads okay. Metlano was setup using pip under virtualenv running Python 3.7.7 on MacOS.

Hi @mukund! You shouldn't need

meltano ui setup

if you're running Meltano locally — that command and

ui.cfg

are only relevant when running Meltano in production. It shouldn't be needed to use a different GitLab instance than

<http://gitlab.com|gitlab.com>

either; running

meltano config tap-gitlab set api_url <URL>

should do the trick. Did you try that?

Thank you for the reply. The suggestion worked. I had to install the dbt transformer and the Gitlab extractor and it appeared to do its job. Unfortunately the Gitlab server has certificate issues as the chain information is not complete. So I ended up with SSL exceptions while trying to connect. I am trying to work with the admins to get that fixed.

Ah yeah, editing the URL in the UI is not currently possible for security reasons. I'd suggest using the CLI as much as possible right now, since that's the most feature complete and best supported!

So I made progress and I was able to view reports.

meltano add transformer dbt

meltano add extractor --include-related tap-gitlab

meltano config tap-gitlab set api_url"<>"

meltano add loader target-postgres

Edit the .env file and add Postgres and Gitlab details. Edit meltano.yml and add the three variables required by dbt for transformation.

entry_table: "{{ env_var('PG_SCHEMA') }}.entry"

generationmix_table: "{{ env_var('PG_SCHEMA') }}.generationmix"

region_table: "{{ env_var('PG_SCHEMA') }}.region"

Finally run

meltano elt tap-gitlab target-postgres --transform run

@mukund I just saw your blog post at https://mukundmr.wordpress.com/2020/06/25/using-custom-gitlab-with-meltano/, nice work! A couple of small details: • You suggest running

meltano config tap-gitlab set api_url="<<your customer server base url>>"

, but the

=

actually breaks the command! Dropping the

=

and just using a space should do. • You write "edit the meltano.yml file and add the following to the vars section of transforms for tap-gitlab.", but I don't think that should actually be needed. Did you see an error without those extra

vars

keys?

I will fix that. The ‘=’ doesn’t belong. Without the entries in meltano.yml, the transformation runs and returns no errors, but nothing happens. The new schema for analytics doesn’t get created and when I explore, the reports and charts fail with SQL errors. Adding the environment variables and running the transforms fixed everything.

@mukund Hmm, interesting. That may be a bug! In the demo-project at https://gitlab.com/meltano/demo-project,

meltano.yml

does not have those 3 extra

vars

you mentioned, just the two that are added automatically when you run

meltano add transform tap-gitlab

: https://gitlab.com/meltano/demo-project/-/blob/master/meltano.yml#L55-59 And

meltano elt tap-gitlab target-postgres --transform=run --job_id=gitlab-to-postgres

runs correctly and ``SELECT * FROM analytics.gitlab_tags LIMIT 1;`` returns the expected result: https://gitlab.com/meltano/demo-project/-/jobs/612297881