i am running meltano locally with docker, i added ...
# troubleshooting
m
i am running meltano locally with docker, i added airflow orchestrator, invoked the webserver and scheduler, but can't get the airflow ui to show on localhost:8080. not sure how to troubleshoot this. any ideas?
this was all done via venv
a
Hi, @mike_planting. I assume you already found the guide here, yes? Do you see any helpful output from
meltano invoke airflow webserver
?
m
Ya
i didn't change the local executor or sql_achemy configs though...
i was able to get to airflow when I ran this without docker implementation,
but when i install via docker and venv, i can't get it show on local host
a
Okay, good to know. Within docker, you’ll need to explicitly map ports to the host and sometimes you have to experiment (depending on your OS) between
localhost
,
0.0.0.0
, and the loopback
127.0. 0.1
. Have you mapped the ports using the
-p
docker run option and/or in the docker-compose file? And if so, can you tell me which OS you are using, since that might be relevant to debugging which endpoint to connect on.
m
macOS BigSure, v 11.4
and i have not mapped ports... do you have commands for that?
a
Sure thing. Are you using
docker run
or
docker-compose up
?
m
ducker run via the walktrhrough on https://meltano.com/
a
I see. Yeah, those commands on the homepage will kick off the
elt
job but they don’t expose the ports or open the ui, unfortunately.
Copy code
docker run \
  --volume $(pwd)/output:/project/output \
  -p 8080:8080 -p 5000:5000 \
  meltano-demo-project:dev \
  elt tap-gitlab target-jsonl --job_id=gitlab-to-jsonl
Before specifying the name of the image to run, you can add one or more
-p
arguments to map the docker container’s ports to the host laptop. https://docs.docker.com/engine/reference/commandline/run/#publish-or-expose-port--p---expose
With docker, since you are basically running an os inside another os, you have to explicitly say which ports from the container os you want available to the host os.
m
ok, i am trying out the above recommendations
that worked, thanks!