I'm just catching up with some recent changes to M...
# announcements
a
I'm just catching up with some recent changes to Meltano and I see that MELTANO_UI_PORT is now aliased to PORT, which is convenient as this is what services like Heroku use. Has anyone deployed Meltano to Heroku? If so, how are you setting the PORT env var - in the Dockerfile or a .env file (or some other way)?
d
@al_whatmough I think @julian_knight is deploying to a Heroku-like system, which is exactly why I added the
PORT
env var! Heroku sets that env var automatically and expects apps to use it, so I don't think you'd need to set it in the
Dockerfile
or
.env
on top of that?
a
Thanks @douwe_maan and @julian_knight, got some good advice from both of you here and it's helped me get closer to finally having this up and running
The issue right now is that on Heroku the pipelines aren't running even though they're scheduled to run
Is it possible to see the Airflow UI so I can verify that Airflow is actually running? I'm currently not sure how best to check its status. As this is deployed on Heroku, the Meltano UI and Airflow are both running on the same dyno
d
You can spin up the Airflow webserver using
meltano invoke airflow webserver
. I assume you are actually running
meltano invoke airflow scheduler
as well?
a
Yep I have the following script which I invoke at the end of the Dockerfile:
Copy code
#!/bin/bash
set -e
export MELTANO_DATABASE_URI=$DATABASE_URL
meltano ui start
meltano invoke airflow scheduler
not sure if this is the best approach(?) but it at least gets the Meltano frontend up and running for me
d
@al_whatmough I think I see the problem! 🎉
meltano ui start
is not backgrounded, so it will never exit, and the script will never get to
meltano invoke airflow scheduler
!
You should see the same thing if you run the script locally
a
@douwe_maan ah very good point! Well spotted, I'll give that a try
I'll give that a go, thanks @douwe_maan Was also thinking about the Airflow webserver but AFAIK Heroku just maps $PORT to port 80, so that lets me access the Meltano UI but not the Airflow UI
@douwe_maan Do you know how to background
meltano ui start
? I tried putting a
&
afterwards but it still runs in the foreground when testing locally. Maybe I should just run Airflow first in the background by passing
-D
and then start Meltano?
d
@al_whatmough If you add
&
, it will probably still send its stdout/stderr to your current shell, but it will actually get to the next command
But yes, running the scheduler with
-D
first would probably work too