Hi all! I'm currently trying to set up `meltano ui...
# troubleshooting
m
Hi all! I'm currently trying to set up
meltano ui
in production, but having some issues similar to previously where the service is calling out to some external URI instead of just relying on local data. We have some pretty strict firewall policies (telco) so opening up an egress to one of these domain is easier said than done. Seems like the UI is calling out to both "discovery.meltano.com" and "pypi.org", see screenshot. In the logs of the container, I see that it has an issue parsing a local discovery.yaml file, which could be the reason why it falls back to reading files from the internet? I don't see why it wouldn't be able to read the file though because the cached
discovery.yml
file is there in the root directory of the project as recommended by the meltano docs. Any ideas what's wrong here?
Development: Turns out I get those error messages even when running in Docker locally, so I guess there is something wrong with my Docker image? Adding the
--log-level=debug
flag gives these clues:
It's the same error as the 800+ linting errors I see in my IDE for that file. Something is definitely wrong here, right? Is it a versioning issue or something?
This is the discovery.yml that is generated. It looks to me like a bug? It looks nothing like the bundled file.
a
@martin_morset I also noticed this when I was working on a feature for meltano on ~Friday to dump the entire project exactly as is parsed/interpreted by meltano to stdout as yaml (like kubectl describe). I got a bunch of those !!comment type things. I was using the
cononical
method in the codebase and figured I was doing something wrong. But maybe its a bigger issue 🤔
c
We recently moved to ruamel.yaml for our yaml parsing in order to preserve anchors, aliases, comments, etc. when parsing and rewriting
meltano.yml
files. At a glance, it would seem that ruamel.yaml might unexpectedly be dumping some additional python objects beyond what's supported by the jsonschemas for
meltano.yml
and
discovery.yml
. @alexander_butler @martin_morset what version of Meltano are you each seeing this issue in? If you've noticed the issue in multiple meltano versions, what release did you first see the issue in? FYI @edgar_ramirez_mondragon
e
meltano.yml
shouldn’t have this problem, but I see how
discovery.yml
could given that it still uses the old yaml library for serde
c
@martin_morset This is what I do to stop meltano from hanging on trying to connect to discovery.meltano.com: `export MELTANO_DISCOVERY_URL=false`https://docs.meltano.com/reference/settings#discovery_url
m
@cody_hanson I'm on
2.3.0
(latest afaik).
meltano.yml
shouldn’t have this problem, but I see how
discovery.yml
could given that it still uses the old yaml library for serde
I think this could definitely be the issue, although I find it difficult to isolate it completely.