I started getting an error when running my product...
# troubleshooting
h
I started getting an error when running my production E/L today (yesterday, US-West time). I have previously had some difficulties with Meltano reaching out to hub.meltano.com, and I ended up opening the FW because the
send_anonymous_usage_stats
flag (as well as the
meltano lock
command) didn’t seem to prevent it from reaching out. Now, it seems meltano tries to contact some random subdomain of
<http://execute-api.us-west-2.amazonaws.com|execute-api.us-west-2.amazonaws.com>
, and fails because it isn’t allowed to. I have not rebuilt the Meltano Docker image or anything, it simply started failing. I am using Meltano version 2.9. Does anyone know of a fix? I am hesitant to keep chasing FW rules as a solution.
This does not seem to be an issue when using
tap-sharepointsites
and
target-snowflake
, which are both on the hub, and one other that isn’t. The error seems to be related to some details around how taps that aren’t on the hub are declared.
e
Hi @Henning Holgersen! We recently switched over our hosting of the Hub API, that's why you're seeing that new URL. Do all your non-custom taps have a lockfile?
c
We recently switched over our hosting of the Hub API,
That's interesting ... I have not whitelisted any new domains in the last weeks. And have not had any issues with installing from the hub. And
<http://execute-api.us-west-2.amazonaws.com|execute-api.us-west-2.amazonaws.com>
is certainly not in my firewall whitelist.
h
When I first set up my project, I had a big fight with the incantations needed to prevent it from calling home. I gave up. At least hub.meltano was seemed an innocuous and self-explanatory domain. Perhaps the most interesting finding right now is that three of my four plugins don’t call home. There are some variations in the settings, so hopefully I will discover what makes the difference. But… it isn’t exactly intuitive to understand. I have to read more about lock files now.
c
Btw. one thing that I use for testing all the egress network communication is a local Unbound DNS Resolver with Query Logging turned on in a development VM.
e
h
@edgar_ramirez_mondragon I got this working again by emulating the config on the custom tap that works. Adding executable, namespace, capabilities and settings did the trick. Don’t know which of them did it, but that’s easy to find out. In short, this might be seen as a documentation issue instead.
e
Cool. It's the
namespace
which qualifies it as a custom plugin