Weird issue: Meltano was working perfectly. Some d...
# announcements
z
Weird issue: Meltano was working perfectly. Some days ago I did some UFW/iptables changes for restricting access to the port and since the changes Meltano doesn't load the data with the buffer days. Details: tap-facebook meltano.yml settings are set at:
insights_buffer_days: '7'
Since the changes (that I now reverted) the problem persists and meltano only extracts data for the CURRENT date. Below I attach screenshots that show airflow's task duration. The task duration has dropped, because it no longer goes back to load the previous 7 days of data. Questions: • Could the iptables be the problem? • Is this a Meltano issue / did I mess up with Meltano's DB? P.S: I guess only @douwe_maan could debug this kind of sorcery...
d
@zlatan Good catch, I've identified the issue and will fix it right away: https://gitlab.com/meltano/meltano/-/issues/2453
Merged: https://gitlab.com/meltano/meltano/-/merge_requests/1955 If you run
meltano config tap-facebook
, you should see
insights_buffer_days: 7
again!
z
Great, it seems that it now works! @douwe_maan would this mean that Meltano loads execution code from external source? I thought that it runs natively on my the server.
d
By default, it loads the manifest of discoverable plugins, with their settings definitions, from https://www.meltano.com/discovery.yml: http://meltano.com/docs/plugins.html#discoverable-plugins http://meltano.com/docs/settings.html#discovery-url
z
Okay, got it! I have a lot to learn. I knew that you will pin down the problem immediately! Thanks for the fast reaction, mate 🙂
d
Thanks for alerting me of this bug! ❤️