Hi I'm running into an issue (?) with the Meltano ...
# announcements
t
Hi I'm running into an issue (?) with the Meltano UI where after a '@once' pipeline job fails and I click Run again, it doesn't seem to actually run anything. It remains spinning in the UI and there doesn't seem to be a new folder in the .meltano/logs directory for that pipeline corresponding to a new job being run. The solution to this seems to be to delete the pipeline and reinstall the extractor
d
@tyler_kahn That's odd! When you run
meltano ui
in debug mode using
meltano --log-level=debug ui
, do you see anything in the logs about the "Run" request?
t
I will try that and let you know what I see. Thanks!
Looks like bouncing the server fixed the issue
d
Well then!
Be aware of this bug with the Run button btw: https://gitlab.com/meltano/meltano/-/issues/2379
t
Actually maybe I spoke too soon. This is what I saw in the logs when I hit 'Run':
Copy code
2020-11-18 18:26:31,799] [24|MainThread|meltano.api.executor] [DEBUG] Defered `run_elt` to the executor with {'ended_at': '2020-11-18T18:06:39Z', 'extractor': 'tap-outreach-autoserve1', 'has_error': True, 'has_ever_succeeded': False, 'interval
': '@once', 'is_running': True, 'job_id': 'outreach-autoserve1-to-pipelinewise-target-redshift3', 'loader': 'pipelinewise-target-redshift', 'name': 'outreach-autoserve1-to-pipelinewise-target-redshift3', 'start_date': '2015-09-21', 'started_at'
: '2020-11-18T17:58:15Z', 'transform': 'skip', 'trigger': 'ui', 'is_deleting': False, 'is_saving': False}.
Looking at meltano.db I see an entry for the job and I see a new folder within pipeline's log folder within .meltano but the elt.log file is empty
d
Hmm
What's the state of the job? running?
t
Yes, RUNNING and the payload is
{}
d
The Run button "just" runs
meltano elt
with the args from the payload as you can see here: https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/api/executor/__init__.py#L33
And since you're seeing a new job record, the command did actually get invoked successfully
If you run that exact
meltano elt
command on the CLI, does logging work as expected?
t
I'll try that
Looks like same issue so it appears that this is unrelated to the UI
d
OK, that's not great, but at least it's something
t
The extractor crashed in a really bad state during the last failing run so that might have something to do with it
d
Some of the output from
meltano elt
should still be making it into the log file
You're seeing the
meltano |
lines etc?
t
The log file is actually empty
d
Right, but you're seeing output in your terminal?
If you want, we can jump on a call for a few moments, maybe I can spot something
t
The command actually terminated without any output but it's possible that's related to the fact I'm running it via docker
Sure a call would be great. I really appreciate it. I don't want to take up too much of your time since it's very possible this is just some weird state I've gotten my Meltano setup into. I had to fork and modify the extractor I'm using
d