Hi all, maybe this was ill-advised but I decided to truncate Meltano's

job

table - I'm running Meltano on Heroku, and I'm using Heroku Postgres as it's free on the basic plan. The limitation, however, is that you can't have more than 10,000 rows. This is reasonable for my use case though as the only thing which really takes up any rows in there is Meltano's job run log. However, now I'm seeing an error on one of my schedules (which involves a transformation step):

meltano         | Running transformation...
meltano         | This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: Instance '<Job at 0x7fa872ecd358>' has been deleted, or its row is otherwise not present. (Background on this error at: <http://sqlalche.me/e/13/7s2a>)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/meltano/core/logging/output_logger.py", line 135, in redirect_logging
    yield
  File "/usr/local/lib/python3.6/site-packages/meltano/cli/elt.py", line 224, in _redirect_output
    yield
  File "/usr/local/lib/python3.6/site-packages/meltano/cli/elt.py", line 241, in _run_elt
    await _run_transform(elt_context, output_logger)
  File "/usr/local/lib/python3.6/site-packages/meltano/cli/elt.py", line 313, in _run_transform
    await dbt_runner.run(**kwargs, log=transformer_log_writer)
  File "/usr/local/lib/python3.6/site-packages/meltano/core/runner/dbt.py", line 60, in run
    dbt = self.context.transformer_invoker()
  File "/usr/local/lib/python3.6/site-packages/meltano/core/elt_context.py", line 96, in transformer_invoker
    return self.invoker_for(PluginType.TRANSFORMERS)
  File "/usr/local/lib/python3.6/site-packages/meltano/core/elt_context.py", line 84, in invoker_for
    run_dir=self.elt_run_dir,
  File "/usr/local/lib/python3.6/site-packages/meltano/core/elt_context.py", line 69, in elt_run_dir
    return self.project.job_dir(self.job.job_id, str(self.job.run_id))
[>>>>>> SNIP >>> the remaining traceback is all from sqlalchemy]

sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: Instance '<Job at 0x7fa872ecd358>' has been deleted, or its row is otherwise not present. (Background on this error at: <http://sqlalche.me/e/13/7s2a>)

Maybe the issue occurs when you are running multiple ELT process? I'm using Meltano default db to store the job id.