I've got a feeling that this has been mentioned before, but I can't seem to find it by searching. How are people handling the fact that the

job

table in Meltano will grow indefinitely? I was thinking of modifying

orchestrate/dags/meltano.py

to include a task which will purge this table on a schedule, but just wondering if there's a better way of handling this?

What’s the concern with having the job table grow? Postgres can scale really well.

That's true in general but in this particular case I have a 10,000 row limit 😅 I'm using the Heroku postgres free tier as a proof of concept

Ah ok 😅 Yeah, you can certainly add another DAG in the orchestrate/dags/ folder to do that. I don’t think it will break anything.