I am coming to the conclusion we don't need a persistent System DB for Meltano, We can use S3 for State and SYSTEMDB can be fresh every time my container runs it doesn't make a difference until I call el with stateid and it in S3 , Am I going right or not having persistent DB is going to cause any issues

That's how I run it in production with the sqlite db. It only causes issues when I have a run that exited suddenly and meltano thinks there is a run in progress. In that case I tend to restart the container and flatten the db. I use Az blob storage for state

perfect Thanks @Andy Carter