Hi All! I am doing full reload in meltano. Full re...
# troubleshooting
s
Hi All! I am doing full reload in meltano. Full reload is time consuming.It ran for 20 hours and failed . Can any one help me for how can I restart meltano from point where it had failed in full refresh. If I will start from beginning it will consume my snowflake credit. Please help me out! Let me know any one has any thought on this☝️
t
The only way I've found to accomplishing this is to do the full refresh in stages (which I know sounds like a contradiction). In our case we're replicating from MySQL to Postgres, so for a full refresh I'll drop all the tables in Postgres, set the select list to A*, then run the pipeline; add B*, run the pipeline; add C*, run the pipeline; etc. That way when the error occurs only 1/26th of the tables are affected, I can fix that, then move on.
s
@thomas_briggs Thanks for looking at this and suggesting me the approach I will give a try to this.