Hey All, I seem to have an issue when trying to change the max_batch_rows when using

meltano config dev-target-redshift-datamill--recharge set max_batch_rows

. Although the terminal will return saying the size has been changed, I will typically have to manually change it myself in order for it to work. Could anyone give me insight as to why this is or if there is something else I need to do? Thanks in advance!

Is there any change at all in the config file after running the command?

Yes, but the data doesn't load into redshift as the newly set size

Hey @aaronsteers and @taylor just wanted to bump this, I think Neil posted yesterday in #C01TCRBBJD7 as well @neil_gorman could you summarize the timeout issue with pagination? I think batch size not working was just a symptom

Looking at the code for that variant, I’m not seeing

max_batch_rows

anywhere in the codebase https://github.com/datamill-co/target-redshift/search?q=max_batch_rows&type=code We have heard from the Datamill folks that they’re not really maintaining their singer targets anymore and the one we currently recommend is the one from the Wise folks.