Hi everyone 👋 Currently we are using https://github.com/signal-ai/tap-redshift but can no longer use it because there is a problem with the package definition. We have opened the MR to fix the bug but the maintainers do not seem to be active anymore. So we are trying to migrate to https://github.com/Monad-Inc/tap-redshift but we have the following issue:

psycopg2.errors.InternalError_: Fetch size 40000 exceeds the limit of 1000 for a single node configuration. Reduce the client fetch/cache size or upgrade to a multi node installation.

Has anyone had this issue ? If yes , please help us 🙂 🙂

It looks like the tap uses a fetch_all call that requests too many records for your Redshift cluster. From this thread it seems your options are to upgrade your cluster 👎 or have the tap paginate the results instead of using fetch_all. It looks like the tap already does what I think is the equivalent of fetchone in the incremental sync code path.

Thanks @pat_nadolny, I understand better. I think the easiest option is to fix the tap that worked well before https://github.com/signal-ai/tap-redshift. I have already opened an MR, I hope someone will validate it https://github.com/signal-ai/tap-redshift/pull/4. In the meantime I will use a fork from the repo. Thank you for all