Trish Gillett
08/12/2024, 8:24 PMThe extractor generated a message exceeding the message size limit of 5.0MiB (half the buffer size of 10.0MiB).
To let this message be processed, increase the 'elt.buffer_size' setting to at least double the size of the largest expected message, and try again.
To learn more, visit <https://docs.meltano.com/reference/settings#eltbuffer_size>
which links to this high level config. I'm working in a system with many Meltano extractors in use, and ideally don't want to change a setting that will impact other users' extractors, so I'm trying to figure out if there's a way to make this change in my extractor only.
This issue looks useful, and is closed as completed with a note Covered by SDK features
, but I'm not sure what that means for my case, anyone able to help?Edgar RamÃrez (Arch.dev)
08/12/2024, 8:49 PMCovered by SDK features
is incorrect 😓. Let me reopen it and explain there why it's still relevant.Trish Gillett
08/12/2024, 9:10 PMTrish Gillett
08/12/2024, 9:30 PMMELTANO_ELT_BUFFER_SIZE
)
• Specifying that environment variable in the plugin config (the # root level plugin env
case shown)
🤔Edgar RamÃrez (Arch.dev)
08/13/2024, 3:09 AMvisch
08/13/2024, 6:34 PM