:waves: I'm running into this error with a tap-git...
# troubleshooting
t
waves I'm running into this error with a tap-github extractor:
Copy code
The extractor generated a message exceeding the message size limit of 5.0MiB (half the buffer size of 10.0MiB).
To let this message be processed, increase the 'elt.buffer_size' setting to at least double the size of the largest expected message, and try again.
To learn more, visit <https://docs.meltano.com/reference/settings#eltbuffer_size>
which links to this high level config. I'm working in a system with many Meltano extractors in use, and ideally don't want to change a setting that will impact other users' extractors, so I'm trying to figure out if there's a way to make this change in my extractor only. This issue looks useful, and is closed as completed with a note
Covered by SDK features
, but I'm not sure what that means for my case, anyone able to help?
👀 1
e
Oh, I think
Covered by SDK features
is incorrect 😓. Let me reopen it and explain there why it's still relevant.
t
Thanks for clarifying!
Ooh, found a potential workaround by combining these two: • Controlling buffer_size via an environment variable (
MELTANO_ELT_BUFFER_SIZE
) • Specifying that environment variable in the plugin config (the
# root level plugin env
case shown) 🤔
e
Ah, that might work!
v
the swap here is pretty harmless too, I think the default is much too low but 🤷