I have one extractor/loader process that takes north of 15 minutes. It only takes about 80 seconds for the API calls portion. The rest is held up on presumably JSON deserialization. Is there any way I can speed this up?

Hey Tanner, I've sent this over to some of Meltano Engineers, they'll pop up here with an answer soon melty

Hi @Tanner Wilcox!

The rest is held up on presumably JSON deserialization

You mean JSON deserialization on the loader's end while parsing Singer messages? Are there any logs you could share of this pipeline?

I could run it to get you some logs but it would look like any other extractor. I think the bottleneck is Python taking the jsonb, serializing that into json, then the python on the end of the pg loader has to take that json and deserialize it before it can put it into postgres. I'm working on a utility that makes the rest calls and pumps the raw jsonb right into postgres so postgres can do the serialization