Hi I am trying to create a REST tap that requires unescaped Meltano #singer-tap-development

Hi! I am trying to create a REST tap that requires...

Henning Holgersen

11/20/2023, 2:47 PM

Hi! I am trying to create a REST tap that requires unescaped parameters. I have overridden

prepare_request_payload

so that it adds parameters as a string to make sure some special characters aren’t escaped. This is a great SDK feature by the way. The

Request

object that gets generated looks good, but when the SDK creates a

PreparedRequest

the url params gets escaped. I have no clue how this happens, when testing the

prepare_request

method on the

requests.Session

class in isolation seems to return the correct URL, but running the tap in debug mode unequivocally shows that the

Request

object does not escape, but the

PreparedRequest

oject returned from

Session.prepare_request

is escaped. Has anyone seen this before? I am using version 0.32.0 of the singer SDK.

Henning Holgersen

11/20/2023, 9:31 PM

It seems this quoting issue pops up when adding the Authenticator to either the request or prepared request. A very weird workaround I found was to create two prepared requests, one with and one without auth, and then add the headers from the request prepared with auth to the one prepared without auth. This kind of has to be a bug?!?

edgar_ramirez_mondragon

11/21/2023, 2:00 PM

Hey @Henning Holgersen! Does sound like a bug, probably because we re-encode query parameters here: https://github.com/meltano/sdk/blob/206c38ebfc68dc88dced16f47b76d98e32f77ce2/singer_sdk/authenticators.py#L46-L48. Do create a GitHub issue 🙂

Open in Slack

Previous Next