Hey all I am trying to contribute to my first open-source project here on meltano by creating a target for teradata 🙂 Right now, I am struggling to understand the behaviour of the json_serializer and json_deserializer parameters that are present in the default create_engine method of the SQLConnector class. Unfortunately, the teradata sqlalchemy dialect does not know these parameters. Thus, the default create_engine method is throwing the following error:

"TypeError: Invalid argument(s) 'json_serializer','json_deserializer' sent to create_engine(), using configuration TeradataDialect/SingletonThreadPool/Engine.  Please check that the keyword arguments are appropriate for this combination of components."

I could get the teradata-target functioning by overriding the create_engine method by uncommenting these two parameters (see below). Can someone tell me what is the purpose of these two attributes? Also, Can you suggest if overriding the create_engine class is somewhat problematic? Any help is warmly appreciated 🙂

def create_engine(self) -> Engine:
        return sqlalchemy.create_engine(
            self.sqlalchemy_url,
            echo=False
            # The json_serializer and json_deserializer potperties are apparently not supported in the sqlalchemy dialect of teradata
            # overiding class to remove those
            #json_serializer=self.serialize_json,
            #json_deserializer=self.deserialize_json,
        )

Hi @matthias_gey! Thanks for contributing to Meltano. It's pretty safe and actually recommended to override the method if it's not supported by the sqlalchemy adapter. See for example https://github.com/MeltanoLabs/tap-athena/blob/99909b31eeb2be45336ece045e06be2bca0351fa/tap_athena/client.py#L41-L54. I think I could use some decorator magic in the singer sdk to handle cases when the parameters aren't supported by the adapter.

Cool! Thanks for the reply 🙂 I will continue coding therefore (y)

That's awesome @matthias_gey! The singer SDK ships a battery of standard tests for targets: https://sdk.meltano.com/en/latest/testing.html#testing-targets.