(Singer SDK SQL Targets) Does `create_empty_table`...
# singer-targets
v
(Singer SDK SQL Targets) Does
create_empty_table
(here https://github.com/meltano/sdk/blob/main/singer_sdk/connectors/sql.py#L565 ) being in the
SQLConnector
make sense? My actual problem I'm trying to solve is I want access to the table metadata which is build as a SQLAlchmy MetaData object with SQL Alchemy columns, but for Values we don't have a good way of knowing which type this object is. Actual actual (lol) issue I'm having I"m having an issue inserting JSON data into a JSONB field in Postgres. I believe I have to wrap the value with the correct construct here https://docs.sqlalchemy.org/en/14/dialects/postgresql.html#postgresql-data-types-and-custom-sql-constructs to use the psycopg JSON extra
psycopg2.extras.Json
here https://www.psycopg.org/docs/extras.html#json-adaptation Not a big deal to force this to work for this instance but I'm trying to help with the general case here in the SDK.
I think @edgar_ramirez_mondragon and I talked a bit about this a month or two ago using sql alchemy constructs more instead of building the insert statements manually
@BuzzCutNorman’s implementation here https://github.com/BuzzCutNorman/target-mssql/blob/main/target_mssql/sinks.py#L104 is pretty good, we may want to play with this some and get something like it into the sdk
e
that implementation’s really nice. It even throws the actual insert inside of a transaction which I’m not sure we really encourage (but should!) in the sdk
b
This has worked for postgres and mssql so far
Copy code
from sqlalchemy import insert
    
    def generate_insert_statement(
        self,
        full_table_name: str,
        schema: dict,
    ) -> str:
        """Generate an insert statement for the given records.

        Args:
            full_table_name: the target table name.
            schema: the JSON schema for the new table.

        Returns:
            An insert statement.
        """
        statement = insert(self.connector.get_table(full_table_name))
        return statement
you can make it more specific for postgres with
from sqlalchemy.dialects.postgresql import insert