I want to replicate our backend database to Snowfl...
# troubleshooting
f
I want to replicate our backend database to Snowflake. We have several country-specific schemas (
us
,
gb
,
fr
) and other miscellaneous schemas. I want to replicate all country schemas to Snowflake as-is (
us
->
us
,
gb
->
gb
) but all other schemas should go to
misc
. How can I achieve this?
I don’t want to hardcode all streams (aka all tables) of my Postgres backend DB.
Copy code
default_target_schema: ${MELTANO_EXTRACT__LOAD_SCHEMA}
Copy code
load_schema: us (or gb or fr)
default_target_schema is set in snowflake target config and load_schema is set in the postgres extractor/tap config.
I think you are going to have to setup multiple extractors for the same database to do this... but there might be a fancy way to do it with map transformer
e
I'd second the use of default_target_schema, used together with schema_mapping (better documented for target-postgres)
Copy code
schema_mapping:
  us:
    target_schema: us
  gb:
    target_schema: gb
  fr:
    target_schema: gb
default_target_schema: misc
w
Right on @edgar_ramirez_mondragon! Haven't used that functionality yet - looks useful. I will also take a look.
f
Thanks everyone 🙏 I’ll try this! I wish this existed with Meltano’s variant!
e
There's actually a feature requests for that: https://github.com/MeltanoLabs/target-postgres/issues/163