Hey! I'm trying to get meltano to pull data from multiple schemas in postgres and I'd like to be able to map those to similar names on the destination side but everything always ends up in the same destination schema regardless, so I'm not sure what I'm missing. Any thoughts or ideas?

version: 1
default_environment: dev
plugins:
  extractors:
  - name: tap-postgres
    variant: transferwise
    pip_url: pipelinewise-tap-postgres
  loaders:
  - name: target-postgres
    variant: transferwise
    pip_url: pipelinewise-target-postgres
environments:
- name: dev
  config:
    plugins:
      extractors:
      - name: tap-postgres
        config:
          filter_schemas: brs,flow
          default_replication_method: INCREMENTAL

      loaders:
      - name: target-postgres
        config:
          add_metadata_columns: true
          default_target_schema: analytics
          schema_mapping:
            brs-my_table: my_schema

I’m not sure if this is the whole issue, but part of it is that you can’t use

default_target_schema

together with

schema_mapping

- if you’re using the latter, don’t set the former.

I just gave that a shot and still nothing. I've narrowed down my tap to only select one table from each schema and tried putting an entry in the schema_mapping for each but it seems like it just ignores it.. am i missing the point of the schema_mapping?

I’m using

target-redshift

and

schema_mapping

successfully, here’s what I’ve got as far as setup: tap:

- name: tap-s3-mongo
      inherit_from: tap-spreadsheets-anywhere

mapper:

- name: meltano-map-transformer
      variant: meltano
      pip_url: git+<https://github.com/MeltanoLabs/meltano-map-transform.git@8329defbf4be569ecd3bbe749ae052ff2291499a>
      mappings:
        - name: split-mongo-streams
          config:
            stream_maps:
              # Note: the naming of these streams (e.g. `payment_service-payments` is very
              # important: they must be in the form `<stream_schema>-<stream_table>` in order for the downstream
              # target-redshift loader to place them in the correct location.
              payment_service-payments:
                __source__: raw_mongodb
                __filter__: 'database == "payment_service" and collection == "Payment"'

loader:

- name: target-redshift-mongo
      inherit_from: target-redshift
      config:
        # <https://cs.github.com/samklr/pipelinewise/blob/eca52d4fa566f27c53b206a59ada86ee7d986a68/pipelinewise/fastsync/commons/utils.py#L149>
        # in order for these mappings to work, the streams provided to this target must have names in the format `<stream_schema>-<stream_table>`.
        # This schema_mapping field defines the mapping from a _stream schema_ to a _database schema_.
        schema_mapping:
          payment_service:
            target_schema: payment_service
        dbname: staging

hey, I got it!