Has anyone managed to use a pipelinewise tap and t...
# announcements
n
Has anyone managed to use a pipelinewise tap and target and configure the schema_mapping?
To achieve the functionality here of specifying a destination schema from the tap. https://transferwise.github.io/pipelinewise/connectors/taps/mysql.html
h
I have been trying to implement a data ingestion from tap_s3_csv and load into postgres db using pipelinewise. So far, everything seems to be set up correctly but running into SSL errors which I'm unable to debug at this point.
d
@niall_woodward You can see an example of
schema_mapping
here: https://github.com/transferwise/pipelinewise/blob/6b4c6957763b66b1e760f91241f60d7578df8be4/tests/units/cli/resources/sample_json_config/target_one/tap_one/inheritable_config.json#L6. Have you tried building the
{ "<stream>": { "target_schema": "<schema>" } }
dictionary yourself?
It's a setting on the target though, not the tap. If you want to have the schema be dependent on the tap, you can use https://meltano.com/docs/plugins.html#load-schema-extra and reference
$MELTANO_EXTRACT__LOAD_SCHEMA
from the
pipelinewise-target-foo
default_target_schema
setting.
n
This is great! Thanks for the link, the second is exactly what I was after.