bob_swain
07/28/2022, 2:42 PMaaronsteers
07/28/2022, 5:52 PMaaronsteers
07/28/2022, 5:58 PMsimplecalc
expression evaluator executor. (For instance schema()
could return the stringified json schema.)
2. Create a custom mapping like add-shopify-schema-audits
for the tap stream(s) that you want to add this column to. Something like { "stream_map_config": { "stream_a": {"_schema_audit": "schema()" }, ...}
.
3. When invoking, you'd put the mapping between the tap and target: meltano run tap-shopify add-shopify-schema-audits target-snowflake
bob_swain
07/28/2022, 6:00 PMaaronsteers
07/28/2022, 6:00 PMbob_swain
07/28/2022, 6:00 PMbob_swain
07/28/2022, 6:01 PMaaronsteers
07/28/2022, 6:03 PMedgar_ramirez_mondragon
07/28/2022, 6:03 PMbob_swain
07/28/2022, 6:04 PMbob_swain
07/28/2022, 6:05 PMaaronsteers
07/28/2022, 6:09 PMmyschema-mytable
) and the json schema definition.
This means a stream map operation in the tap could append the schema name as a property but a stream map operation in the target or in an intermediate mapper plugin might not be able to. Just as a heads up.bob_swain
07/28/2022, 6:12 PMbob_swain
07/28/2022, 6:12 PM