Hi there, I’m trying to create a mapper that conve...
# plugins-general
j
Hi there, I’m trying to create a mapper that converts json (or specifically type hstore in postgres) into string to be written to redshift. Chatgpt gave me the following snippet to put in my meltano.yml:
Copy code
- name: meltano-map-transformer
    variant: meltano
    pip_url: meltano-map-transform
    namespace: meltano_map_transformer
    executable: meltano-map-transformer
    config:
      mappings:
        - name: stringify-json-fields  # <-- this is the mapping name
          input: tap-postgres
          transform: |
            {% for key, value in record.items() %}
            {% if value is mapping %}
            record["{{ key }}"] = to_json(value)
            {% elif value is iterable and value | length > 0 and value[0] is mapping %}
            record["{{ key }}"] = to_json(value)
            {% endif %}
            {% endfor %}
            return record
which is met with
block violates set requirements: Expected unique mappings name not the mapper plugin name: stringify-json-mapper.
I’ve been looking through the docs on how to define mappers but I’m wondering if someone can point me to the right direction.
1
e
Hi @Justin Yang 👋 The correct syntax would be something like
Copy code
plugins:
  mappers:
  - name: meltano-map-transformer
    variant: meltano
    pip_url: meltano-map-transform
    mappings:
    - name: stringify-json-fields
      config:
        stream_maps:
          "*":
            json_field_1: json.dumps(json_field_1)
            ...
            json_field_n: json.dumps(json_field_n)
I've also got a PR in the works to support hstore natively: https://github.com/MeltanoLabs/tap-postgres/pull/636
🙌 1
j
hey I got it to work! thank for that, helped a lot. also that hstore native support would be amazing
np 1