hey folks, trying to run a python script to decrypt columns from postgres via a mapper like so

- name: decrypt_metadata_mapper
        config:
          stream_maps:
            public-user_details:
              decrypted_metadata: >
                __import__('transform.utils.decrypt_metadata', fromlist=['decrypt_metadata']).decrypt_metadata(
                  record['metadata_encrypted'], record['salt']
                )
              salt: __NULL__

Running

tap-core-db-source decrypt_metadata_mapper target-snowflake-core-db-source

I am getting some import errors. Is this the correct way to do this or am i doing an antipattern? What is the pattern for trying to run a python script in flight ? I am trying to decrypt some columns between the tap and target. Is a custom utility a better way ?

What does the custom script look like?

The source data has a encrypted column and a salt column the python script uses both to decrypt them. Leverages the cryptography python package .

But does it inherit from the SDK's mapper class?

Oh, no it does not. Just a one off script

Then

stream_maps

is definitely not going to work. The simplest version of what you want probably looks something like https://github.com/edgarrmondragon/singer-playground/blob/main/merge_streams/map.py.

sounds good @Edgar Ramírez (Arch.dev) so I created a folder like so (just to make in installable) and defined the custom meltano mapper plugin that inherits from

MapperPlugin

. I am having trouble passing the

meltano install

. I defined in my my transformers yaml like so:

plugins:
  mappers:

  - name: decrypt-metadata-mapper
    namespace: decrypt_metadata_mapper
    pip_url: -e ./transform/custom_decrypt_mapper
    executable: custom-decrypt-metadata
    config:
      decryption_targets:
        public-user_details:
          - encrypted_col: metadata_encrypted
            salt_col: salt
            output_col: decrypted_metadata

but running into

2025-05-12T18:07:17.963854Z [error    ] Mapper 'decrypt-metadata-mapper' could not be installed: Failed to install plugin 'decrypt-metadata-mapper'.
2025-05-12T18:07:17.963916Z [info     ] ERROR: ./transform/custom_decrypt_mapper is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with bzr+http, bzr+https, bzr+ssh, bzr+sftp, bzr+ftp, bzr+lp, bzr+file, git+http, git+https, git+ssh, git+git, git+file, hg+file, hg+http, hg+https, hg+ssh, hg+static-http, svn+ssh, svn+http, svn+https, svn+svn, svn+file).

it is a local path ?