"Mapper not known" Hi - I've previously asked abou...
# troubleshooting
a
"Mapper not known" Hi - I've previously asked about how to deal with malformed bytes in a stream, and am now trying to use a mapper to see if I can fix the issue. However, I can't seem to get Meltano to recognise the mapper. I've done the following in my project:
meltano add mapper meltano-map-transformer
This adds the mapper OK (if I run it again, it recognises that the mapper already exists) i have added a mapper to my config under "plugins":
mappers:
- name: meltano-map-transformer
pip_url: git+<https://github.com/MeltanoLabs/meltano-map-transform.git>
mappings:
- name: fix_bad_bytes
config:
stream_maps:
customers:
emergency_contact: md5(emergency_contact)
(This just md5's the field for testing) When I try:
meltano run tap-mssql fix_bad_bytes target-snowflake
I get: `Mapper 'meltano-map-transformer' is not known to Meltano. Try running
meltano lock --update --all
to ensure your plugins are up to date.` and running:
meltano lock --update --all
returns:
PS D:\Users\data.analyst\Documents\dev\data_collection_testing\tch_mssql_snowflake> meltano lock --update --all
2025-01-13T10:11:04.738019Z [warning  ] Failed to create symlink to 'meltano.exe': administrator privilege required
Locking 4 plugin(s)...
Locked definition for extractor tap-mssql
Locked definition for loader target-snowflake
Locked definition for mapper meltano-map-transformer
Locked definition for mapper meltano-map-transformer
So it appears all is good, however, on running:
meltano config meltano-map-transformer list
I get:
2025-01-13T10:30:07.481599Z [warning  ] Failed to create symlink to 'meltano.exe': administrator privilege required
`2025-01-13T103007.488828Z [info ] The default environment 'dev' will be ignored for
meltano config
. To configure a specific environment, please use the option
--environment=<environment name>
.` `Mapper 'meltano-map-transformer' is not known to Meltano. Try running
meltano lock --update --all
to ensure your plugins are up to date.` I can't seem to get it to recognise that the mapper is there - I'm guessing I'm missing something straightforward?
e
I think for some reason your mapper is missing the
variant
field? Try
Copy code
mappers:
  - name: meltano-map-transformer
    variant: meltano
     ...
a
Thanks @Edgar Ramírez (Arch.dev) - think you are right that that was causing an issue. I managed to clear everything out to do with mappings, and recreate - and now there is a variant in there. Potentially me mixing up a tutorial script and the cli installation and ended up missing that line.
e
Awesome! I'll double check the tutorials and getting started guides in case we're missing something there.