When using `meltano-map-transform` is it possible ...
# troubleshooting
i
When using
meltano-map-transform
is it possible to do this, or something like it? I'm not 100% clear from the examples how flexible the mapper is and whether it is possible to use inline functions or arbitrary Python (eg. decimal.Decimal() ) I'm trying to circumvent the
MultipleOf
validation error that occurs with float values.
Copy code
mappings:
      - name: fix_float
        config:
          stream_maps:
            finance_option:
              interest_rate_percentage: "decimal.Decimal(interest_rate_percentage)"
t
https://github.com/MeltanoLabs/meltano-map-transform/blob/main/examples/README.md should give you a sense of what you can do. that’s not a complete listing. https://sdk.meltano.com/en/latest/stream_maps.html#constructing-expressions indicates it uses the simpleval library for parsing
u
If that doesnt work, another alternative is to cast to a string so the value is preserved and it can be handled in the downstream system
i
Casting to string works and circumvents the
MultipleOf
issue (I have tried it now), might be useful to add to the cast functions in mapper. eg.
decimal()
đŸ™‚ Hoping to see this PR https://github.com/meltano/sdk/pull/1809 by @edgar_ramirez_mondragon go through too
e
There's already a PR open to bump the SDK in target-snowflake: https://github.com/MeltanoLabs/target-snowflake/pull/71
u
@edgar_ramirez_mondragon I also have a branch going for this https://github.com/MeltanoLabs/target-snowflake/tree/bump_sdk_0.29.0 since it requires some code changes in the tests and we were override a bunch of stuff thats now in the SDK so it can be removed. I'm stuck on some dependency issues though