Hello! I'm having an issue with the meltano mapper. When mapping a field in a stream coming from the postgres tap, it happens that some times (in the case of deleted records) the message is mostly empty and only the ID and

_sdc_deleted_at

are available in the record. Thus, the field that I'm mapping does not exist and I'm getting

singer_sdk.exceptions.MapExpressionError: Failed to evaluate simpleeval expressions int(amount)

which makes sense since the field is not in the record which looks like

{'id': '<redacted ID>', '_sdc_deleted_at': '2024-05-31T15:18:35.471584+00:00'}

The configuration looks like:

mappers:
    - name: meltano-map-transformer
      variant: meltano
      pip_url: git+<https://github.com/MeltanoLabs/meltano-map-transform.git>
      executable: meltano-map-transform
      mappings:
      - name: cast_amounts
        config:
          stream_maps:
            public-my_table_name:
              # Integers only
              amount: int(amount)

Any ideas on how to work around these edge cases?

Maybe something like

amount: "int(record['amount']) if 'amount' in record else 0"

That worked. I had tested a few other conditional expressions but didn't think of leveraging the record dict. Thanks a lot @Edgar Ramírez (Arch.dev)

Np!