Hi everyone, I’m trying to store DynamoDB JSON obj...
# troubleshooting
a
Hi everyone, I’m trying to store DynamoDB JSON objects as string on BigQuery, and using stream maps for that. The stream maps work fine when the JSON object is present in the record, but the new records don’t have the field present immediately, which is breaking the stream map. Tried using
self
as the documentation seemed like it takes care of value present or not, but got error that
self
is not defined in the first place. Can somebody please help me transform the field to string if it exists, and do nothing if it doesn’t?
p
@abhishek_ajmera I'm not sure about the
self
issue but similar to this example https://docs.meltano.com/guide/mappers#example you should be able to use a default value with the python get() method like:
Copy code
transactionId: str(record.get('transactionId', ''))
I'm using a standalone mapper for this example since my plugins were not SDK based but I do something similar for a case where my json record either has
ipv4prefix
or
ipv6prefix
and never both, so I parse out each and default to an empty string if it doenst exist https://github.com/meltano/squared/blob/f0048719743862a4f4601b152e6606e3280e6f83/data/extract/mappers.meltano.yml#L13
a
Got it, thanks!