Hi everyone. A simple question. Can I add a new fi...
# troubleshooting
e
Hi everyone. A simple question. Can I add a new field to a stream before loading it? I already tried stream_maps config in the tap like in the screenshot. But I got "...raise MapExpressionError(singer_sdk.exceptions.MapExpressionError: Failed to evaluate simpleeval expressions datetime.datetime.now().""" all the time. I also tested the example in the SDK documentation that is in here: https://sdk.meltano.com/en/latest/stream_maps.html#add-a-property-with-a-string-literal-value but the issue still persists like "... Failed to evaluate simpleeval expressions client-123". More, there are other errors in the terminal like "...raise NameNotDefined(node.id, self.expr)singer_sdk.helpers._simpleeval.NameNotDefined: 'datetime' is not defined for expression 'datetime.datetime.now()'" . Did you ever face the issue like that? Is it applicable?
1
e
Hi @Emre Üstündağ Which tap is this?
datetime
support was added to stream maps a long while ago (v0.14.0 of singer-sdk) but the tap might be on an even older version.
e
tap-amazon-sp hotgluexyz variant
e
Yeah, so that tap is on a really old version of the SDK: https://gitlab.com/hotglue/tap-amazon-seller/-/blob/main/pyproject.toml?ref_type=heads#L15 It's on
^0.4.4
, which is equivalent to
>=0.4.4 <0.5.0
, which is gonna get you at most
0.4.9
(released May 12, 2022). It might be worth forking the tap to bump the version.
e
I got that. thanks
Let me ask you something else Edgar. Ok I cannot do that because this tap has an older version. I am seeking the possible solutions, without editting the original tap. Can I do the same thing with meltano's inline data mapper plugin? Or should I use dbt? My current stack uses dbt seperately from meltano. But I only need to add a loadedAt or extractedAt field to some streams as a simple transformation. Edit: I installed meltano map transformer and solve the issue without any configuration on tap or target. Thanks again
e
Ah yeah, that was an option too of course. Glad you figured it out!
🙌 1
1
🙏 1