hey guys, let's say im using schema files (like campaigns.json) and i want to add a transformation move (for example, remove any property that is of type object) where would i fire that off? was thinking in

streams.py

but it seems that it's only using the path to the schema file, and not opening it up there... so, let's say i want to do something like

field_names = [field for field in schema['properties'].keys() if schema['properties'][field]['type'][0] != 'object']

If you want to do record transformations in a tap built with the SDK, you should define a

post_process

method for your stream class and implement that logic there. https://sdk.meltano.com/en/latest/classes/singer_sdk.Stream.html#singer_sdk.Stream.post_process

@Reuben (Matatika) hey i actually dont want to transform anything, i want to manipulate the schema file that i made for one stream. so basically if in the schema file i have like 3 fields, i want to be able to remove one field using string replacement on the file. so wondering if i can drop a quick python function or move somewhere before meltano reads the file

I don't think there is any mechanism to modify incoming

.json

schemas - you can define schemas as Python classes as well though, so you might be able to do some property filtering with some Python expression in that case. Related: https://meltano.slack.com/archives/C013EKWA2Q1/p1685461495074349?thread_ts=1685432497.736699&cid=C013EKWA2Q1