luis_atala
03/31/2021, 2:29 PMaaronsteers
03/31/2021, 3:42 PMluis_atala
03/31/2021, 4:36 PMaaronsteers
04/02/2021, 12:47 AMStream.post_process()
method. That method takes a record dict, gives you the option to modify it in any way you want and then return the result.luis_atala
04/02/2021, 12:49 AMaaronsteers
04/02/2021, 12:50 AMdef post_process(self, row: dict, partition: Optional[dict] = None) -> dict:
"""As needed, append or transform raw data to match expected structure."""
row["new_name"] = row.pop("old_name")
return row
luis_atala
04/02/2021, 12:55 AM