stephen_lloyd
04/12/2021, 3:39 AM{
"teams": [
{...},
{...}
]
}
Some endpoints do it, some don’t. I want to insert the objects in teams[] into Snowflake instead as rows instead of a single row called teams. I worked it out this way:
Add def parse_response()
to my class and yield
each object in the array with a for loop.
def parse_response(self, response) -> Iterable[dict]:
"""Parse Google Analytics API response into individual records."""
data =response.json().get("teams")
for team in data:
yield team
stephen_bailey
04/12/2021, 1:29 PMtap-immuta
as well. i added response_result_key
property to the top-level Stream class, which could be defined by each subclass. Then i did the same thing as you in `parse_response`:
if self.response_result_key:
resp_json = resp_json.get(self.response_result_key)
stephen_bailey
04/12/2021, 1:34 PM