https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
e

edward_chen

03/24/2022, 6:50 PM
Hi all, another quick question. I'm trying to write a target into S3 so I came across this one that's already written, https://github.com/transferwise/pipelinewise-target-s3-csv. It seems like it'll get the job done from a JSON output that I get but it's hitting an issue with decoding the JSON. I took a look at the output and it's returning back something like this:
Copy code
{
    "type": "SCHEMA",
    "stream": "licenses",
    "schema": {
        "properties": {...},
    "key_properties": [
        "id"
    ]
}
{
    "type": "RECORD",
    "stream": "licenses",
    "record": {
        "id": "random id",
        "severity": "none",
        "instructions": "",
        "dependencies": [],
        "projects": []
    },
    "time_extracted": "2022-03-24T07:46:29.299498Z"
}
So my thoughts are that I must be doing the
parse_response
incorrectly but the function is pretty simple:
Copy code
def parse_response(self, response) -> Iterable[dict]:
        """Parse Snyk License API response into individual records."""
        data = response.json().get("results")
        for record in data:
            yield record
My main question is where does
Copy code
{
    "type": "SCHEMA",
    "stream": "licenses",
    "schema": {
        "properties": {...},
    "key_properties": [
        "id"
    ]
}
get generated? Or even taking a step back, would it be quicker to just append a comma between the JSON objects?