Hey all Not sure if this the best place to ask but I m tryin Meltano #plugins-general

Hey all! Not sure if this the best place to ask, b...

saadat_qadri

04/05/2021, 4:52 PM

Hey all! Not sure if this the best place to ask, but I’m trying to specify a JSON schema for an API endpoint where one of the objects has an auto-incrementing key, like so:

Copy code

{
  "data": {
    "2": {
      "id": 2,
      "type": "HIDDEN",
      "question": "Question1",
      "section_id": 1,
      "shown": true
    },
    "3": {
      "id": 3,
      "type": "HIDDEN",
      "question": "Email Address",
      "section_id": 1,
      "shown": true
    },
    ....
}

I’ve tried a few options for my

schema.json

file for this tap, but no luck. Any clues?

saadat_qadri

04/05/2021, 4:53 PM

I have a feeling this has to do with the $id param in the JSON spec, but this attempt didn’t work:

saadat_qadri

04/05/2021, 4:54 PM

Copy code

"data": {
            "type": "object",
            "properties": {
                "$id": {
                    "id": {
                        "type": "integer"
                    },
                    "type": {
                        "type": "string"
                    },
                    "question": {
                        "type": "string"
                    },
                    "section_id": {
                        "type": "integer"
                    },
                    "shown": {
                        "type": "boolean"
                    }
                }
            }
        }

aaronsteers

04/05/2021, 6:38 PM

This is an interesting question! Do you intend to treat each sequential entry as a separate row? If so, it might make more sense to process each entry as it’s own record, and then your json schema would only need to describe the inner object.

aaronsteers

04/05/2021, 6:41 PM

That doesn’t answer your question so much as perhaps works around it. 🙂 Another option might be to try detecting with this tool: JSON Schema Tool

juan_sebastian_suarez_valencia

04/05/2021, 7:15 PM

OMG @aaronsteers thank you ! I was looking for a tool like this for days

saadat_qadri

04/05/2021, 7:46 PM

@aaronsteers thanks for the tip. I was going to try treating each entry as a separate row, but I need all the metadata around the object as well. Looks like JSONschema supports something like

patternProperties

for this usecase.

aaronsteers

04/05/2021, 9:20 PM

@juan_sebastian_suarez_valencia and @saadat_qadri - I’m glad this is helpful for you! I’ve been very impressed with this site myself. There’s just one point I want to call out, which is that for compatibility reasons most Singer components use JSON Schema Draft 4. To get that behavior in the jsonschema.net site, you’ll want to use the gear icon on the left pane:

Open in Slack

Previous Next