First problem Your input requires a SCHEMA message before th Meltano #singer-target-development

Join Slack

First problem. Your input requires a SCHEMA messag...

# singer-target-development

alexander_butler

04/17/2023, 2:41 PM

First problem. Your input requires a SCHEMA message before the record message to be a valid singer stream.

michael_lan

04/19/2023, 3:27 PM

I am passing the schema in the config file, Do I move it to input file?

alexander_butler

04/19/2023, 5:07 PM

A valid singer stream requires a SCHEMA message. If you are using

--input

, my understanding is it is basically feeding that as-is into the targets stdin. So I am just pointing out that your input file is not a valid singer stream which can cause the error you are seeing. Lastly, you dont pass the

schema

extra to a target AFAIK, that goes to a tap.

alexander_butler

04/19/2023, 5:09 PM

I recommend getting a real singer dump...

alexander_butler

04/19/2023, 5:10 PM

You can use some random one I made for a test:

Copy code

{"type": "SCHEMA", "stream": "{stream_name}", "schema": {"properties": {"id": {"type": ["integer", "null"]}, "rep_key": {"type": ["integer", "null"]}}, "type": "object"}, "key_properties": ["id"], "bookmark_properties": ["rep_key"]}
{"type": "RECORD", "stream": "{stream_name}", "record": {"id": 0, "rep_key": 0}, "time_extracted": "2022-11-11T20:40:56.806676+00:00"}
{"type": "STATE", "value": {"bookmarks": {"{stream_name}": {"starting_replication_value": null, "replication_key": "rep_key", "replication_key_value": 0}}}}
{"type": "RECORD", "stream": "{stream_name}", "record": {"id": 1, "rep_key": 1}, "time_extracted": "2022-11-11T20:40:56.807284+00:00"}
{"type": "RECORD", "stream": "{stream_name}", "record": {"id": 2, "rep_key": 2}, "time_extracted": "2022-11-11T20:40:56.807796+00:00"}
{"type": "RECORD", "stream": "{stream_name}", "record": {"id": 3, "rep_key": 3}, "time_extracted": "2022-11-11T20:40:56.808289+00:00"}
{"type": "RECORD", "stream": "{stream_name}", "record": {"id": 4, "rep_key": 4}, "time_extracted": "2022-11-11T20:40:56.808782+00:00"}

alexander_butler

04/19/2023, 5:15 PM

Oh and last thing! 😄

Copy code

{
  "streams": [
    {
      "tap_stream_id": "target-one",
      "replication_method": "FULL_TABLE",
      "key_properties": [
        "id"
      ],
      "schema": {...
      },
      "stream": "target-one",
      "metadata": [...
      ]
    }
  ]
}

This is not a config, this is called a catalog and has nothing to do with a target btw

michael_lan

04/26/2023, 6:35 PM

Thank you very much, I have finally been able to do it, thanks to you.

alexander_butler

04/26/2023, 6:35 PM

Yay! 🚀

Open in Slack

Previous Next