mathew_fournier
06/02/2023, 11:11 PMget_records
that is missing that field there is no error, not even a warning. TYPE_CONFORMANCE_LEVEL
seems to only check for extra keys. But it doesn’t validate a message against the actual schema as far as I can tell in _typing.py
. Is message -> schema validation left as an implementation detail for the implementors of a stream/tap?visch
06/02/2023, 11:22 PMmathew_fournier
06/02/2023, 11:23 PMschema = th.PropertiesList(
th.Property("id", th.IntegerType, required=True),
th.Property("name", th.StringType, required=True),
th.Property("company", th.StringType),
th.Property("offset", th.IntegerType),
th.Property("test", th.StringType)
).to_dict()
mathew_fournier
06/02/2023, 11:24 PMget_records
in a class that subclasses Stream
that don’t have the id
or name
fields. E.g. , required
is not being enforced by anything in the SDK within core.py
mathew_fournier
06/02/2023, 11:26 PMmathew_fournier
06/02/2023, 11:26 PMmathew_fournier
06/02/2023, 11:26 PMmathew_fournier
06/02/2023, 11:27 PMid
doesn’t match, and it is missing the name
field.mathew_fournier
06/02/2023, 11:27 PMedgar_ramirez_mondragon
06/03/2023, 12:15 PMIs message -> schema validation left as an implementation detail for the implementors of a stream/tap?
Not really. The SDK doesn't expect schema validation at runtime in the tap, but the testing framework recently got schema validation: https://github.com/meltano/sdk/pull/1711
visch
06/03/2023, 1:35 PMmathew_fournier
06/05/2023, 4:06 AMvisch
06/05/2023, 12:23 PMmathew_fournier
06/05/2023, 2:58 PM