alexandre_brehelin
03/08/2022, 9:13 PMjsonschema.exceptions.ValidationError: 48.867082 is not a multiple of 1e-06.
This issue seems to appear during the extractor step. In addition the previous error, the extractor raise error as below:
Failed validating 'multipleOf' in schema['properties']['lat']:
2022-03-08T18:31:59.748597Z [info ] {'inclusion': 'available',
2022-03-08T18:31:59.748733Z [info ] 'multipleOf': 1e-06,
2022-03-08T18:31:59.748840Z [info ] 'type': ['null', 'number']}
2022-03-08T18:31:59.748960Z [info ]
2022-03-08T18:31:59.831977Z [info ] On instance['lat']:
2022-03-08T18:31:59.833501Z [info ] 48.867082
We try to resolve it with schema setting to override the catalog schema from tap-mysql.
meltano config tap-mysql set _schema some_stream_id lat multipleOf 1e-08
meltano config tap-mysql set _metadata some_stream_id lat multipleOf 1e-08
But, any previous command resolve the issue. Have you ever encountered this problem?
This issue seems already appear on other target like target-csv (https://gitlab.com/meltano/target-csv/-/issues/3)alexandre_brehelin
03/09/2022, 8:48 AMalexandre_brehelin
03/09/2022, 9:44 AMschema = utils.float_to_decimal(self.schema)
alexandre_brehelin
03/09/2022, 9:44 AMalexandre_brehelin
03/09/2022, 9:44 AMedgar_ramirez_mondragon
03/09/2022, 4:18 PM>>> import jsonschema
>>> jsonschema.validate(48.867082, {"type": "number", "multipleOf": 0.000001})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/edgarramirez/Library/Caches/pypoetry/virtualenvs/singer-sdk-S0flctL8-py3.9/lib/python3.9/site-packages/jsonschema/validators.py", line 934, in validate
raise error
jsonschema.exceptions.ValidationError: 48.867082 is not a multiple of 1e-06
Failed validating 'multipleOf' in schema:
{'multipleOf': 1e-06, 'type': 'number'}
On instance:
48.867082
>>> import decimal
>>> jsonschema.validate(decimal.Decimal("48.867082"), {"type": "number", "multipleOf": decimal.Decimal("0.000001")})
I think we can do that in the SDK so MeltanoLabs/target-athena
and MeltanoLabs/target-csv
would inherit that behavior.
I popped https://gitlab.com/meltano/sdk/-/issues/346 to discuss the solution.
@alexandre_brehelin can you leave a 👍 if you think that'll solve your issue or a comment to further explain the problem you're having?alexandre_brehelin
03/09/2022, 10:31 PMalexandre_brehelin
03/09/2022, 10:33 PMalexandre_brehelin
03/09/2022, 10:34 PMself._validator.validate(utils.float_to_decimal(record))
alexandre_brehelin
03/09/2022, 10:34 PMschema = utils.float_to_decimal(self.schema)
in process_batchalexandre_brehelin
03/09/2022, 10:38 PM