Hi all, I keep getting an error when using tap-my...
# troubleshooting
a
Hi all, I keep getting an error when using tap-mysql and target-bigquery to sync a particular column. The schema validation fails when it shouldn’t while doing “multipleOf”: 0.01 checks. I think it’s probably some Python rounding issue? The checks get put in there because the MySQL column type is decimal(10, 2).
jsonschema.exceptions.ValidationError: 34.23 is not a multiple of 0.01
. How do I properly get rid of this constraint? I don’t really need the data validation. I have dumped the catalog and removed this constraint, then tried passing the --catalog flag every which way. I do get a
[info     ] Found catalog in /projects/catalog.json
log line right at the start of the run but then I still get log lines looking like
[info     ] INFO:target-bigquery:Received schema for <table name>: {'properties': {'<column name>': {'inclusion': 'available', 'multipleOf': 0.01, …
showing that the constraint has been added back. What’s the right way to pass the catalog into
el
or
run
or otherwise specify the schema?
If I go into the .meltano/run directory I can see
tap.properties.json
files that do have the constraint still in there which I’m guessing is what the run is looking at. But those seem to be dynamically generated for each run. I just have no idea how to override how that’s being generated.
e
This is unfortunately a known issue with some (mostly legacy, I think?) connectors: https://github.com/python-jsonschema/jsonschema/issues/701. You could probably use https://docs.meltano.com/concepts/plugins/#schema-extra and continue relying on the auto-generated catalog, which would be patched:
Copy code
extractors:
- name: tap-mysql
  schema:
    <stream>:
      <field>:
        type: [your types here]
        multipleOf: null