Hello! I'm trying to get some data from google she...
# troubleshooting
i
Hello! I'm trying to get some data from google sheets into postgres, i'm using tag-google-sheets but it's not picking up the types correctly (it seem to parse everything as string). No big deal, I'm trying to work around that with inline mappers and it's all good if i make a new field/column for the mapped value, but if i do try to map it to the same field I get a validation error:
Copy code
Failed validating 'type' in schema['properties']['amount']:  {'type': ['string', 'null']} On instance['amount']:Decimal('500.0')
This is my config, I also tried to manually override the schema:
Copy code
extractors:
  - name: tap-google-sheets
    variant: matatika
    pip_url: git+<https://github.com/Matatika/tap-google-sheets.git>
    config:
      child_sheet_name: ''
      flattening_enabled: false
      flattening_max_depth: 5
      stream_maps:
        finance:
          amount: float(record['amount'])
    schema:
      amount: 
        type: ["float","null"]
If i just call invoke on tap-google-sheets i get this in the logs:
Copy code
2023-09-16 20:24:32,070 | INFO     | tap-google-sheets    | Tap has custom mapper. Using 1 provided map(s).
{"type": "SCHEMA", "stream": "finance", "schema": {"properties": {"date": {"type": ["string", "null"]}, "what": {"type": ["string", "null"]}, "amount": {"type": ["number", "null"]}, "account": {"type": ["string", "null"]}, "category": {"type": ["string", "null"]}, "one_time": {"type": ["string", "null"]}}, "type": "object"}, "key_properties": []}
which has the correct type on amount, but then a few lines below it seems to use the auto generated schema:
Copy code
`{"type": "SCHEMA", "stream": "finance", "schema": {"type": "object", "properties": {"date": {"type": ["string", "null"]}, "what": {"type": ["string", "null"]}, "amount": {"type": ["string", "null"]}, "account": {"type": ["string", "null"]}, "category": {"type": ["string", "null"]}, "one_time": {"type": ["string", "null"]}}}, "key_properties": []}
Does anyone know what I'm doing wrong? Thanks in advance
a
No sure to be honest. Looks like it is some strange behaviour because of the mapper.