https://linen.dev logo
Join Slack
Powered by
hey :hello: I am getting this error whiel running...
# getting-started
k
hey hello I am getting this error whiel running meltano elt fron hellobaton to snowflake:
Copy code
2022-09-14T16:21:06.172983Z [info     ] CRITICAL None is not of type 'string' cmd_type=loader job_id=baton-to-snowflake name=target-snowflake run_id=f9931695-689e-46a3-aeab-5da042eb1146 stdio=stderr
2022-09-14T16:21:06.173162Z [info     ] CRITICAL                       cmd_type=loader job_id=baton-to-snowflake name=target-snowflake run_id=f9931695-689e-46a3-aeab-5da042eb1146 stdio=stderr
2022-09-14T16:21:06.173312Z [info     ] CRITICAL Failed validating 'type' in schema['properties']['created_by']: cmd_type=loader job_id=baton-to-snowflake name=target-snowflake run_id=f9931695-689e-46a3-aeab-5da042eb1146 stdio=stderr
2022-09-14T16:21:06.173496Z [info     ] CRITICAL     {'type': 'string'} cmd_type=loader job_id=baton-to-snowflake name=target-snowflake run_id=f9931695-689e-46a3-aeab-5da042eb1146 stdio=stderr
2022-09-14T16:21:06.173644Z [info     ] CRITICAL                       cmd_type=loader job_id=baton-to-snowflake name=target-snowflake run_id=f9931695-689e-46a3-aeab-5da042eb1146 stdio=stderr
2022-09-14T16:21:06.173783Z [info     ] CRITICAL On instance['created_by']: cmd_type=loader job_id=baton-to-snowflake name=target-snowflake run_id=f9931695-689e-46a3-aeab-5da042eb1146 stdio=stderr
2022-09-14T16:21:06.173920Z [info     ] CRITICAL     None              cmd_type=loader job_id=baton-to-snowflake name=target-snowflake run_id=f9931695-689e-46a3-aeab-5da042eb1146 stdio=stderr
looks like its validating a data type. Is this an issue that needs to be resolved by the code of the tap? or is there something we can do in meltano that can override this
e
Hi @kevin. You should be able to override the schema for 
created_by
. It’s not clear which of the streams is having this problem, but seems to be one of 
project_attachments
, 
time_entries
but you can use the 
schema
extra for both:
Copy code
extractors:
- name: tap-hellobaton
  schema:
    project_attachments:
      created_by:
        type: ["string", "null"]
    time_entries:
      created_by:
        type: ["string", "null"]
k
are we subbing the extra schema right into the 
meltano.yml
file ?
e
yup
k
got it thanks @edgar_ramirez_mondragon! Will let you know if i have any further questions
im now getting:
Copy code
2022-09-14T18:39:52.981838Z [info     ] CRITICAL Failed validating 'type' in schema['properties']['user']: cmd_type=loader job_id=baton-to-snowflake name=target-snowflake run_id=cf33e0b3-98bb-4fe3-8da9-3205f6cbf6f2 stdio=stderr
would i just add in the user field ?
Copy code
plugins:
  extractors:
  - name: tap-hellobaton
    schema:
     project_attachments:
       created_by:
         type: ["string", "null"]
     time_entries:
       created_by:
         type: ["string", "null"]
       extractors:
  - name: tap-hellobaton
    schema:
     project_attachments:
       created_by:
         type: ["string", "null"]
     time_entries:
       created_by:
         type: ["string", "null"]
     project_users:
       user:
         type: ["string", "null"]
     time_entries:
       user:
         type: ["string", "null"]
all good solved it!
e
@kevin just to confirm, was the solution to also override the schema for 
time_entries.users
?
k
yup!
hey @edgar_ramirez_mondragon just following up on this, if i wanted to override the types in (https://github.com/dluftspring/tap-hellobaton/blob/5dcf53f0daba6f40c2468dd2d3ccbfd9d26951dd/tap_hellobaton/schemas/time_entries.json#:~:tex[…]%20%7B)
Copy code
"rate": {
            "type": ["object", "null"],
            "properties": {
                "id": {
                    "type": "integer"
                },
                "hourly_rate": {
                    "type": "string"
                }
            }
        },
would the part of the meltano.yml look like:
Copy code
schema:
      time_entries:
        rate:
          id:
            type: [integer, 'null']
          hourly_rate:
            type: [string, 'null']
I think im messing up the yml part.
e
Can you take a look at the produced catalog from that config? You may need to need to add the intermediate 
properties
key inside 
rate
Open in Slack
PreviousNext
Powered by