Hey everyone - I’m trying to debug the

tap-workramp

which is erroring:

singer.transform.SchemaMismatch: Errors during transform cmd_type=extractor name=tap-workramp run_id=d7a12536-09fd-4732-bb26-fc6f1a2130c6 state_id=2022-11-11T144832--tap-workramp--target-bigquery stdio=stderr

2022-11-11T15:04:55.060283Z [info     ] 	certifications.0.id: data does not match {'type': ['integer', 'null']} cmd_type=extractor name=tap-workramp run_id=d7a12536-09fd-4732-bb26-fc6f1a2130c6 state_id=2022-11-11T144832--tap-workramp--target-bigquery stdio=stderr

2022-11-11T15:04:55.060354Z [info     ] 	certifications.0: data does not match {'properties': {'id': {'type': ['integer', 'null']}, 'title': {'type': ['string', 'null']}}, 'type': ['object', 'null'], 'additionalProperties': False} cmd_type=extractor name=tap-workramp run_id=d7a12536-09fd-4732-bb26-fc6f1a2130c6 state_id=2022-11-11T144832--tap-workramp--target-bigquery stdio=stderr
2022-11-11T15:04:55.060421Z [info     ] 	certifications.1.id: data does not match {'type': ['integer', 'null']} cmd_type=extractor name=tap-workramp run_id=d7a12536-09fd-4732-bb26-fc6f1a2130c6 state_id=2022-11-11T144832--tap-workramp--target-bigquery stdio=stderr

I’m not totally clear on what my options are for: • avoiding this issue/issues like it or • skipping the error I’m running:

meltano --log-level=debug elt tap-workramp target-bigquery

So no transform. My meltano.yml is:

version: 1
default_environment: dev
project_id: xxxxxxxxxx
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-workramp
    variant: pathlight
    pip_url: git+<https://github.com/Pathlight/tap-workramp.git>
    config:
      access_token: xxxxxxxxxx {<--- not ideal but just a one time proof of concept!}
      start_date: '2021-01-01'
  loaders:
  - name: target-bigquery
    variant: adswerve
    pip_url: git+<https://github.com/adswerve/target-bigquery.git@0.12.0>
    config:
      credentials_path: xxxxxxxxxxxx.json
      project_id: xxxxxx
      dataset_id: meltano_local_source
      location: us-central1
      add_metadata_columns: true
      replication_method: append
      table_prefix: _source_
  transformers:
  - name: dbt-bigquery
    variant: dbt-labs
    pip_url: dbt-core~=1.3.0 dbt-bigquery~=1.3.0

Hi @stephen_hazeltine! It looks like some of the

users

records have

certifications

arrays that don’t match the expected schema: https://github.com/Pathlight/tap-workramp/blob/16f685e0f7c5b51ec88c382f420ffa4bc05a2620/tap_workramp/schemas/users.json#L82-L102 This drift is unfortunately more or less common in APIs, but their docs don’t reflect any changes to the schema: https://developers.workramp.com/reference/get-all-users-in-enterprise Can you test the endpoint using the link above and confirm what the

certifications

array look like?

@edgar_ramirez_mondragon thanks for the help! They all look like this:

{
        "id": "11111f71-1111-11ff-1111-11f11f11f1ff",
        "title": "[[[string]]],
        "isDeleted": false,
        "createdAt": 1668030468509,
        "updatedAt": 1668030468509
    },

Yeah so it looks like the certification ID is not really an integer as suggested by the docs