daniel_luftspring
11/07/2021, 7:11 PMmeltano elt <tap> <target>
that are pretty hard to figure out. The error message is claiming that a data type is misspecified but it seems to be mixing up the endpoints.
tap-hellobaton | extractor | time=2021-11-07 14:05:44 name=tap-hellobaton level=INFO message=Beginning full_table sync of 'tasks'...
tap-hellobaton | extractor | time=2021-11-07 14:05:44 name=tap-hellobaton level=INFO message=Tap has custom mapper. Using 1 provided map(s).
target-snowflake | loader | CRITICAL '12345.00' is not of type 'integer'
target-snowflake | loader | CRITICAL
target-snowflake | loader | CRITICAL Failed validating 'type' in schema['properties']['annual_contract_value']:
target-snowflake | loader | CRITICAL {'type': 'integer'}
target-snowflake | loader | CRITICAL
target-snowflake | loader | CRITICAL On instance['annual_contract_value']:
target-snowflake | loader | CRITICAL '12345.00'
tap-hellobaton | extractor | time=2021-11-07 14:05:44 name=tap-hellobaton level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 0.741282, 'tags': {'endpoint': '/tasks/', 'http_status_code': 200, 'status': 'succeeded'}}
meltano | elt | [Errno 32] Broken pipe
The endpoint tasks
doesn't have that key listed anywhere in the JSON schema. Weirder still in the json schemas for endpoints that do have the property annual_contract_value
they are all type=string
daniel_luftspring
11/07/2021, 7:14 PMvisch
11/08/2021, 1:19 AMtarget-snowflake | loader | CRITICAL Failed validating 'type' in schema['properties']['annual_contract_value']:
Was a bit more helpful.
The issue here is that the tap and the target are not attached. tap | target
So annual_contract_value
is coming from a different stream would be my best guess. It just happens that the tap was sending data for a task at the same time.
It'd be cool to think about a more generic way to debug these kinds of things easiervisch
11/08/2021, 1:20 AMdaniel_luftspring
11/08/2021, 1:05 PMvisch
11/08/2021, 1:35 PM