I’m using pipelinewise tap-mysql and adswerve target-bigquery, and I’ve found a pretty serious bug in each. Has there been any thinking/work done on Singer spec conformance testing?

tap-mysql

turns out to not properly implement FULL_TABLE sync, and

target-bigquery

ignores

STATE

messages that do a

singer.statediff.Remove

state operation. (The combination together causes a catastrophic state corruption that breaks the state and requires manual state patching to sync new rows on a per-table basis). I wonder if there is a generic test harness that we can come up with to demonstrate conformance? Or perhaps the

singer

/

meltano

SDKs can offer some stub test suites for taps/targets that drive some common test data? For example if we had some target conformance tests simulating

tap-csv => your-target

(leaving the target’s tests to define what is the correct end-state in the loaded dataset) then some of the edge-cases could start to be fleshed out.

Do you have a link to the issue with tap-mysql? I'd like to know what it is so I can make sure to avoid it. 😉

@thomas_briggs https://github.com/transferwise/pipelinewise-tap-mysql/issues/91

Got it. Thanks. Thankfully we don't have many (any?) auto-incrementing PKs. 😉