So I'm taking a whack at creating a MySQL target using the Meltano SDK. That's probably a bad idea because I don't really know Python, but hey. 😛 I used the SQLite target as a starting point, and got to the point where the target starts and connects to the database. When I feed it some output generated by a tap, however, it immediately fails because the destination table doesn't exist in the database. The processing of the activate_version record is trying to do a DELETE from the table because hard_delete defaults to true, but the table isn't there. I expected the processing of the schema record to create the table, but it does not. I don't know if that's a bug or an incorrect expectation on my part though. I get the same result with the latest version of SQLite target too (the port-to-sdk branch), so at the very least I don't think the problem is my code. Any suggestions?

That definitely sounds like a bug-- those events should be creating the table if it doesn't exist. I recently developed a target for DuckDB using the postgres target from pipelinewise as inspiration- maybe try that as a basis instead of the sqlite one (which is IIRC very old): https://github.com/jwills/target-duckdb/tree/main/target_duckdb

https://github.com/siilats/siilats-target-mysql might be a good starting point as it uses the SDK

Thanks gents. I'm specifically trying to leverage the SDK, which I don't think the PW targets use. 😕 I didn't realize the ODBC one used the SDK; I've learned some things from looking at that. 👍🏻

in most SQL dialects you'd do something like IF EXISTS DELETE x 🤷

Yeah, the more I think about it the more I think that's just a bug in the SDK... it just isn't safe to assume the table exists

I'd argue it's up to the implementation to figure that out but 🤷 Something like if (does table exist) else noop

If I were doing something to handle that record I'd agree with you. As it is though the SDK is just magically handling it itself with no involvement (or even intent) on my part, so... it's gonna not fail if it's gonna do that (IMO anyway 😉 )