Hi everyone 👋 I'm new to Meltano .. and I'd like to configure it to extract from Mysql server using ssh / pem file .. having hard time to find a proper samples/documentation .. any reference or sample is greatly appreciated 🙂 Note: tried to connect to a local mysql server (just to discover the extractor (tap-mysql default one) but getting this error also:

meltano --environment=staging config tap-mysql test
2023-06-26T16:01:08.934830Z [info     ] Environment 'staging' is active
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
Exception: only INCREMENTAL, LOG_BASED, and FULL TABLE replication methods are supported

@bassel we have an open issue on the hub to document this piece a bit better especially when configuration is needed before the tap will work https://github.com/meltano/hub/issues/1366. You need to set

replication-method

settings metadata to

FULL_TABLE

or one of the other options

@pat_nadolny Curious about your thoughts on this: should extractors like

tap-mssql

set default

metadata

to avoid problems like this? I recognise that assuming

FULL_TABLE

is acceptable for all use-cases is an issue in of itself, but surely it's got to be better than throwing an error? Either way, that issue proposal would be useful. New related thread for context: https://meltano.slack.com/archives/CMN8HELB0/p1688130547416099

@Reuben (Matatika) I somehow missed this message but got linked back from https://github.com/meltano/hub/pull/1427#issuecomment-1634522333. Yeah to me the downside of full table is that its not optimized but functioning in a non-optimized way seems better than an error. It would be up to the user to configure incremental syncing and their replication keys. I'll create a PR for tap-mssql too