For

tap-airbyte-wrapper

, does it work for Airbyte sources which are not written in python? I'm specifically looking at https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-mssql but I am getting this error: "tap_airbyte.tap.AirbyteException: Discover failed with return code 1: Unknown option: discover"

Any particular reason you want to use the airbyte connector? We added xml column type support to the pipeline wise variant and it works for our customers https://github.com/Matatika/pipelinewise-tap-mssql

@aaron_phethean I was trying to test the performance between the two actually! We are using the pipelinewise variant currently and it works, but I want to see if the Airbyte variant is more performant, especially when doing full syncs.

iirc it runs airbytes taps in a container so yeah it should work

I’d start with the network if you’re looking at the performance. (including size of the each ‘batch’) Let us know what you find out 👊

@visch I did a bit more digging and it looks like native sources work (ones available on pypi and that do not require docker).

maybe an update to the shim?

@visch The shim in this case would be the airbyte wrapper -- right?

yes

@visch No worries, appreciate the feedback!

@Conner Panarella

tap-airbyte-wrapper

is broken due to a recent change, https://meltano.slack.com/archives/C068YBQQF1V/p1722005464454259?thread_ts=1722003940.255219&cid=C068YBQQF1V

@Conner Panarella were you able to get the airbyte sql server source working? Curious to know how or if you were able to set the cursors for incremental replication

@steven_wang Yes, although the maintainer hasn't merged any of the fixes I made a PR for 😅. What do you mean by setting the cursor?

Saw your PR! I removed the extra

--

in the docker command and got the tap to run. Thanks for that! By cursor I mean the replication key per stream used for incremental replication. I haven't figured out how to manually set it yet, seems like it defaults to the PK of each table I'm syncing

Gotcha, I use CDC so I haven't tried cursor-based replication