Hi does anyone has examples of a well tested tap o...
# getting-started
a
Hi does anyone has examples of a well tested tap or target? Using pytest or any library to test the streams end to end ?
v
We are constantly working on what "well tested" means. It's much easier on the tap side then the target but the range of potential inputs is hard to test. I've written about 50 taps and targets now but it ranges pretty hard how well I test everything. https://github.com/MeltanoLabs/target-postgres has a lot of tests ( uses pytest and docker compose) but it's not using all the suite test features the sdk has. https://github.com/MeltanoLabs/target-snowflake written by the meltano team is probably much more "standard" I haven't dove too deep yet! Tap wise https://github.com/MeltanoLabs/tap-postgres and https://github.com/AutoIDM/tap-clickup/blob/b90324eb07eb9e2ed9a27b7a5672f731da6bf554/.github/workflows/ci.yml#L60 has some full end to end testing as I wanted it due to some schema issues I had hit earlier. Hope that helps
u
Derek gave some great suggestions - target-snowflake is where we spent a lot of time implementing the new test framework, that feels pretty robust. I also worked on https://github.com/MeltanoLabs/tap-cloudwatch/tree/main/tap_cloudwatch/tests where I had to write a bunch of very precise unit tests related to how the API is accessed and with what parameters as it paginates, might be somewhat helpful for more obscure custom APIs.
a
Thank you very much guys I have written many taps and targets too. I am an Integrations Engineer at hotglue. I will take a look closely to the repositories you recommended me!