Asked this in the singer slack but I guess quite quiet there Meltano #random

Asked this in the singer slack, but I guess quite ...

matt_arderne

10/19/2021, 3:36 PM

Asked this in the singer slack, but I guess quite quiet there. Thinking about adding a singer tap to Stitch to save on having to use the Import API - any thoughts/suggestions on either adding the tap or using the Import API would be appreciated 😄 https://singer-io.slack.com/archives/C2TGFCZEV/p1634657457149400

aaron_phethean

10/19/2021, 4:26 PM

@visch said he conformed to the Stitch style guide the other day, but don't think this was to actually run the tap on Stitch. @matt_arderne is your goal that you want to have the tap hosted / run somewhere?

visch

10/19/2021, 4:42 PM

@aaron_phethean actually it was to run on the Stitch platform! Client I"m working with wanted support for stitch, so I ran through the steps stitch gave (still waiting on a response). Form is here https://docs.google.com/forms/d/e/1FAIpQLSdjuB3jrsuCoK5juLVvFzSPMLrxP5Wz_Xy1Fy5yVhP748bg8w/viewform (Link from https://www.singer.io/how-to-contribute/ ) Note that the SDK handles so much of this for you it's great. There is a piece with linting you have to add yourself, I baked their linting requirements into my tests for

tap-clickup

@matt_arderne running

target-stitch

with Meltano ie

meltano elt tap-clickup target-stitch

was chef kiss , worked perfect for us

matt_arderne

10/19/2021, 5:40 PM

@visch yeah it's exactly that form, though I suspect it will take too long and I'll end up running with

meltano elt

instead 🚀 🚀 Will have a look at your

tap-clickup

matt_arderne

10/20/2021, 9:53 AM

Hi @visch any chance you could help with the

meltano.yml

for the

target-stitch

? I keep getting

meltano       | Loading failed (1): Exception: Configuration is requires batch_size_preferences dictionary

Edit: Got it working with below config - somehow the

batch_size_preferences

wasnt registering as an object, so I manually added a

{'full_table_streams': []}

object and that seemed to work. This seems poorly documented on the Stitch side I think.

Copy code

config:
      client_id: 123
      small_batch_url: <https://api.stitchdata.com/v2/import/batch>
      big_batch_url: <https://api.stitchdata.com/v2/import/batch>
      batch_size_preferences:
        full_table_streams: []

visch

10/20/2021, 12:21 PM

haha yeah I did the same thing I'll find mine https://github.com/AutoIDM/tap-clickup/blob/main/meltano.yml#L35-L47

Copy code

config:
      client_id: 191229
      small_batch_url: <https://api.stitchdata.com/v2/import/batch>
      big_batch_url: <https://api.stitchdata.com/v2/import/batch>
      batch_size_preferences: 
        a : "a"

visch

10/20/2021, 12:23 PM

I just put in a filler for the same key

matt_arderne

10/20/2021, 1:11 PM

🙈 haha ok, opened an issue for what it's worth 😄 https://github.com/singer-io/target-stitch/issues/103

Open in Slack

Previous Next