I ve having an issue when setting up `tap shopify` to use ba Meltano #troubleshooting

I've having an issue when setting up `tap-shopify`...

omar_salama

09/25/2023, 3:06 PM

I've having an issue when setting up

tap-shopify

to use batch configuration, where I keep getting the error

No RECORD message received

I turned on debug logs and I can see that I am getting a

BATCH

message, which contains the actual data I do expect to receive from the tap, but there are indeed no

RECORD

messages. If I edit the

meltano.yml

to remove all batch config then it works fine these are the removed config lines that are causing the problem:

Copy code

batch_config:
        encoding:
          format: jsonl
          compression: gzip
        storage:
          root: <file://output>
          prefix: batchtest

Would this be a core Meltano issue or

tap-shopify

issue?

omar_salama

09/25/2023, 6:24 PM

after playing around a bit more, it seems like the

tap-shopify--matatika

default variant has a bunch of schema objects defined with null type, but switching to

tap-shopify--singer-io/

made everything start working right away, with much simpler configuration items.

omar_salama

09/25/2023, 6:35 PM

full list of the null types I'm seeing in Matatika schema, in case someone knows how to get them updated: These are all under the

abandoned_checkouts

stream •

billing_address.company

•

closed_at

•

completed_at

•

customer.note

•

note

•

shipping_address.company

omar_salama

09/25/2023, 6:37 PM

also, I see the schema definition for

transactions

stream is repeated 23 times in the tap output, when I run

meltano invoke tap-shopify > output.json

omar_salama

09/25/2023, 7:37 PM

these nulls aside, there are a huge number of differences between the singer-io schemas and the matatika schemas. I tried working through all the errors but I think it'll be easier to just take the singer schema directly...

aaron_phethean

10/02/2023, 9:41 AM

Hi @omar_salama we're working on a release this week that will hopefully resolve the majority of the schema issues people are seeing. @Reuben (Matatika)

Reuben (Matatika)

10/06/2023, 8:29 AM

Hi @omar_salama, we released

v0.2.0

tap-shopify

yesterday, which addresses a load of schema issues - including the ones you have raised here for

abandoned_checkouts

Copy code

meltano lock --update tap-shopify
meltano install --clean extractor tap-shopify

Open in Slack

Previous Next