I'm trying to create workaround for a bug in the codebase by modifying the attribute in the meltano.yml file. Bug here https://github.com/singer-io/tap-shopify/blob/master/tap_shopify/schemas/orders.json#L9-L14 Recent attempts attached. I tried a csv loader for comparison, it ran fine. Any suggestions? I would be very grateful!

@royzac there may still be other fields like those: https://github.com/singer-io/tap-shopify/blob/master/tap_shopify/schemas/orders.json#L72-L73. Your second screenshot has the correct structure, Meltano adds

properties

in between for you. Can you invoke the tap with debugging turned on:

meltano --log-level=debug invoke tap-shopify

You should notice something like

[2021-09-08 09:54:37,297] [3209|MainThread|root] [DEBUG] Visiting CatalogNode.PROPERTY at '.streams[1].schema.properties.key'.
[2021-09-08 09:54:37,297] [3209|MainThread|root] [DEBUG] Setting '.streams[1].schema.properties.key' to {'type': 'string'}

And in the actual tap output, schema messages should have the changed properties:

{"type": "SCHEMA", "stream": "ledger", "schema": {"properties": {..., "key": {"type": "string"}}, "type": "object"}, "key_properties": ["id"], "bookmark_properties": []}

Hi Edgar, this makes sense but for some reason no modification seems to be occuring based on the debug log. My yml schema is attached and I have DM you my log info.

ah, you have extra double quotes at the end of field names e.g.

total_shipping_price_set"

🤦

Yup. Seems like the target is defaulting to gzip. You can override in

meltano.yml

plugins:
  loaders:
    name: target-athena
    settings:
      ...
      - name: compression
    config:
      compression: null

Curious, what might be the reason for the default zip?

Save a few pennies on storage probably 😅

hah, I see. Thanks Edgar!

np!

@edgar_ramirez_mondragon what is the functional difference between target-s3 and target-athena if any?

@royzac target-athena registers and updates tables in the Athena catalog

Thanks!