Hello #C069CQNHDNF @Edgar Ramírez (Arch.dev) I am stuck while extracting data from extractor (Magento) Below one is my configuration extractors: - name: tap-magento variant: hotgluexyz pip_url: git+https://gitlab.com/hotglue/tap-magento.git config: store_url: https://eshop-stage.simform.solutions/ username: admin start_date: '2024-01-01' flattening_enabled: true flattening_max_depth: 2 select: - carts*.* # All fields from carts - inventory_stocks*.* # All fields from inventory_stocks - store_views*.* - products*.* # All fields from store_views - stores*.* # All fields from customer_address_entity - users*.* # All fields from customer_log - orders*.* # All fields from customer_visitor - order_items*.* - credit_memos*.* metadata: "'*'": replication-method: FULL_TABLE Here I am not able see any data for products, orders(it shows empty table) and in carts table I am able to see data but for customer column I have json and configured flatting enabled but it is not working properly so can you guide me here my destination is clickhouse loaders: - name: target-clickhouse variant: shaped-ai pip_url: git+https://github.com/shaped-ai/target-clickhouse.git config: database: customer_raw_data username: default host: localhost port: 8123 batch_size_rows: 1000 optimize_after: false secure: false verify: true

Here I am not able see any data for products, orders(it shows empty table)

You can confirm your selections work as expected by looking at the output from

meltano select tap-magento --list --all

in carts table I am able to see data but for customer column I have json and configured flatting enabled but it is not working properly so can you guide me here

can you run

meltano invoke tap-magento --discover > catalog.json

and look at the contents of

catalog.json

. We're looking for

streams[].schema.properties.customer

to see what's its

type

.

Here I have json data in customer column but I want to extract it like tabular data so how I can do? I tried flattening_enabled: true flattening_max_depth: 1

I see what's happening. The

customer

field doesn't have any details to its properties in the schema: https://gitlab.com/hotglue/tap-magento/-/blob/d6cbf5f951d8e25fb8487409f432a00917c78e74/tap_magento/streams.py#L706 One approach that might work is overriding the schema for it with the fields you want to flatten out:

extractors:
- name: tap-magento
  schema:
    carts:
      customer:
        type: ["object"]
        properties:
          id:
            type: integer
          group_id:
            type: integer
          ...

https://docs.meltano.com/concepts/plugins/#schema-extra