struggling to get `target-parquet` to work after l...
# troubleshooting
a
struggling to get
target-parquet
to work after loading my csv files and not sure why. It's working fine with
meltano run tap-spreadsheets-anywhere target-jsonl
but
meltano run tap-spreadsheets-anywhere target-parquet
fails
meltano.yml
Copy code
version: 1
default_environment: dev
project_id: 3fa34653-d5ae-4b33-8f72-7480e702a3be
environments:
- name: dev
- name: staging
- name: prod
hub_url: <https://deploy-preview-1158--meltano-hub.netlify.app>
plugins:
  extractors:
  - name: tap-spreadsheets-anywhere
    variant: ets
    pip_url: git+<https://github.com/ets/tap-spreadsheets-anywhere.git>
    config:
      tables:
      - path: file://./data
        name: orders
        start_date: '2001-01-01T00:00:00Z'
        key_properties: [id]
        pattern: orders.csv
        format: csv
      - path: file://./data
        name: reviews
        start_date: '2001-01-01T00:00:00Z'
        key_properties: [id]
        pattern: reviews.csv
        format: csv
  loaders:
  - name: target-parquet
    variant: estrategiahq
    pip_url: git+<https://github.com/estrategiahq/target-parquet.git>
    config:
      destination_path: ./data/parquet
      compression_method: snappy
      streams_in_separate_folder: true
      add_record_metadata: true
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
    config:
      destination_path: ./data/json
Not finding the trace super useful
e
a
okay cool. I've managed to get this working using csv files directly
rather than dumping csvs to parquet, just reading the csv files into dbt
a
I experienced the same issue, couldn't find a way around it.