hi, im using tap-csv to ingest to duckdb, but i go...
# singer-taps
a
hi, im using tap-csv to ingest to duckdb, but i got this error
2026-03-23T04:47:18.043389Z [info     ] target-duckdb _duckdb.InvalidInputException: Invalid Input Error: Error when sniffing file "C:\Users\Admin\AppData\Local\Temp\tmp_f39cebcc_3261_467b_8c3c_27ae768bd46d.csv".
full log
Copy code
meltano run tap-csv target-duckdb
2026-03-23T04:47:13.415371Z [info     ] meltano      Environment 'dev' is active   
2026-03-23T04:47:13.648897Z [info     ] meltano      Running job with name 'dev:tap-csv-to-target-duckdb' and run ID '019d1904-bab6-7053-bbc3-d0ad0a3f05d4'
2026-03-23T04:47:14.208155Z [info     ] tap-csv      tap-csv v1.2.0, Meltano SDK v0.53.6
2026-03-23T04:47:14.208155Z [info     ] tap-csv      Skipping parse of env var settings...
2026-03-23T04:47:14.208155Z [info     ] tap-csv      Beginning sync of 'users' in full_table mode
2026-03-23T04:47:16.911079Z [info     ] tap-csv      METRIC: {"type":"timer","metric":"sync_duration","value":2.702923536300659,"tags":{"stream":"users","pid":23216,"context":{},"status":"succeeded"}}
2026-03-23T04:47:16.911079Z [info     ] tap-csv      METRIC: {"type":"counter","metric":"record_count","value":99441,"tags":{"stream":"users","pid":23216,"context":{}}}
2026-03-23T04:47:18.043389Z [info     ] target-duckdb Traceback (most recent call last):
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "<frozen runpy>", line 198, in _run_module_as_main
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "<frozen runpy>", line 88, in _run_code
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "D:\Workspace\VS\Code\data-engineering-project\elt\.meltano\loaders\target-duckdb\venv\Scripts\target-duckdb.exe\__main__.py", line 10, in <module>
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "D:\Workspace\VS\Code\data-engineering-project\elt\.meltano\loaders\target-duckdb\venv\Lib\site-packages\target_duckdb\__init__.py", line 429, in main
2026-03-23T04:47:18.043389Z [info     ] target-duckdb     persist_lines(connection, config, singer_messages)
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "D:\Workspace\VS\Code\data-engineering-project\elt\.meltano\loaders\target-duckdb\venv\Lib\site-packages\target_duckdb\__init__.py", line 316, in persist_lines
2026-03-23T04:47:18.043389Z [info     ] target-duckdb     flushed_state = flush_streams(
2026-03-23T04:47:18.043389Z [info     ] target-duckdb                     ^^^^^^^^^^^^^^
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "D:\Workspace\VS\Code\data-engineering-project\elt\.meltano\loaders\target-duckdb\venv\Lib\site-packages\target_duckdb\__init__.py", line 353, in flush_streams
2026-03-23T04:47:18.043389Z [info     ] target-duckdb     load_stream_batch(        
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "D:\Workspace\VS\Code\data-engineering-project\elt\.meltano\loaders\target-duckdb\venv\Lib\site-packages\target_duckdb\__init__.py", line 394, in load_stream_batch
2026-03-23T04:47:18.043389Z [info     ] target-duckdb     flush_records(stream, records_to_load, row_count[stream], db_sync, temp_dir)
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "D:\Workspace\VS\Code\data-engineering-project\elt\.meltano\loaders\target-duckdb\venv\Lib\site-packages\target_duckdb\__init__.py", line 411, in flush_records
2026-03-23T04:47:18.043389Z [info     ] target-duckdb     db_sync.load_rows(records_to_load.values(), row_count, temp_dir)
2026-03-23T04:47:18.043389Z [info     ] target-duckdb   File "D:\Workspace\VS\Code\data-engineering-project\elt\.meltano\loaders\target-duckdb\venv\Lib\site-packages\target_duckdb\db_sync.py", line 393, in load_rows
2026-03-23T04:47:18.043389Z [info     ] target-duckdb     cur.execute(              
2026-03-23T04:47:18.043389Z [info     ] target-duckdb _duckdb.InvalidInputException: Invalid Input Error: Error when sniffing file "C:\Users\Admin\AppData\Local\Temp\tmp_f39cebcc_3261_467b_8c3c_27ae768bd46d.csv".
2026-03-23T04:47:18.043389Z [info     ] target-duckdb It was not possible to automatically detect the CSV parsing dialect
2026-03-23T04:47:18.043389Z [info     ] target-duckdb The search space used was:    
2026-03-23T04:47:18.043389Z [info     ] target-duckdb Delimiter Candidates: ',', '|', ';', '    '
2026-03-23T04:47:18.043389Z [info     ] target-duckdb Quote/Escape Candidates: ['(no quote)','(no escape)'],['"','(no escape)'],['"','"'],['"','''],['"','\'],[''','(no escape)'],[''','''],[''','"'],[''','\']
2026-03-23T04:47:18.043389Z [info     ] target-duckdb Comment Candidates: '\0', '#' 
2026-03-23T04:47:18.043389Z [info     ] target-duckdb Encoding: utf-8               
2026-03-23T04:47:18.043389Z [info     ] target-duckdb Possible fixes:               
2026-03-23T04:47:18.043389Z [info     ] target-duckdb * Disable the parser's strict mode (strict_mode=false) to allow reading rows that do not comply with the CSV standard.
2026-03-23T04:47:18.043389Z [info     ] target-duckdb * Columns are set as: "columns = { '_sdc_batched_at' : 'TIMESTAMP', '_sdc_deleted_at' : 'VARCHAR', '_sdc_extracted_at' : 'TIMESTAMP', '_sdc_source_file' : 'VARCHAR', '_sdc_source_file_mtime' : 'TIMESTAMP', '_sdc_source_lineno' : 'HUGEINT', 'customer_city' : 'VARCHAR', 'customer_state' : 'VARCHAR', 'customer_zip_code' : 'VARCHAR', 'user_name' : 'VARCHAR'}", and they contain: 10 columns. It does not match the number of columns found by the sniffer: 0. Verify the columns parameter is correctly set.
2026-03-23T04:47:18.043389Z [info     ] target-duckdb * Make sure you are using the correct file encoding. If not, set it (e.g., encoding = 'utf-16').
2026-03-23T04:47:18.043389Z [info     ] target-duckdb * Set delimiter (e.g., delim=',')
2026-03-23T04:47:18.051025Z [info     ] target-duckdb * Set quote (e.g., quote='"') 
2026-03-23T04:47:18.051025Z [info     ] target-duckdb * Set escape (e.g., escape='"')
2026-03-23T04:47:18.051025Z [info     ] target-duckdb * Set comment (e.g., comment='#')
2026-03-23T04:47:18.051025Z [info     ] target-duckdb * Set skip (skip=${n}) to skip ${n} lines at the top of the file
2026-03-23T04:47:18.051025Z [info     ] target-duckdb * Enable ignore errors (ignore_errors=true) to ignore potential errors
2026-03-23T04:47:18.051828Z [info     ] target-duckdb * Enable null padding (null_padding=true) to pad missing columns with NULL values
2026-03-23T04:47:18.052167Z [info     ] target-duckdb * Check you are using the correct file compression, otherwise set it (e.g., compression = 'zstd')
2026-03-23T04:47:18.052167Z [info     ] target-duckdb * Be sure that the maximum line size is set to an appropriate value, otherwise set it (e.g., max_line_size=10000000)
2026-03-23T04:47:18.121332Z [error    ] meltano      Loader failed                 
2026-03-23T04:47:18.121332Z [error    ] meltano      Block run completed           
2026-03-23T04:47:18.121332Z [info     ] meltano      Run completed
this is my sample data:
Copy code
user_name,customer_zip_code,customer_city,customer_state
861eff4711a542e4b93843c6dd7febb0,14409,KABUPATEN PEKALONGAN,JAWA TENGAH
290c77bc529b7ac935b93aa66c333dc3,9790,KOTA BEKASI,JAWA BARAT
pls help me, thanks
e
The following configuration works for me:
Copy code
# <https://meltano.slack.com/archives/C06A1MD6A6L/p1774241501529149>
default_environment: dev
environments:
- name: dev
- name: staging
- name: prod
send_anonymous_usage_stats: false
plugins:
  extractors:
  - name: tap-csv
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-csv.git>
    config:
      files:
      - entity: customers
        path: $MELTANO_PROJECT_ROOT/data.csv
        keys: [user_name]
  loaders:
  - name: target-duckdb
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/target-duckdb.git>
Copy code
$ duckdb output/warehouse.duckdb -c 'table tap_csv.customers;'
┌──────────────────────┬────────────────┬───────────────────┬──────────────────────────────────┐
│    customer_city     │ customer_state │ customer_zip_code │            user_name             │
│       varchar        │    varchar     │      varchar      │             varchar              │
├──────────────────────┼────────────────┼───────────────────┼──────────────────────────────────┤
│ KABUPATEN PEKALONGAN │ JAWA TENGAH    │ 14409             │ 861eff4711a542e4b93843c6dd7febb0 │
│ KOTA BEKASI          │ JAWA BARAT     │ 9790              │ 290c77bc529b7ac935b93aa66c333dc3 │
└──────────────────────┴────────────────┴───────────────────┴──────────────────────────────────┘