this is my configuration `version: 1` `default_en...
# plugins-general
p
this is my configuration
version: 1
default_environment: dev
project_id: 08bc4f11-df33-41ae-ad0a-3455e11026f0
environments:
- name: dev
- name: staging
- name: prod
plugins:
extractors:
- name: tap-csv
variant: meltanolabs
pip_url: git+<https://github.com/MeltanoLabs/tap-csv.git>
config:
csv_files_definition: files-def.json
p
@peter that config looks correct and the fact that youre getting a record message is good. What commands are you running? What does your files-def.json look like? If you got a configuration is invalid error then you shouldnt be sending RECORD messages 🤔
p
files-def.json
[
{
"entity": "sample",
"path": "sample.csv",
"delimiter": ",",
"keys": [
"col1",
"col2",
"col3"
],
"encoding": "utf-8",
"escapechar": "",
"strict": false,
"skipinitialspace": true,
"quotechar": "'",
"add_metadata_columns": false,
"csv_files_definition": [
"sample.csv"
]
}
]
@pat_nadolny I am using the meltano test command
@pat_nadolny I use this to test if the extract config is correct I get that message
meltano config tap-csv test
@pat_nadolny Any idea on how to fix this?
p
@peter hmm 🤔 im not sure whats causing this. I replicated your exact setup and couldnt reproduce this. If I get the
Plugin configuration is invalid
error it tells me why like
Exception: File path does not exist data/bad.csv
afterwards. Its especially weird that you see a record also because the test command is asserting that its able to extract at least 1 record, which it clearly can do.