#C069CQNHDNF I am facing this issue while testing tap-s3-csv for both: • Steve Clarke (default):second_place_medal: • Wise and there is no Integer field in .csv file please let me know how we can fix it.

Can you post your config?

means meltano.yml?

Can you send the full meltano.yml? Removing any secrets you may have

This is the complete details for extractor

can you try quoting the start_date?

no still same error

It is hard to debug without all the info, but I would try quoting the values of the config just to see if that works. If all the plugins report the same error with the config, it sounds like something with that

can we connect on zoom?

I'm not available right now, but did you try quoting the values?

Yes tried quoting all values

There's a huge difference between the default and

sys.maxsize

though:

>>> csv.field_size_limit()
131072
>>> sys.maxsize
922337203685477580

Can you confirm that your CSV has a field with more than 131072 bytes?

I don't have any field whose size is more than this default max size.

Can you test with another CSV file just to ensure it's not a problem with the config?

I tested it with multiple files

Even simple ones?

Yes even with simple one- where only one column containing country names.

Can you provide that one?

population.csv

Can you send the config after you quoted the values?

image.png

I had no problem ingesting both those files.

- name: tap-s3-csv
    pip_url: git+<https://github.com/s7clarke10/pipelinewise-tap-s3-csv.git@v3.0.0>
    variant: s7clarke10
  - name: tap-s3-csv--population
    inherit_from: tap-s3-csv
    config:
      bucket: "mybucket"
      start_date: "2000-01-01T00:00:00Z"
      tables:
      - table_name: "population"
        key_properties:
        - "Country"
        search_prefix: "tst/dev/"
        search_pattern: ".*population.*[.]csv"
        delimiter: ','
        datatype_overrides:
          Country: "string"
          Population: "integer"
          GDP: "string"
          Area: "integer"

- name: tap-s3-csv pip_url: git+https://github.com/s7clarke10/pipelinewise-tap-s3-csv.git@v3.0.0 variant: s7clarke10 - name: tap-s3-csv--population inherit_from: tap-s3-csv config: bucket: "audiencex-test-files" start_date: "2000-01-01T000000Z" tables: - table_name: "population" key_properties: - "Country" search_prefix: "ipr5zct-piewie-clicks/" search_pattern: ".*population.*[.]csv" delimiter: ',' datatype_overrides: Country: "string" Population: "integer" GDP: "string" Area: "integer" Here I kept everything as you shared and just changed the bucket and search prefix but still the same issue. I am using Windows OS FYI

Looking at the error message it seems like a permissions and config issue with Meltano. Here is the results of my config.

(.venv) [sclarke@server]$ meltano config tap-s3-csv--population
{
  "aws_access_key_id": "<removed>",
  "aws_secret_access_key": "<removed>",
  "bucket": "mybucket",
  "start_date": "2000-01-01T00:00:00Z",
  "tables": [
    {
      "table_name": "population",
      "key_properties": [
        "Country"
      ],
      "search_prefix": "tst/dev/",
      "search_pattern": ".*population.*[.]csv",
      "delimiter": ",",
      "datatype_overrides": {
        "Country": "string",
        "Population": "integer",
        "GDP": "string",
        "Area": "integer"
      }
    }
  ]
}

Tried this also but not worked