would be alot more helpful if this error was more descriptive:

root@60789494ab28:/projects/taps/tap-pardot# cat results/account.txt | meltano --log-level=debug invoke target-redshift
2023-07-07T20:35:44.637367Z [debug    ] /etc/timezone found, contents:
 Etc/UTC

2023-07-07T20:35:44.638332Z [debug    ] /etc/localtime found
2023-07-07T20:35:44.640264Z [debug    ] 2 found:
 {'/etc/timezone': 'Etc/UTC', '/etc/localtime is a symlink to': 'Etc/UTC'}
2023-07-07T20:35:44.645692Z [info     ] Environment 'test' is active
2023-07-07T20:35:44.685084Z [debug    ] Creating engine '<meltano.core.project.Project object at 0x7f230ed7eaf0>@sqlite:////projects/taps/tap-pardot/.meltano/meltano.db'
2023-07-07T20:35:44.847052Z [debug    ] Found plugin parent            parent=target-redshift plugin=target-redshift source=<DefinitionSource.LOCKFILE: 8>
2023-07-07T20:35:45.490795Z [debug    ] Created configuration at /projects/taps/tap-pardot/.meltano/run/target-redshift/target.31559af2-815a-4d6b-a0bc-0d884a0381e5.config.json
2023-07-07T20:35:45.500188Z [debug    ] Invoking: ['/projects/taps/tap-pardot/.meltano/loaders/target-redshift/venv/bin/target-redshift', '--config', '/projects/taps/tap-pardot/.meltano/run/target-redshift/target.31559af2-815a-4d6b-a0bc-0d884a0381e5.config.json']
time=2023-07-07 20:35:45 name=target_redshift level=INFO message=Getting catalog objects from table cache...
Traceback (most recent call last):
  File "/projects/taps/tap-pardot/.meltano/loaders/target-redshift/venv/bin/target-redshift", line 8, in <module>
    sys.exit(main())
  File "/projects/taps/tap-pardot/.meltano/loaders/target-redshift/venv/lib/python3.9/site-packages/target_redshift/__init__.py", line 447, in main
    persist_lines(config, singer_messages, table_cache)
  File "/projects/taps/tap-pardot/.meltano/loaders/target-redshift/venv/lib/python3.9/site-packages/target_redshift/__init__.py", line 237, in persist_lines
    stream_to_sync[stream] = DbSync(config, add_metadata_columns_to_schema(o), table_cache)
  File "/projects/taps/tap-pardot/.meltano/loaders/target-redshift/venv/lib/python3.9/site-packages/target_redshift/db_sync.py", line 318, in __init__
    self.flatten_schema = flatten_schema(stream_schema_message['schema'], max_level=self.data_flattening_max_level)
  File "/projects/taps/tap-pardot/.meltano/loaders/target-redshift/venv/lib/python3.9/site-packages/target_redshift/db_sync.py", line 130, in flatten_schema
    if list(v.values())[0][0]['type'] == 'string':
TypeError: string indices must be integers
2023-07-07T20:35:46.440015Z [debug    ] Deleted configuration at /projects/taps/tap-pardot/.meltano/run/target-redshift/target.31559af2-815a-4d6b-a0bc-0d884a0381e5.config.json
root@60789494ab28:/projects/taps/tap-pardot#

im guessing that the input data is in the wrong format, but it's hard to know exactly how to fix it. in my parent tap class i have:

records_jsonpath = "$.values[*]"

and in my actual stream class i have

records_jsonpath = "$[*]"

because this one stream is different from the rest. but im not sure if this is the underlying issue or how to fix it

The error seems to be in the target. There's probably a mismatch between the record and schema that causes flattening on the target side to fail. But.. I agree that the error messages should be more readable and something we should take into account for the target SDK (that target you're using is not based on it btw).

hey yeah thanks, actually it was just that the schema had something bad in it - but its would be great if i could pinpoint it faster... so far i love meltano just debugging is really hard

Agreed! It's the downside to taps and targets being black boxes, I guess... meltano has no knowledge of what output the plugins are producing so logging and detecting errors and such is impossible. 😕