guido_trucco
04/19/2023, 3:05 PMtarget-s3
without success. This is the error I'm getting:
```2023-04-19T150241.855769Z [info ] Traceback (most recent call last): cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.856082Z [info ] File ".../venv/bin/target-s3", line 8, in <module> cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.856522Z [info ] sys.exit(Targets3.cli()) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.856812Z [info ] File ".../venv/lib/python3.9/site-packages/click/core.py", line 1130, in call cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.857079Z [info ] return self.main(*args, **kwargs) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.857340Z [info ] File ".../venv/lib/python3.9/site-packages/click/core.py", line 1055, in main cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.857606Z [info ] rv = self.invoke(ctx) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.857863Z [info ] File ".../venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.858215Z [info ] return ctx.invoke(self.callback, **ctx.params) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.858504Z [info ] File ".../venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.858773Z [info ] return __callback(*args, **kwargs) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859058Z [info ] File ".../venv/lib/python3.9/site-packages/singer_sdk/target_base.py", line 564, in cli cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859248Z [info ] target.listen(file_input) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859373Z [info ] File ".../venv/lib/python3.9/site-packages/singer_sdk/io_base.py", line 34, in listen cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859481Z [info ] self._process_lines(file_input) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859585Z [info ] File ".../venv/lib/python3.9/site-packages/singer_sdk/target_base.py", line 265, in _process_lines cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859689Z [info ] counter = super()._process_lines(file_input) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859794Z [info ] File ".../venv/lib/python3.9/site-packages/singer_sdk/io_base.py", line 78, in _process_lines cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859896Z [info ] self._process_schema_message(line_dict) cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.859997Z [info ] File ".../venv/lib/python3.9/site-packages/singer_sdk/target_base.py", line 364, in _process_schema_message cmd_type=elb consumer=True name=target-s3 producer=False stdio=stderr string_id=target-s3
2023-04-19T150241.860102Z [info ] …guido_trucco
04/19/2023, 3:07 PM{
"aws_access_key": "",
"aws_secret_access_key": "",
"aws_region": "us-east-1",
"bucket": "meltano",
"prefix": "meltano-test",
"append_date_to_prefix": true,
"append_date_to_prefix_grain": "day",
"append_date_to_filename": false,
"append_date_to_filename_grain": "day",
"format_type": "json",
"flatten_records": false,
"set_dtype_string": false,
"flattening_enabled": false
}
pat_nadolny
04/19/2023, 3:13 PMI was thinking this target could become target-data-lake and serve as a multi-cloud storage/multi-format target.
We'll either pin the pre-1.0 release in meltanohub or remove this target in its current state as target-s3. I would suggest trying one of the other s3 file type specific variants like https://hub.meltano.com/loaders/target-s3-csv/ or https://hub.meltano.com/loaders/target-s3-parquet/ or https://hub.meltano.com/loaders/target-s3-avro/guido_trucco
04/19/2023, 4:41 PMpat_nadolny
04/19/2023, 9:34 PM