emcp
03/05/2022, 6:14 PMmeltano add --custom loader target-parquet
Picked some sane defaults (I only customized the namespace) .. but now it just sits here. anyone experienced this ?emcp
03/05/2022, 6:15 PMmeltano add --custom loader target-parquet
Adding new custom loader with name 'target-parquet'...
Specify the plugin's namespace, which will serve as the:
- identifier to find related/compatible plugins
- default target database dialect (`dialect` extra),
for use by transformers that connect with the database
Hit Return to accept the default: plugin name with underscores instead of dashes
(namespace) [target_parquet]: dump_stonks_raw
Specify the plugin's `pip install` argument, for example:
- PyPI package name:
target-parquet
- Git repository URL:
git+<https://gitlab.com/meltano/target-parquet.git>
- local directory, in editable/development mode:
-e extract/target-parquet
- 'n' if using a local executable (nothing to install)
Default: plugin name as PyPI package name
(pip_url) [target-parquet]:
Specify the plugin's executable name
Default: name derived from `pip_url`
(executable) [target-parquet]:
Specify the target's supported settings (`config.json` keys)
Multiple setting names (keys) can be separated using commas.
A setting kind can be specified alongside the name (key) by using the `:` delimiter,
e.g. `port:integer` to set the kind `integer` for the name `port`
Supported setting kinds:
string | integer | boolean | date_iso8601 | email | password | oauth | options | file | array | object | hidden
- Credentials and other sensitive setting types should use the password kind.
- If not specified, setting kind defaults to string.
- Nested properties can be represented using the `.` separator, e.g. `auth.username` for `{ "auth": { "username": value } }`.
- To find out what settings a target supports, reference its documentation.
Default: no settings
(settings) [[]]:
Added loader 'target-parquet' to your Meltano project
2022-03-05T18:04:53.049721Z [info ] Emitter initialized with endpoint <http://sp.meltano.com/i>
Installing loader 'target-parquet'...
emcp
03/05/2022, 6:39 PM03-05T18:37:40.179269Z [warning ] Python-dotenv could not parse statement starting at line 1
2022-03-05T18:37:40.179602Z [warning ] Python-dotenv could not parse statement starting at line 1
2022-03-05T18:37:40.179881Z [warning ] Python-dotenv could not parse statement starting at line 1
2022-03-05T18:37:40.180270Z [warning ] Python-dotenv could not parse statement starting at line 1
2022-03-05T18:37:40.180748Z [warning ] Python-dotenv could not parse statement starting at line 1
2022-03-05T18:37:40.180978Z [warning ] Python-dotenv could not parse statement starting at line 1
2022-03-05T18:37:40.182940Z [warning ] No state was found, complete import.
2022-03-05T18:37:40.183213Z [warning ] Python-dotenv could not parse statement starting at line 1
2022-03-05T18:37:40.261995Z [info ] ELT could not be completed: Cannot start extractor: Catalog discovery failed: command ['/meltano-stonks-ib/.meltano/extractors/tap-postgres/venv/bin/tap-postgres', '--config', '/meltano-stonks-ib/.meltano/run/elt/postgres-to-parquet/3977c95d-6ac7-48a7-ba24-92ceb1e29169/tap.ed6a0b5b-7ef8-4ffd-869a-cbe3a826b0f8.config.json', '--discover'] returned 1. cmd_type=elt job_id=postgres-to-parquet name=meltano run_id=3977c95d-6ac7-48a7-ba24-92ceb1e29169 stdio=stderr
2022-03-05T18:37:40.262250Z [info ] For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag. cmd_type=elt job_id=postgres-to-parquet name=meltano run_id=3977c95d-6ac7-48a7-ba24-92ceb1e29169 stdio=stderr
2022-03-05T18:37:40.262396Z [info ] Note that you can also check the generated log file at '/meltano-stonks-ib/.meltano/logs/elt/postgres-to-parquet/3977c95d-6ac7-48a7-ba24-92ceb1e29169/elt.log'. cmd_type=elt job_id=postgres-to-parquet name=meltano run_id=3977c95d-6ac7-48a7-ba24-92ceb1e29169 stdio=stderr
2022-03-05T18:37:40.262526Z [info ] For more information on debugging and logging: <https://docs.meltano.com/reference/command-line-interface#debugging> cmd_type=elt job_id=postgres-to-parquet name=meltano run_id=3977c95d-6ac7-48a7-ba24-92ceb1e29169 stdio=stderr
emcp
03/05/2022, 6:41 PMemcp
03/05/2022, 6:41 PMemcp
03/05/2022, 6:51 PMtarget-parquet
.. Ive tripple checked my setup and now it seems the last error says the loader is not installed.. and when I go to install it I just get it hanging again..emcp
03/05/2022, 8:26 PMpip install git+<https://github.com/estrategiahq/target-parquet.git>
seemed to reveal more of the issueemcp
03/05/2022, 9:04 PMemcp
03/05/2022, 10:08 PMemcp
03/05/2022, 10:09 PMemcp
03/06/2022, 8:54 AMtarget-parquet
alias… nothing finishes and I am led to believe it’s failed but.. unsure where to check or fix it?emcp
03/06/2022, 8:54 AMemcp
03/06/2022, 8:56 AMemcp
03/06/2022, 9:03 AMemcp
03/06/2022, 9:08 AMtaylor
03/07/2022, 11:00 PMtarget-parquet
in that namespace. https://hub.meltano.com/targets/parquetemcp
03/09/2022, 7:51 AMmeltano add --custom loader target-parquet
taylor
03/09/2022, 4:50 PMtaylor
03/09/2022, 4:53 PMemcp
03/09/2022, 7:06 PMtaylor
03/09/2022, 8:04 PM