`meltano config target-jsonl set --interactive` r...
# troubleshooting
j
meltano config target-jsonl set --interactive
ran successfully when i set to LOG_BASED as default_replication_method, however it failed once i changed to the INCREMENTAL. And idea?
Copy code
2025-01-15T01:06:58.732578Z [info     ] Environment 'dev' is active
2025-01-15T01:07:00.767142Z [info     ] time=2025-01-14 17:07:00 name=singer level=INFO message=Running in cx mode cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768036Z [info     ] time=2025-01-14 17:07:00 name=singer level=WARNING message=cx_oracle is no longer maintained, use python-oracledb cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768213Z [info     ]                                cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768367Z [info     ] To switch to python-oracledb set the environment variable ORA_PYTHON_DRIVER_TYPE=thin or thick. cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768517Z [info     ]                                cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768733Z [info     ] Documentation for python-oracledb can be found here: <https://oracle.github.io/python-oracledb/> cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.771719Z [info     ] time=2025-01-14 17:07:00 name=singer level=INFO message=Selected streams: ['IFSAPP-ABC_CLASS_TAB'] cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772098Z [info     ] time=2025-01-14 17:07:00 name=singer level=INFO message=No currently_syncing found cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772266Z [info     ] time=2025-01-14 17:07:00 name=singer level=INFO message=Beginning sync of stream(IFSAPP-ABC_CLASS_TAB) with sync method(incremental) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772459Z [info     ] time=2025-01-14 17:07:00 name=singer level=CRITICAL message=No replication key selected for key-based incremental replication cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772623Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772760Z [info     ]   File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/bin/tap-oracle", line 8, in <module> cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773071Z [info     ]     sys.exit(main())           cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773292Z [info     ]   File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 686, in main cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773499Z [info     ]     raise exc                  cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773667Z [info     ]   File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 683, in main cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773821Z [info     ]     main_impl()                cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774012Z [info     ]   File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 677, in main_impl cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774311Z [info     ]     do_sync(conn_config, args.catalog, args.config.get('default_replication_method'), state) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774525Z [info     ]   File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 619, in do_sync cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774753Z [info     ]     state = sync_traditional_stream(conn_config, stream, state, sync_method_lookup[stream.tap_stream_id], end_scn) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774967Z [info     ]   File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 574, in sync_traditional_stream cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.775374Z [info     ]     state = do_sync_incremental(conn_config, stream, state, desired_columns) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.775620Z [info     ]   File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 448, in do_sync_incremental cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.775836Z [info     ]     raise Exception("No replication key selected for key-based incremental replication") cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet
 name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.776018Z [info     ] Exception: No replication key selected for key-based incremental replication cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:01.517992Z [info     ] 2025-01-14 17:07:01,517 | INFO     | target-parquet       | Target 'target-parquet' is listening for input from tap. cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=target-parquet
2025-01-15T01:07:01.518343Z [info     ] 2025-01-14 17:07:01,517 | INFO     | target-parquet       | Target 'target-parquet' completed reading 0 lines of input (0 schemas, 0 records, 0 batch manifests, 0 state messages). cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=target-parquet
2025-01-15T01:07:01.518595Z [info     ] 2025-01-14 17:07:01,518 | INFO     | target-parquet       | Emitting completed target state {} cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=target-parquet
2025-01-15T01:07:01.546953Z [info     ] Incremental state has been updated at 2025-01-15 01:07:01.546913+00:00.
2025-01-15T01:07:01.637417Z [error    ] Extractor failed
2025-01-15T01:07:01.637671Z [error    ] Block run completed.           block_type=ExtractLoadBlocks err=RunnerError('Extractor failed') exit_codes={<PluginType.EXTRACTORS: 'extractors'>: 1} set_number=0 success=False
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Extractor failed
Copy code
No replication key selected for key-based incremental replication cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
Looks like it missing a replication key, but i am sure where i can set it.
e
What does
meltano.yml
look like?
j
extractors: - name: tap-oracle variant: s7clarke10 pip_url: git+https://github.com/s7clarke10/pipelinewise-tap-oracle.git config: default_replication_method: INCREMENTAL filter_schemas: IFSAPP filter_tables: - IFSAPP-ABC_CLASS_TAB host: --- port: 1521 service_name: --- user: ifsapp
fya, i removed host and service name.
e
I think you still need to add a replication key, see https://docs.meltano.com/concepts/plugins/#metadata-extra