Jun Pei Liang
01/15/2025, 1:09 AMmeltano config target-jsonl set --interactive
ran successfully when i set to LOG_BASED as default_replication_method, however it failed once i changed to the INCREMENTAL. And idea?
2025-01-15T01:06:58.732578Z [info ] Environment 'dev' is active
2025-01-15T01:07:00.767142Z [info ] time=2025-01-14 17:07:00 name=singer level=INFO message=Running in cx mode cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768036Z [info ] time=2025-01-14 17:07:00 name=singer level=WARNING message=cx_oracle is no longer maintained, use python-oracledb cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768213Z [info ] cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768367Z [info ] To switch to python-oracledb set the environment variable ORA_PYTHON_DRIVER_TYPE=thin or thick. cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768517Z [info ] cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.768733Z [info ] Documentation for python-oracledb can be found here: <https://oracle.github.io/python-oracledb/> cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.771719Z [info ] time=2025-01-14 17:07:00 name=singer level=INFO message=Selected streams: ['IFSAPP-ABC_CLASS_TAB'] cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772098Z [info ] time=2025-01-14 17:07:00 name=singer level=INFO message=No currently_syncing found cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772266Z [info ] time=2025-01-14 17:07:00 name=singer level=INFO message=Beginning sync of stream(IFSAPP-ABC_CLASS_TAB) with sync method(incremental) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772459Z [info ] time=2025-01-14 17:07:00 name=singer level=CRITICAL message=No replication key selected for key-based incremental replication cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772623Z [info ] Traceback (most recent call last): cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.772760Z [info ] File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/bin/tap-oracle", line 8, in <module> cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773071Z [info ] sys.exit(main()) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773292Z [info ] File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 686, in main cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773499Z [info ] raise exc cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773667Z [info ] File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 683, in main cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.773821Z [info ] main_impl() cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774012Z [info ] File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 677, in main_impl cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774311Z [info ] do_sync(conn_config, args.catalog, args.config.get('default_replication_method'), state) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774525Z [info ] File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 619, in do_sync cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774753Z [info ] state = sync_traditional_stream(conn_config, stream, state, sync_method_lookup[stream.tap_stream_id], end_scn) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.774967Z [info ] File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 574, in sync_traditional_stream cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.775374Z [info ] state = do_sync_incremental(conn_config, stream, state, desired_columns) cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.775620Z [info ] File "/projects/my-meltano-project/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/__init__.py", line 448, in do_sync_incremental cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.775836Z [info ] raise Exception("No replication key selected for key-based incremental replication") cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet
name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:00.776018Z [info ] Exception: No replication key selected for key-based incremental replication cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
2025-01-15T01:07:01.517992Z [info ] 2025-01-14 17:07:01,517 | INFO | target-parquet | Target 'target-parquet' is listening for input from tap. cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=target-parquet
2025-01-15T01:07:01.518343Z [info ] 2025-01-14 17:07:01,517 | INFO | target-parquet | Target 'target-parquet' completed reading 0 lines of input (0 schemas, 0 records, 0 batch manifests, 0 state messages). cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=target-parquet
2025-01-15T01:07:01.518595Z [info ] 2025-01-14 17:07:01,518 | INFO | target-parquet | Emitting completed target state {} cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=target-parquet
2025-01-15T01:07:01.546953Z [info ] Incremental state has been updated at 2025-01-15 01:07:01.546913+00:00.
2025-01-15T01:07:01.637417Z [error ] Extractor failed
2025-01-15T01:07:01.637671Z [error ] Block run completed. block_type=ExtractLoadBlocks err=RunnerError('Extractor failed') exit_codes={<PluginType.EXTRACTORS: 'extractors'>: 1} set_number=0 success=False
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.
Run invocation could not be completed as block failed: Extractor failed
Jun Pei Liang
01/15/2025, 1:18 AMNo replication key selected for key-based incremental replication cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=9df1601f-6e9e-4084-bed3-ebfefec4c1db stdio=stderr string_id=tap-oracle
Looks like it missing a replication key, but i am sure where i can set it.Edgar Ramírez (Arch.dev)
01/15/2025, 4:08 AMmeltano.yml
look like?Jun Pei Liang
01/15/2025, 5:37 PMJun Pei Liang
01/15/2025, 5:38 PMEdgar Ramírez (Arch.dev)
01/16/2025, 7:40 PM