https://linen.dev logo
Join Slack
Powered by
I have oracle-tap setup as in the following. It ru...
# troubleshooting
j
I have oracle-tap setup as in the following. It run the fine the first time 
meltano run tap-oracle target-parquet
, then i delete a row in oracle , and rerun it, i got the error. Can someone please help?
Copy code
- name: tap-oracle
    variant: s7clarke10
    pip_url: git+<https://github.com/s7clarke10/pipelinewise-tap-oracle.git>
    config:
      default_replication_method: LOG_BASED
      filter_schemas: IFSAPP
      filter_tables:
      - IFSAPP-ABC_CLASS_TAB
      host: xxxx
      port: 1521
      service_name: xxxx
      user: ifsapp
    metadata:
      IFSAPP-ABC_CLASS_TAB:
        replication-method: LOG_BASED
=============
Copy code
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Extractor failed
e
Can you run with 
--log-level=debug
and share the results?
j
@Edgar Ramírez (Arch.dev) sorry, i been off the last two weeks. i ran the command but got the following error:
Copy code
root@0f247a4b4e9a:/projects/my-new-project# meltano run tap-oracle target-parquet --log-level=debug
2025-03-20T23:17:56.990824Z [debug    ] Meltano 3.6.0, Python 3.10.16, Linux (x86_64)
2025-03-20T23:17:56.993059Z [debug    ] /etc/timezone found, contents:
 America/Los_Angeles

2025-03-20T23:17:56.993757Z [debug    ] /etc/localtime found
2025-03-20T23:17:56.995159Z [debug    ] 2 found:
 {'/etc/timezone': 'America/Los_Angeles', '/etc/localtime is a symlink to': 'America/Los_Angeles'}
Usage: meltano run [OPTIONS] [BLOCKS]...
Try 'meltano run --help' for help.

Error: No such option: --log-level
e
Oh, sorry it should rather be
Copy code
meltano --log-level=debug run tap-oracle target-parquet
j
Hi @Edgar Ramírez (Arch.dev) Please see the debug file.
Debug.txt
e
I'm see in there:
Copy code
cx_Oracle.DatabaseError: ORA-12170: Cannot connect. TCP connect timeout of 20s for host 10.10.131.69 port 1521
not sure what it means, though
j
I see, i wasn't connect the right network. I will run again so that you can see the correct log for the error.
👍 1
I think i find the issue. [Unable to replicate with logminer for stream(IFSAPP-ABC_CLASS_TAB) because supplmental_log_data is not set to 'ALL' for either the table or the database.] Is there Oracle Tap only require enable all supplemental logging only for specific tables? Our DBA is concerned about the performance after turning supplemental at database level . So we went wit the minimum logging. Table Level Logging
Copy code
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
ALTER TABLE "<schema>"."<table>" ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS
Database Level Logging
Copy code
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS
Copy code
2025-03-24T17:37:01.947056Z [info     ]   File "/meltano_projects/data_test1/.meltano/extractors/tap-oracle/venv/lib/python3.10/site-packages/tap_oracle/sync_strategies/log_miner.py", line 117, in sync_tables cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=tap-oracle
2025-03-24T17:37:01.949877Z [info     ]     raise Exception("""        cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=tap-oracle
2025-03-24T17:37:01.950084Z [info     ] Exception:                     cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=tap-oracle
2025-03-24T17:37:01.950260Z [info     ]       Unable to replicate with logminer for stream(IFSAPP-ABC_CLASS_TAB) because supplmental_log_data is not set to 'ALL' for either the table or the database. cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=tap-oracle
2025-03-24T17:37:01.950437Z [info     ]       Please run: ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS; cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=tap-oracle
2025-03-24T17:37:01.950584Z [info     ]                                cmd_type=elb consumer=False job_name=dev:tap-oracle-to-target-parquet name=tap-oracle producer=True run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=tap-oracle
2025-03-24T17:37:02.040014Z [debug    ] head producer completed first as expected name=tap-oracle
2025-03-24T17:37:02.040494Z [debug    ] tail consumer is next block, wrapping up
2025-03-24T17:37:02.089612Z [info     ] 2025-03-24 10:37:02,089 | INFO     | target-parquet       | Target 'target-parquet' is listening for input from tap. cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=target-parquet
2025-03-24T17:37:02.090173Z [info     ] 2025-03-24 10:37:02,090 | INFO     | target-parquet       | Target 'target-parquet' completed reading 0 lines of input (0 schemas, 0 records, 0 batch manifests, 0 state messages). cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=target-parquet
2025-03-24T17:37:02.090892Z [info     ] 2025-03-24 10:37:02,090 | INFO     | target-parquet       | Emitting completed target state {} cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stderr string_id=target-parquet
2025-03-24T17:37:02.099130Z [debug    ] Added to state dev:tap-oracle-to-target-parquet state payload {'singer_state': {}}
2025-03-24T17:37:02.105287Z [info     ] Incremental state has been updated at 2025-03-24 17:37:02.105263+00:00.
2025-03-24T17:37:02.105868Z [debug    ] Incremental state: {}
2025-03-24T17:37:02.106080Z [info     ] {}                             cmd_type=elb consumer=True job_name=dev:tap-oracle-to-target-parquet name=target-parquet producer=False run_id=5cbc26aa-b9a4-4464-91c1-e7080c287b68 stdio=stdout string_id=target-parquet
2025-03-24T17:37:02.196163Z [debug    ] Deleted configuration at /meltano_projects/data_test1/.meltano/run/tap-oracle/tap.c4f161d8-fbb3-4de2-9878-2493d46dea51.config.json
2025-03-24T17:37:02.197210Z [debug    ] Deleted configuration at /meltano_projects/data_test1/.meltano/run/target-parquet/target.60373dee-e2b1-4581-b585-ae36f6c536b6.config.json
2025-03-24T17:37:02.207594Z [error    ] Extractor failed
2025-03-24T17:37:02.209023Z [error    ] Block run completed.           block_type=ExtractLoadBlocks err=RunnerError('Extractor failed') exit_codes={<PluginType.EXTRACTORS: 'extractors'>: 1} set_number=0 success=False
2025-03-24T17:37:02.210885Z [debug    ] Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.
6 Views
Open in Slack
PreviousNext
Powered by