Hi Team!
I’m working on a tap-posgtres (transferw...
# troubleshootingp
pramit
12/04/2022, 7:09 PMHi Team!
I’m working on a tap-posgtres (transferwise variant) => target-s3-parquet extraction. I’m not doing any transformations yet, so those are the only two steps.
Pls help with this error:
2022-12-02T185943.705344Z [error ] Loader failed
/Users/asd/.local/pipx/venvs/meltano/lib/python3.9/site-packages/meltano/core/block/ │
│ extract_load.py:487 in run_with_job │
│ │
│ 484 │ │ │
│ 485 │ │ with closing(self.context.session) as session: │
│ 486 │ │ │ async with job.run(session): │
│ ❱ 487 │ │ │ │ await self.execute() │
│ 488 │ │
│ 489 │ async def terminate(self, graceful: bool = False) -> None: │
│ 490 │ │ """Terminate an in flight ExtractLoad execution, potentially disruptive. │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ existing = None │ │
│ │ job = <repr-error 'Instance <Job at 0x10819b340*>* is not bound to a Session; attribute │ │
│ │ refresh operation cannot proceed *(*Background on this error at: │ │
│ │ https://sqlalche.me/e/14/bhk3*)*'> │ │
│ │ self = <meltano.core.block.extract_load.ExtractLoadBlocks object at 0x10819b520*>* │ │
│ │ session = <sqlalchemy.orm.session.Session object at 0x1081b7a60*>* │ │
pramit
12/05/2022, 7:55 AMReplication is incremental and key is defined. I noticed that parquet file is exported to S3 but the "*RunnerError:* Loader failed" is logged on console.
pramit
12/06/2022, 11:22 AMPls help here. Run fails (last line below). More logs:
```2022-12-06T111650.498798Z [info ] time=2022-12-06 164650 name=tap-postgres level=INFO message=Beginning incremental sync of 'public-customers'... cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stderr string_id=tap-postgres
2022-12-06T111650.498865Z [info ] time=2022-12-06 164650 name=tap-postgres level=INFO message=Tap has custom mapper. Using 1 provided map(s). cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stderr string_id=tap-postgres
2022-12-06T111650.498945Z [info ] {"type": "SCHEMA", "stream": "public-customers", "schema": {"properties": {"customer_id": {"type": ["integer"]}, "company": {"type": ["string"]}, "last_name": {"type": ["string"]}, "first_name": {"type": ["string"]}, "email_address": {"type": ["string"]}}, "type": "object"}, "key_properties": ["customer_id"], "bookmark_properties": ["customer_id"]} cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stdout string_id=tap-postgres
2022-12-06T111650.954632Z [info ] time=2022-12-06 164650 name=tap-postgres level=INFO message=Properties () were present in the 'public-customers' stream but not found in catalog schema. Ignoring. cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stderr string_id=tap-postgres
2022-12-06T111650.954907Z [info ] {"type": "RECORD", "stream": "public-customers", "record": {"customer_id": 1, "company": "R", "last_name": "N", "first_name": "C", "email_address": "n@rk.com"}, "time_extracted": "2022-12-06T111650.954521Z"} cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stdout string_id=tap-postgres
2022-12-06T111650.955078Z [info ] {"type": "STATE", "value": {"bookmarks": {"public-customers": {"starting_replication_value": null, "progress_markers": {"Note": "Progress is not resumable if interrupted.", "replication_key": "customer_id", "replication_key_value": 1}}}}} cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stdout string_id=tap-postgres
2022-12-06T111650.955175Z [info ] {"type": "RECORD", "stream": "public-customers", "record": {"customer_id": 2, "company": "Y", "last_name": "A", "first_name": "C", "email_address": "amie@y.com"}, "time_extracted": "2022-12-06T111650.954806Z"} cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stdout string_id=tap-postgres
2022-12-06T111650.955316Z [info ] {"type": "RECORD", "stream": "public-customers", "record": {"customer_id": 3, "company": "A", "last_name": "B", "first_name": "C", "email_address": "D"}, "time_extracted": "2022-12-06T111650.954912Z"} cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stdout string_id=tap-postgres
2022-12-06T111650.955475Z [info ] {"type": "RECORD", "stream": "public-customers", "record": {"customer_id": 4, "company": "A4", "last_name": "B4", "first_name": "C4", "email_address": "D4"}, "time_extracted": "2022-12-06T111650.955084Z"} cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stdout string_id=tap-postgres
2022-12-06T111650.955649Z [info ] time=2022-12-06 164650 name=tap-postgres level=INFO message=INFO METRIC: {'type': 'counter', 'metric': 'record_count', 'value': 4, 'tags': {'stream': 'public-customers'}} cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stderr string_id=tap-postgres
2022-12-06T111650.955819Z [info ] {"type": "STATE", "value": {"bookmarks": {"public-customers": {"replication_key": "customer_id", "replication_key_value": 4}}}} cmd_type=elb consumer=False name=tap-postgres producer=True stdio=stdout string_id=tap-postgres
2022-12-06T111652.291304Z [info ] time=2022-12-06 164652 name=target-s3-parquet level=INFO message=target-s3-parquet v0.0.1, Meltano SDK v0.5.0) cmd_type=elb consumer=True name=target-s3-parquet producer=False stdio=stderr string_id=target-s3-parquet
2022-12-06T111652.293558Z [info …
j
josh_lloyd
02/28/2023, 6:57 PMDid you ever figure out a solution to this? getting the same thing. Found this https://docs.sqlalchemy.org/en/14/errors.html#error-bhk3
2 Views