I am using the target-jsonl target. When I run meltano elt tap-zendesk-chat target-jsonl I get s success message but nothing is written to my files directory (my destination_path). This is my first time working with Meltano so I think I should be getting files written but am not sure.

@mark_moesch Can you share the full

meltano elt

output please?

meltano | Running extract & load... tap-zendesk-chat | INFO METRIC: {"type": "timer", "metric": "http_request_duration", "value": 1.843609094619751, "tags": {"endpoint": "departments", "http_status_code": 200, "status": "succeeded"}} tap-zendesk-chat | INFO METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.2225022315979004, "tags": {"endpoint": "account", "http_status_code": 200, "status": "succeeded"}} meltano | Incremental state has been updated at 2021-09-10 150928.138930. meltano | Extract & load complete! meltano | Transformation skipped.

Can you run again with

meltano --log-level=debug

? That should be able to tell us if

tap-zendesk-chat

is finding any records at all!

The sure did produce a lot of words! Would you like to see all of it or are there more important sections?

Do you see any lines starting with

tap-zendesk-chat (out)

?

tap-zendesk-chat (out) | {"type": "STATE", "value": {"currently_syncing": null}} target-jsonl (out)     | {"currently_syncing": null} meltano                | INFO Incremental state has been updated at 2021-09-10 154818.971676. meltano                | DEBUG Incremental state: {'currently_syncing': None} meltano                | DEBUG Deleted configuration at

OK, since we’re not seeing any

type: RECORD

messages, this indicates that the tap was not actually able to find any records, so it makes sense that the target didn’t create any jsonl files 🙂

name: tap-zendesk-chat namespace: tap_zendesk_chat pip_url: git+https://github.com/singer-io/tap-zendesk-chat.git executable: tap-zendesk-chat

Ah, there we go! It’s missing the

discover

,

properties

, and

state

capabilitites:

name: tap-zendesk-chat
    namespace: tap_zendesk_chat
    pip_url: git+<https://github.com/singer-io/tap-zendesk-chat.git>
    executable: tap-zendesk-chat
    capabilities:
    - discover
    - properties
    - state

YEp

OK good. If you add those capabilities and try again, the result should be different

I will follow the doco and see if that works.

That’s great to hear! I can’t spend as much time here as I like to, but especially in the mornings when I’m catching up on emails and other Slack stuff, I can’t help myself, and it’s always educational 🙂

Will do!

Hi again - I thought I had this but when I put in a capability of catalog or property ( I tried both) I get Applying catalog rules failed: catalog file is missing despite having a catalog.json or catalog-file.json. Any ideas?

@mark_moesch Can you please share your

meltano.yml

definition again, and the exact command you’re running?

project_id: 177c7e40-9191-49a9-99c2-7ea509e4f40a plugins: extractors: - name: tap-zendesk-chat namespace: tap_zendesk_chat pip_url: git+https://github.com/singer-io/tap-zendesk-chat.git executable: tap-zendesk-chat capabilities: - properties config: start_date: '2021-01-01' access_token: $TAP_ZENDESK_CHAT__ACCESS_TOKEN select: $TAP_ZENDESK_CHAT__SELECT loaders: - name: target-jsonl variant: andyh1203 pip_url: target-jsonl config: destination_path: my_jsonl_files - name: target-redshift variant: transferwise pip_url: pipelinewise-target-redshift schedules: - name: zendesk-chat-to-jsonl extractor: tap-zendesk-chat loader: target-jsonl transform: skip interval: '@once' start_date: 2021-01-01 000000

OK, under

capabilities

you still need

discover

and

state

, as in https://meltano.slack.com/archives/C01TCRBBJD7/p1631289601168300?thread_ts=1631288219.165800&cid=C01TCRBBJD7 🙂

Can I get rid of the catalog.json then too?

Yep!

so I should have catalog and properties or just properties?

Just properties

I get No state was found, complete import. meltano | ELT could not be completed: Cannot start extractor: Catalog discovery failed:

Any additional output indicating why discovery may have failed?

Catalog discovery failed: command ['/Users/markmoesch/projects/zendeskchat/zendeskchat/.meltano/extractors/tap-zendesk-chat/venv/bin/tap-zendesk-chat', '--config', '/Users/markmoesch/projects/zendeskchat/zendeskchat/.meltano/run/elt/2021-09-10T191434--tap-zendesk-chat--target-jsonl/9e504638-d652-43f6-8456-3b072e0a1d3b/tap.6db050b9-cd1c-4ba1-9906-efd9b34e0792.config.json', '--discover'] returned 1: INFO METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.6919238567352295, "tags": {"endpoint": "departments", "http_status_code": 401, "status": "succeeded"}}

@mark_moesch Can you run

meltano invoke tap-zendesk-chat --discover

and see if that prints an error?

INFO METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.6741118431091309, "tags": {"endpoint": "departments", "http_status_code": 401, "status": "succeeded"}} CRITICAL 401 Client Error: UNAUTHORIZED for url: https://www.zopim.com/api/v2/departments

Yep looks like an authentication issue. Does

meltano config tap-zendesk-chat

still show the

access_token

as expected?

It returns my access token wrapped in ""

It looks like the access token is passed along in the Authorization header: https://github.com/singer-io/tap-zendesk-chat/blob/master/tap_zendesk_chat/http.py#L28 Is that the same thing you’re doing in postman?

yes- I see the token and a header prefix of bearer

That’s about as far as I can help with debugging, I’m afraid. At this point the issue doesn’t appear to be Meltano, but rather the tap itself and the way it’s creating the request

Well I appreciate all your help you went above and beyond. I'll let my team know of the good service

I appreciate it 😄 Good luck!