I'm following the tutorial (so starting from a blank slate) and want to use https://hub.meltano.com/extractors/tap-postmark/ as a tap, but I get an error from the get-go:

2022-12-09T08:19:52.623064Z [info     ] Environment 'staging' is active
2022-12-09T08:19:54.770821Z [warning  ] No state was found, complete import.
2022-12-09T08:19:55.123172Z [info     ] INFO >>> Running tap-postmark v0.1.0 cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.137587Z [info     ] INFO Sync                      cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.138077Z [info     ] INFO Syncing stream: messages_opens cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.138472Z [info     ] CRITICAL tap_postmark.postmark.Postmark.messages_opens() argument after ** must be a mapping, not NoneType cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.138685Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.138858Z [info     ]   File "/Users/mathieu/repositories/otainsight/elt-meltano/otainsight/.meltano/extractors/tap-postmark/venv/bin/tap-postmark", line 8, in <module> cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.139029Z [info     ]     sys.exit(main())           cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.139184Z [info     ]   File "/Users/mathieu/repositories/otainsight/elt-meltano/otainsight/.meltano/extractors/tap-postmark/venv/lib/python3.9/site-packages/singer/utils.py", line 235, in wrapped cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.139355Z [info     ]     return fnc(*args, **kwargs) cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.139511Z [info     ]   File "/Users/mathieu/repositories/otainsight/elt-meltano/otainsight/.meltano/extractors/tap-postmark/venv/lib/python3.9/site-packages/tap_postmark/tap.py", line 49, in main cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.139667Z [info     ]     sync(postmark, args.state, catalog, args.config['start_date']) cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.139827Z [info     ]   File "/Users/mathieu/repositories/otainsight/elt-meltano/otainsight/.meltano/extractors/tap-postmark/venv/lib/python3.9/site-packages/tap_postmark/sync.py", line 67, in sync cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.140037Z [info     ]     for row in tap_data(**stream_state): cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.140201Z [info     ] TypeError: tap_postmark.postmark.Postmark.messages_opens() argument after ** must be a mapping, not NoneType cmd_type=elb consumer=False name=tap-postmark producer=True stdio=stderr string_id=tap-postmark
2022-12-09T08:19:55.306491Z [error    ] Extractor failed

Hello @mathieu_hinderyckx, mind sharing your meltano.yml file (blank out your token)? Did you provide the plugin with a "start_date" and a "postmark_server_token"?

version: 1
default_environment: staging
project_id: 54ca0770-16c2-441f-a848-9aee0de6b5bc
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-postmark
    variant: yoast
    pip_url: git+<https://github.com/Yoast/singer-tap-postmark.git>
    config:
      start_date: '2022-11-01'
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl

The start date is filled in and the server token is configured in the .env file (via the interactive config wizard from the meltano cli)

Ok now that is interesting. First of all, I think that tap is trouble. Your error occurs because the variable "state" is empty (not a dict) https://github.com/Yoast/singer-tap-postmark/blob/d44108772a3d8019237c085ff80cba41f017b4ec/tap_postmark/sync.py . That is because you did not configure the "state.json", and that's apparently necessary (according to the readme and the code). Put it another way, it seems like you have to have a state to run this tap (weird). You can supply a state.json (like the example given inside the taps repository) to certain meltano CLI commands, but I'm not sure whether that works for invoke & run. So short answer: I need ask @aaronsteers!

But isn't the state created by meltano and stored in a db, rather than me having to manually create it? I do not want to work with json state files anymore...

Me neither @mathieu_hinderyckx. That's a weird dependency inside that tap.

I've made an issue on their github, which unfortunately is completely empty, as running it standalone is not working either. I'll have a look if I can write a tap myself. I've done so before and the documentation was quite confusing at the time, meltano doesn't have particular docs around making one either it seems?

@paulo_rijnberg wrote a custom tap himself I think, so maybe you can consult him if you stumble over problems in the process. Our tutorial on building a custom extractor is located here: https://docs.meltano.com/tutorials/custom-extractor.

I'll have a look thanks!

@mathieu_hinderyckx definitely re-writing the tap using the SDK will be more robust but I'm still curious how close this tap is to working. I added a fix for that start_date bug you found. If you want you can try my fork https://github.com/pnadolny13/singer-tap-postmark by updating your pip_url in your meltano.yml to

git+<https://github.com/pnadolny13/singer-tap-postmark.git>

and running

meltano install extractor tap-postmark --clean

.

When i tried using this tap, i couldnt make it work. Using the SDK was imo much faster. So I just created a simple custom postmark tap

Nice! Let us know once you've got it working if its a public repo that we can list on the hub to help out the next person who needs this tap

I am at the moment quite busy with some work, so if people are waiting for this, this might take a while. Will also be taking off for 2 weeks aswell during christmas. @mathieu_hinderyckx are you waiting for this to be public?

I had some holiday time off as well, apologies for the late reply. I am waiting for this yes, was planning to implement something myself using the SDK. I have however never done so, so the initial ramp-up takes a while. If you have a public tap somewhere, we're willing to help with coding as well, joined forces are better than reinventing the wheel 🙂 . I'll see if and where I get with this and if I have something working I'll put it online as well.