Hello! I'm new to meltano and just learning how to...
# singer-taps
o
Hello! I'm new to meltano and just learning how to use it, it seems my team set up a tap for adyen data that has started failing as of recently. It seems the repo has been removed? Do you have any suggestions on ways to work around this for adyen-tap? https://hub.meltano.com/extractors/tap-adyen/
1
a
There is a variant here - not sure if that helps? https://github.com/Yoast/singer-tap-adyen
1
🙏 1
r
PR to remove the broken plugin from Meltano Hub so no-one else hits this issue: https://github.com/meltano/hub/pull/2137
github merged 1
i
Thanks a lot Andy and Reuben. We will check this and get back if further help needed 🙏
np 1
Hello again, The extractor test fails somehow. I checked the plugins several times (my password is also correctly stored in .env file). Can you perhaps help me understanding this error message? debug does not work neither.
Copy code
plugins:
  extractors:
  - name: tap-adyen
    variant: yoast
    pip_url: git+<https://github.com/Yoast/singer-tap-adyen.git>
    config:
      company_account: S*******B
      merchant_account: S******M
      report_user: r***********B
      start_date: '2023-01-01'
      test: false
    select:
      - settlement_details.*
r
Do you have any state for that tap?
Copy code
meltano state list
meltano state get <state_id>
i
Since the extractor tap doesn't run successfully,
meltano state list
does not generate any result.
r
Yeah, it's a bug in the tap then: https://github.com/Yoast/singer-tap-adyen/blob/2a65e1df50b8bd13e7291406198bd4a084ef63dc/tap_adyen/tools.py#L22-L36 That will return
None
if there is no state for any given stream, and later in the code it tries to unpack (
**
) that
None
value as if it were a dict (mapping): https://github.com/Yoast/singer-tap-adyen/blob/2a65e1df50b8bd13e7291406198bd4a084ef63dc/tap_adyen/sync.py#L69 The fix would be to fork the tap and patch
tools.get_stream_state
to always return a dict, or hand-crank a state for the tap to use that has an existing bookmark entry for the streams you want to sync, i.e.
Copy code
{
  "bookmarks": {
    "settlement_details": {}
  }
}
You might want to try using one of the existing forks - some look like they already have a fix for this issue, e.g. https://github.com/recoord/singer-tap-adyen/commit/a102a4f0ab81aa3275facf72ddec3cd2cc96fe10#diff-949a5d522ae72be372bd4493a258b39b98ab31c8b5c3c031fe11b133f636a4a3L52-R58
Copy code
pip_url: git+<https://github.com/recoord/singer-tap-adyen.git>
i
Thank you! Will try 🙏
It's very strange, I am using recoord url, and looks like it indeed solves that error, but now I am getting this.
No RECORD or BATCH message received
r
Given that you only have
settlement_details
selected, is is possible there is no data for that stream?
i
Not really, we have data. And this report user is already authorized to fetch data from settlement_details. Could it be because in this fork all these 3 schemas changed to "selected":false ?
r
Maybe... What does
Copy code
meltano select tap-adyen --list --all
show?
i
Hmmm, only my selection is selected. So that change in fork should not affect anything but it turns as a default value.
r
You could check the generated catalog:
Copy code
meltano invoke --dump catalog tap-adyen > tap-adyen.catalog.json
👍 1