Hello, I am looking for a generic tap to reach out to a Rest API to retrieve the JSON payload. I have come across the following singer tap https://github.com/anelendata/tap-rest-api or on pypi here https://pypi.org/project/tap-rest-api/ . It looks like it might possibly meet my needs of pushing content from that API location to Snowflake. Is there a generic api available on the Meltano Hub already for doing this or would I need to use this tap? Second question if I do need to use this tap, how could I add this to my meltano project? I have installed standard taps e.g. Salesforce and custom taps e.g. tap-oracle but I'm not sure how to add this if it is not on the Meltano Hub. I did try adding this in my meltano.yml extractors: - name: tap-rest-api--covid-tracer-summary inherit_from: tap-rest-api pip_url: git+https://github.com/anelendata/tap-rest-api But it returned an error message : Could not find parent plugin for extractor 'tap-rest-api--covid-tracer-summary': Extractor 'anelendata/tap-rest-api' is not known to Meltano Any help pointing me in the right direction either to an existing tap on the hub I could use or how I could install this singer tap would be appreciated. Thanks Steve

Hi Steve, I think that since tap-rest-api isn't one of the extractors listed as "supported out of the box" at hub.meltano.com, that you must register it as a custom extractor.

Hi @steve_clarke! Even if the tap is in the Hub, you should still be able to add it to your project. You'll need to declare capabilities and settings along with that pip URL, The easiest way to add it is with the CLI prompts:

meltano add --custom extractor tap-rest-api

After filling in the prompts, you should end up with something like this in `meltano.yml`:

extractors:
  - name: tap-rest-api
    capabilities:
      - discover
      - state
      - catalog
    settings:
      - name: url
      - name: timestamp_key

This particular tap may require additional CLI options (

--infer_schema

,

spec_file

) that will not work with Meltano, though

Just to also throw out here — writing a new REST tap using the meltano SDK is very straightforward!

@stephen_bailey indeed! 😄

Thank you very much for the quick responses. I have been able to install the extractor plugin now. I will have a go and see if I am able to extract what I need without the --infer_schema and spec_file options.

you might try using our version of tap-rest-api. https://github.com/Widen/tap-rest-api we haven’t listed it on the hub yet, but we built it using the SDK and borrowed much of the idea from anelendata. We’ve been using it in production already for some of our internal apis.