Hello, Sorry if this is too basic a question. I am trying to use rest API extractor to get data and store to target-postgres. I couldnt figure out what should be configuration of

tap-rest-api-msdk

- appreciate a help.

- name: tap-rest-api-msdk
    variant: widen
    pip_url: tap-rest-api-msdk
    config:
      api_url: <http://api.exchangeratesapi.io/v1/latest>
      pagination_request_style: ''
      pagination_response_style: ''
      pagination_page_size: 10000
      path: ''
      params:
        access_key: xxxxxxxxxxxxxxxxxx
        base: GBP
      records_path: ''
      primary_keys: []
      num_inference_records: 1000
      streams:
      stream_map_config: {}
      flattening_enabled: true
      flattening_max_depth: 10

I am expecting data of this format that I would like to store in target-postgres

{
    "success": true,
    "timestamp": 1620386223,
    "source": "GBP",
    "quotes": {
        "GBPAED": 5.109727,
        "GBPAFN": 107.750768,
        "GBPALL": 141.659078,
        "GBPAMD": 725.255305,
    }
}

why not use https://hub.meltano.com/extractors/tap-exchangeratesapi?

good idea thanks - but it would be great if I can understand this generic API extractor

We should add more examples but did you see https://hub.meltano.com/extractors/tap-rest-api-msdk#examples? That example might answer some questions for you

Finaly I was able to make it work but the endpoint always returned 0 results for exchangeratesapi. I changed the API provider and below configuration seems to work flawlessly for this provider;

- name: tap-rest-api-msdk
    variant: widen
    pip_url: tap-rest-api-msdk
    config:
      api_url: <https://anyapi.io/api/v1/exchange/rates>
      streams:
      - name: currency_dataset
        params:
          apiKey: xxxxxx
          base: USD
    select:
    - '*.*'

thanks all.