Just playing.... Trying <https://hub.meltano.com/e...
# plugins-general
j
Just playing.... Trying https://hub.meltano.com/extractors/tap-exchangeratehost/ Works like a charm until complete history is loaded (per 366 days in each meltano execution). Then next meltano run fails:
Copy code
2023-02-07T19:26:23.469910Z [info     ] INFO {"base": "EUR", "start_date": "2023-02-08", "end_date": "2023-02-07"} cmd_type=elb consumer=False name=tap-exchangeratehost producer=True stdio=stderr string_id=tap-exchangeratehost
.....
2023-02-07T19:26:23.584435Z [info     ]     for rate in response["rates"][start_date]: cmd_type=elb consumer=False name=tap-exchangeratehost producer=True stdio=stderr string_id=tap-exchangeratehost
2023-02-07T19:26:23.584518Z [info     ] KeyError: '2023-02-08'         cmd_type=elb consumer=False name=tap-exchangeratehost producer=True stdio=stderr string_id=tap-exchangeratehost
Is it caused by Meltano or by the tap? It seems that the request is made for the day after today, which is obviously not found in the result payload...
a
@jan_soubusta - I just dove into the source code. Start date defaults to 'today' basically. End date defaults to something like 'today+1' - except it looks like it is not excluding 'tomorrow' properly, and likewise not gracefully handling 'tomorrow' not being in the returned data set.
Providing an explicit end_date should (probably) solve this. And it may be that something about timezones and datetime math is causing this to fail for you when (presumably) others have run it without running into the issue.
Additional checks here could probably fix the issue in the tap itself.
j
Will try to create a PR, we'll see if the owner is still activate and will merge it