Hi everyone! Need advice: So I'm working on a tap...
# singer-tap-development
a
Hi everyone! Need advice: So I'm working on a tap that extracts data from API, this API have a specific: • API can be queried with granularity of one day, i.e. I can ask for events starting from 2023-02-24 but not from specific time What I need to do: • Express in stream schema, that stream is incremental • Set incrementality to synthetic date field • Emit bookmark manually when all the records for a specific date are emitted • Iterate by date in REST API calls Questions: • Is RESTStream a good base for such tap? • How do I provide information about date? Do I ingest sythetic field like
event_date
to each event and rely on built-in functionality? • How do I ensure that bookmark is emitted at the end of the batch?