Hi, quick question about target development. If I ...
# announcements
d
Hi, quick question about target development. If I understand correctly meltano will save state object for each job (with job id as unique ref). If the tap emits multiple STATE messages (one for each stream, for example), should I merge those state messages and emit single state message? Or can meltano handle multiple state messages? Thanks!
d
@dan_aharoni Your target can directly forward the STATE messages coming from the tap once all of the RECORDs up to that point have been persisted, without merging, because the tap already makes sure each next STATE message contains at least the bookmarks of the previous one. Meltano will update the pipeline's state in the system database whenever it receives a new STATE message from the target, so that the pipeline can be interrupted and restarted at any point. If the pipeline succeeds, only the final STATE message will remain in the DB.
d
@douwe_maan Perfect. Thank you so much for the detailed answer!