#2684 Easier way to resume earlier ELT pipeline by discovering or providing initial replication key values per stream Issue created by MeltyBot Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/2734 Originally created by @ers81239 on 2021-04-28 214612 --- Problem to solve (Summarize the problem we are trying to solve in the form of we need to do [A], so that [B] can [C]) Target audience (For whom are we doing this? Include a persona) Further details As a Meltano user, I want to resume ELT for existing target tables so that I can replace existing ELT tools with Meltano. With Incremental loading, this boils down to telling the extractor to start at the max_replication_value in the destination table. Today, this is very difficult and involves either using elt --dump and elt --state or else manually editing the state in meltano.db. Proposal 1. Whole enchilada: Stop tracking the max_replication_value in the state and instead read it from the loader on every execution. 2. Much easier: Allow max_replication_value (and for completeness min_replication_value) as metadata settings for incremental extractors. This supports all kind of use cases where the user wants to sync a portion of a source for resource/time management reasons. What does success look like, and how can we measure that? Increased adoption due to lower barrier to entry. Regression test N/A? Links / references Please note that this was taken from GitLab, to be changed accordingly meltano/meltano