Hi 👋 I wondered if someone might be able to help? is it possible to set the start point for an incremental replication in the

meltano.yml

at the entity level? I basically want to default the first bookmark to a particular value for a specific entity. I am using the Wise variant of the MySQL extractor and I am declaring the metadata for an entity like so. Looking at the Singer spec there doesn’t appear to be a metadata key to declare a start point.

metadata:

   db_name-table_name:
    replication-method: INCREMENTAL
    replication-key: id
    table-key-properties:
    - id

Found documentation relating to the use of the tap via Pipelinewise and the following pattern is possible, but not sure how I would implement this in the

meltano.yml

- table_name: "table_three"
        replication_method: "LOG_BASED"
        sync_start_from:                   # Optional, applies for then first sync and fast sync
          column: "column_name"            # Column name to be picked for partial sync with incremental or timestamp value
          static_value: "start_value"      # A static value which the first sync always starts from column >= static_value
          drop_target_table: true          # Optional, drops target table before syncing. default value is false

Many thanks 🙂

Hey @AndyPeter Dunn! There's no clean way to do it as far as I know. One approach I would try, is to seed a state

initial_state.json

file with the desired bookmarks:

meltano state set <you-state-id> --input-file ./initial_state.json

The problems with that approach are: 1. It's specific to a pipeline via the state ID, e.g.

dev:tap-mysql-to-target-bigquery

2. You need to know the structure of the state file. It's usually something like

{"singer_state": {"bookmarks": {"items": {"replication_key": "updated_at", "replication_key_value": "2024-01-01T00:00:00+00:00"}}}}

Hi @Edgar Ramírez (Arch.dev) thank you very much for your help 🙂