Hello, I'm using this tap: https://github.com/MeltanoLabs/tap-universal-file My config:

config:
      file_type: jsonl
      file_path: input-jsonl/
      protocol: file
      additional_info: true
    metadata:
      "*":
        replication-method: INCREMENTAL
        replication-key: _sdc_last_modified

After running

meltano run tap-universal-file target-jsonl

the data is appended to the output, the state in meltano.db also not updating the replication key value. The state in meltano.db:

dev:tap-universal-file-to-target-jsonl|{}|{"singer_state": {"bookmarks": {"file": {"replication_key": "_sdc_last_modified", "replication_key_value": "2024-08-21T10:25:23+00:00"}}}}|2024-08-22 11:17:13.316437

Am I missing something?

can you share your full meltano.yml I bet you're missing a capability

yes sure, here is my meltano.yml

version: 1
default_environment: dev
project_id: cccf1e40-6522-4772-a813-f6a1e30bc954
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-universal-file
    namespace: tap_universal_file
    pip_url: git+<https://github.com/MeltanoLabs/tap-universal-file.git>
    capabilities:
    - state
    - catalog
    - discover
    - about
    - stream-maps
    settings:
    - name: stream_name
    - name: protocol
    - name: file_path
    - name: file_regex
    - name: file_type
    - name: compression
    - name: additional_info
      kind: boolean
    - name: start_date
      kind: date_iso8601
    - name: delimited_error_handling
    - name: delimited_delimiter
    - name: delimited_quote_character
    - name: delimited_header_skip
      kind: integer
    - name: delimited_footer_skip
      kind: integer
    - name: delimited_override_headers
      kind: array
    - name: jsonl_error_handling
    - name: jsonl_sampling_strategy
    - name: jsonl_type_coercion_strategy
    - name: avro_type_coercion_strategy
    - name: parquet_type_coercion_strategy
    - name: s3_anonymous_connection
    - name: AWS_ACCESS_KEY_ID
      kind: string
      sensitive: true
    - name: AWS_SECRET_ACCESS_KEY
      kind: string
      sensitive: true
    - name: caching_strategy
    config:
      file_type: jsonl
      file_path: input-jsonl/
      protocol: file
      additional_info: true
    metadata:
      "*":
        replication-method: INCREMENTAL
        replication-key: _sdc_last_modified

  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl

your meltano.yml file looks correct. If you put a new file on the s3 bucket (with a new modified date) does the date update as you'd expect?

i'm just trying to replication sync jsonl files from 1 folder to another folder and will apply dbt later, we don't plan to use on s3

so if you put a new file in the input-jsonl/ directory does it update the date as you'd expect?