I am having trouble getting incremental state to w...
# troubleshooting
j
I am having trouble getting incremental state to work with tap-facebook varient: airbyte (tap-airbyte-wrapper) I am saving state in a GCS bucket which is working as expected. But each day the tap is doing a full refresh and takes about 6 hours. I have many duplicate JSONs in my Data Warehouse Example State entry:
Copy code
{
  "type": "STREAM",
  "stream": {
    "stream_descriptor": {
      "name": "ads_insights_age_and_gender",
      "namespace": null
    },
    "stream_state": {
      "REDACTED": {
        "date_start": "2024-12-02T00:00:00+00:00",
        "slices": [],
        "filter_statuses": [
          "ACTIVE",
          "ADSET_PAUSED",
          "ARCHIVED",
          "CAMPAIGN_PAUSED",
          "DELETED",
          "DISAPPROVED",
          "IN_PROCESS",
          "PAUSED",
          "PENDING_BILLING_INFO",
          "PENDING_REVIEW",
          "PREAPPROVED",
          "WITH_ISSUES"
        ]
      },
      "time_increment": 1
    }
  }
}
Meltano yaml extractor:
Copy code
- name: tap-facebook
    variant: airbyte
    pip_url: git+<https://github.com/MeltanoLabs/tap-airbyte-wrapper.git>
    config:
      airbyte_config:
        start_date: ${START_DATE}T00:00:00Z
    select:
    - ads_insights_*.*
    - ad_account.*
    - ad_sets.*
    - ads.*
    - campaigns.*