Hi guys, I struggle with tap-taboola extractor. I have this tap-taboola configuration in the project's meltano.yml

config:
      account_id: ${TAP_TABOOLA_ACCOUNT_ID}
      client_id: ${TAP_TABOOLA_CLIENT_ID}
      client_secret: ${TAP_TABOOLA_CLIENT_SECRET}
      username: ${TAP_TABOOLA_USERNAME}
      password: ${TAP_TABOOLA_PASSWORD}
      start_date: ${MELTANO_VAR_START_DATE}
      end_date: ${MELTANO_VAR_END_DATE}
      report_type: campaign_summary
      dimensions:
      - campaign
      - day
      metrics:
      - clicks
      - impressions
      - spend

The ${} configurations are all stored in Airflow variables as i run an Airflow DAG to pull data from taboola and store it in mysql using target-mysql. However, i noticed that the end_date here doesn't matter. It always pull data from what I defined in start_date until the very last day. Then, i checked the repository on github, I found that the params look like this in _`__init__.py`_:

params = {
        'start_date': state.get('start_date', config.get('start_date')),
        'end_date': datetime.date.today(),
    }

This perhaps explains why end_date config doesn't matter. But, I forked the repo from singer-io, and I changed it to:

params = {
        'start_date': state.get('start_date', config.get('start_date')),
        'end_date': state.get('end_date', config.get('end_date')),
    }

Then, I changed the pip_url to my forked repo in the project's meltano.yml. However, this didn't solve the issue. the end_date is still being recorded as today in the log. How do I change the end_date to follow the configuration and not always today?

Did you do

meltano install

after you updated the

pip_url

?