Hey all! I have a custom tap already working with incremental replication from an API. Without affecting the incremental state, I would like the ability to occasionally do a one-off backfill for a specific date range. Something like this:

meltano run tap-mycustomtap target-postgres --start '2024-01-01' --end '2024-01-08'

I know in my tap I can specify start/end params to the API in the Stream

get_url_params

. Is there a way to somehow provide start/end values via the meltano CLI to the tap so they’re available in

get_url_params

?

Easiest solution I think is to add end date to the tap as a config, run it with a separate run id and you're good to go

+1 on making them tap settings, and maybe using

TAP_MY_CUSTOM_TAP_START_DATE=2024-01-01 TAP_MY_CUSTOM_TAP_END_DATE=2024-01-08 meltano run --no-state-update tap-mycustomtap target-postgres

You don't want it to use the last state either though so that's close but not quite right I think

Ah, so also

--full-refresh

That sounds perfect, thanks! I’ll give it a try.