I've got a container deployed that runs a single p...
# best-practices
d
I've got a container deployed that runs a single pipeline every hour in incremental mode. Recently we've updated the source data definition and meltano/singer has picked up the new column and on all new records, the extra properties have come through However I need historical data updated. I understand there is a
--full-refresh
switch I that I can provide on the CLI to force a full refresh, but not sure what the best practice is when I am running the process in a container.... I don't have SSH access to the container (only the UI)
Current thinking: 1. Pause the airflow scheduler 2. Run the CLI command manually from another instance 3. Restart the scheduler
d
@david_tout That’s the recommended approach right now, but a better solution is discussed in https://gitlab.com/meltano/meltano/-/issues/2568