ian_lewis
11/09/2023, 12:47 PMtap-singer-jsonl -> target-redshift
and although I have the S3 State Backend configured it doesn't seem to be used.
Every run of the pipeline results in all of the files in the S3 bucket being read.
Anyone else encountered this?edgar_ramirez_mondragon
11/09/2023, 6:35 PMian_lewis
11/10/2023, 10:39 AMian_lewis
11/13/2023, 2:47 PMtap-spreadsheets-anywhere
- the caveat being that I can't seem to use any environment variables to modify the config for different source environments.