I am using `tap-s3-csv` to load the data from a cs...
# troubleshooting
s
I am using
tap-s3-csv
to load the data from a csv file in a s3 bucket to a table in Postgresql. After I copied a new CSV file to the bucket, for some reason,
tap-s3-csv
doesn't load the new columns. I ran the command in my local env. Dose any one know how I can fix the issue?
Copy code
- name: tap-s3-csv
    variant: transferwise
I used the debug log level, and I can't see the missing fields in the SCHEMA of the stream.
I found the issue. meltano run created tap.properties.json file under .meltano/run/tap-s3-csv. I have to manually delete it if the CSV file was changed. --full-refresh doesn't help on this issue. Is this the right way to do?
u
@sean_han that was what I was about to recommend, sounds like you figured it out yourself 👍 . Sometimes building that tap.properties.json file is expensive so meltano caches it for you. I just created https://github.com/meltano/meltano/issues/7899 feel free to add you thoughts and experience in there if you'd like
s
got. it, thanks, @pat_nadolny