I am using tap-kustomer and target-s3 and getting ...
# troubleshootingp
prakhar_srivastava
09/28/2023, 3:18 PMI am using tap-kustomer and target-s3 and getting duplicate records in the stream while using tap-kusotmer and target-jsonl there seems to be no duplicate. Is there a possibility if the target is throttling (when a tap is making more records than the target can sink) it generates duplicate records?
prakhar_srivastava
09/28/2023, 3:19 PMI have verified the pagination logic and also the target-jsonl output, can't seem to figure this one out
prakhar_srivastava
09/29/2023, 1:33 PMNo I see something is weird I can see the same duplicate record in jsonl too
prakhar_srivastava
09/29/2023, 1:33 PMbut not all the records are duplicated and the number of duplicated records are different every run
prakhar_srivastava
09/29/2023, 1:38 PMI'm manually checking kustomer api
prakhar_srivastava
09/29/2023, 2:36 PMturns out the api is returning right records
prakhar_srivastava
09/29/2023, 2:36 PMno duplicates
prakhar_srivastava
09/29/2023, 3:39 PMokay so I was able to identify that API is sending duplicate records erractically
prakhar_srivastava
09/29/2023, 3:39 PMPAIN
prakhar_srivastava
09/29/2023, 3:43 PMmessage has been deleted