https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
p

prratek_ramchandani

10/18/2021, 3:24 PM
has anyone used a singer tap to extract gzipped csv files from s3? i’ve used the pipelinewise tap-s3-csv for uncompressed files and see this open PR for zip file support but am not sure how i’d go about extending that to support gzip files. from my understanding, the challenge is to replace this call to
get_file_handle()
with something that can take an s3 path and return a decompressed file stream without having to first write the decompressed csv file somewhere. any ideas? this SO answer looks promising but i was scared off by the fact that i don’t really understand what it’s doing lol