Hi, I am trying to use the tap-gitlab with multiple entities in the select and use the target-csv , I am only getting 1 csv from the target. If I comment out other selects, I successfully get thhose csv's but 1 at a time? Thoughts?
a
aaronsteers
10/14/2021, 7:54 PM
Hi, @carolus_holman - (nice to see you! 👋) I think this may be a problem in the target. That target is mostly used for POCs and it might not have ability to save to multiple files. I jumped into the docs and didn't see anything that allows you to save to one file per stream, which is the norm for other similar targets like target-jsonl. Can you use target-jsonl by chance?
aaronsteers
10/14/2021, 7:54 PM
(Others please chime in if there's a way to do this with the existing CSV target.)
c
carolus_holman
10/14/2021, 8:26 PM
The jsonl worked for this use case
carolus_holman
10/14/2021, 8:29 PM
FYI, the commits.* and project.* did extract both, it seems that when you cherry pick the api columns, that's where it doesn't work as intended.
a
aaronsteers
10/14/2021, 11:42 PM
Cool - thanks for letting us know, @carolus_holman.
I was inspired by your inquiry (and past questions about target-csv) to build an SDK-based port of target-csv. It's not quite production-ready yet but in the future we may migrate over to it in order to ensure a more robust handling.