Are there any existing s3 taps that work with JSON data? Or would I have to write my own? I see this one here but looks like it is Docker-based and we run ECS so not sure if that will be a problem in our deployment.
giancarlo_perrone
06/15/2023, 5:31 PM
This brings up another question for yall, how would I be able to write a tap that sort of ignores any schema, in the case that I am trying to make a generic s3Json tap that should be able to handle any json file? I saw something about dynamically creating the schema but is that the only option here?
t
taylor
06/15/2023, 5:38 PM
@visch is actually working on a more generic
tap-file
with initial support for S3 and CSV/JSON
g
giancarlo_perrone
06/15/2023, 5:40 PM
Awesome! I would love to follow the development there if possible