https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
m

michael_horvath

03/06/2023, 9:22 PM
I would like to get a sanity check for a tap I'm considering developing. I've not done one before, and I wanted to see if there were any red flags or get a little affirmation that I'm on the correct path. I want to add the data from a series of xml files I have locally. Following https://docs.meltano.com/tutorials/custom-extractor, I intend to create a tap using the cookiecutter from the meltano sdk, stream type other, no auth. Input should include a path (dir or file), keys, and an optional xsl to transform the xml to something 'flat'. The plugin should support key-based replication, incremental replication (by date of most recently processed file). I will rely heavily upon pandas to use their read_xml function to process the xslt and form the data into rows.