Any advice on how to smoothly handle schema-update...
# singer-targets
k
Any advice on how to smoothly handle schema-updates that lead to pipeline-failures would be much appreciated (specifically with target-bigquery)
v
I'm not very smooth but we altered our tap-oracle to pull
SYS.*
data (information sys stuff) to get basically all the same things singer uses for a catalog. Great because we can use this to find changes in tables/columns and then alter / take actions on those changes. Really depends on what you need to do of course!
k
No you're quite spot-on, this seems like what we have to do as well. Although with tap-hubspot, thanks for the answer!
v
it'd be nice if you could just get the catalog data sent to your DW. There's a few issues about it in Meltano. Meltano elt tap-meltano target-db , there is a tap-meltano out there for the internal DB, if we added the catalog information that would be interesting. https://github.com/Matatika/tap-meltano I know Meltano has some issues in about this too 🤷