Has anyone considered a very generic tap for microsoft's common data model?
https://learn.microsoft.com/en-us/common-data-model/model-json
I'm working a lot with Synapse Link data from dynamics, so I've probably built something which does 90% of the job without realising what I was doing.
👀 1
v
visch
09/26/2024, 1:06 PM
Had a customer I talked to a while back think about it
visch
09/26/2024, 1:06 PM
iirc it was also for dynamics data
a
Andy Carter
09/26/2024, 2:00 PM
Yes, its basically a json schema and unheadered csv files partitioned by entity type and month-year. So
tap-spreadsheets-anywhere
but all of the non-Azure bits cut out.
âž• 1
v
visch
09/26/2024, 2:04 PM
I've implemented this pattern for someone who was dumping mssql data (i think?) as well. Similar idea, there's a schema and a folder structure. State should keep track of each table individually so you only pull the latest changes on each run