Hi Everyone, I’ve forked the default tap-mssql and added a new feature: CDC on Views. You can find the details and implementation in the repository below: GitHub Repo: pipelinewise-tap-mssql - CDC on Views Why This Feature Was Created: In our organization, we stream data to BigQuery using views that include joins and then some calculated columns based on the secondary table. This streaming is currently handled by a custom .NET application using change tracking, which has a delay of 15-20 minutes. The process involves generating CSV files and performing merges, adding to the latency. While working on a proof of concept (POC), I found Meltano to be much faster. By running sync operations immediately after the previous run completes, we can achieve near real-time data updates. The Challenge: CDC (Change Data Capture) in MSSQL does not natively work with views, which created a roadblock for our use case. The Solution: To overcome this, I forked the tap-mssql repository and implemented custom logic to enable CDC on views. The approach involves: • Extracting CDC data from the primary table associated with the view. • Joining this CDC data with the view to fetch only the latest changes. • Delivering data directly from the view while ensuring that calculated columns and transformations are preserved. This enhancement allows me to leverage CDC from the Primary Table and Actual Data from views and achieve near real-time data streaming without the delay associated with the previous solution. Feedback and suggestions are welcome! Do let me know if any thing can be done better.

Thanks for sharing @Abhishek Singh! I've added this to the Office Hours board 🙂 https://github.com/orgs/meltano/projects/9

@Edgar Ramírez (Arch.dev) Cool 🙌 Do let me know if i have made mistakes in the implementation as this is the first time i have contributed to a open source project And I'm kinda new to DE.