Hi all ! my company has accounts on social media platforms like whatsapp, reels (instagram), IGTV and other streaming platforms that do not have dedicated APIs. Any pointers to how can I get the data  to measure the platform performance ? Is web scraping through python the only solution ??

I don’t have any experience pulling from those sources, but without a dedicated API it’s going to be tough (and might be against their terms and conditions).

If the platforms offer reporting on the site, then you can use chrome dev tools to see if they're calling a backend API to pull the reporting data. If they are, then you can try passing the same request headers to the backend API url to see if it'll return the data. We've had mixed success implementing this for clients. Sometimes the auth the backend APIs use changes and is annoying to update on your request, but other times it isn't a problem. Like Taylor said, be sure to check their terms and conditions first. Lmk if you have any questions on using this method!

This thread is a blast from the past 😅 😄

oh shit, I saw it was from february and didn't see it was 2 years ago! I was searching to see if anyone had connected a tool like Scrapy with Meltano and found this thread haha

evergreen content here on the Meltano slack 🫡 😄