Hello to everyone, I’m quite new to these “meltano-world”. Sorry if this is not the place to make this question, but I’m looking for some advice on what’s the best way to solve the task I have in hand. I was asked to develop an administration site for a Marketing agency that handles several accounts (MCC on Google and Business account on FB). Data: Google Ads, Google Analytics and Facebook Ads (*choices). I was looking to send all the accounts data to Google BigQuery and use it’s API to consult that information in the reports on my site, but I came across Meltano so now I’m trying to figure out my best choice to implement this reporting system. Meltano offers to implement the pipeline of getting the data from my sources choice (*choices) and saving them in my targets choice (BigQuery), but in the docs I saw there’s a Meltano Dashboard UI which offers a dashboard to generate reports. The loader is PostgreSQL, so all the data is stored there? As I have several accounts to connect with I thought BigQuery was a good choice. I don’t want to make this longer that already is. Any help/advice given is really appreciated. Thanks!

Hey Pedro, you have the right idea. Get all the data into one database so that you can write reports on top of it

Adam thank you for the reply! I’m still trying to find the right option for the project, because the timeframe is quite short and we’re lacking developers for implementation. So, I’m also looking for a payed solution, any recommendations? (Supermetrics, Fivetran, ...) So to get the data in GBQ and use the developing time on better reporting site. And latter on implement metlano or singer solution. Just one more question, what’s the average metlano pipeline implementation time? It’s as easy as the demo app shows? Running in a server and leave the pipelines running?

I don't know Supermetrics. I have heard good things about Fivetran. We use Stitch (which uses Singer behind the scenes), and I hope we can start using Meltano

@pedro_barri I agree with @adam_roderick that Meltano deserves more consideration for its ELT functionality than its BI/Analytics functionality in your scenario, although the UI is probably worth at least a look: . I would suggest running

meltano invoke airflow scheduler -D

on a server, making sure it doesn't die, and then additionally running

meltano invoke airflow webserver -D

so you can interact with your pipelines and their history (logs etc) through Airflow's interface at http://localhost:8080. In that scenario, your setup will be as reliable as the combination of your server, Airflow (very reliable),

meltano elt

(reasonably reliable), and the specific tap and target you're using (tap-adwords, tap-facebook and tap-google-analytics are all reasonably reliable, but target-bigquery I cannot speak to). I think that setup is at least worth a try as opposed to shelling out for one of the off-the-shelve solutions, but it's important not to forget that at this stage of the project, you get what you pay for.

Thanks Douwe for your answers! By multiple profiles do you mean, for instance a Google MCC account which handles multiple accounts? That’s not possible, right now with Meltano?

@pedro_barri Ah, no, that should be OK. If you wanted to set up different pipelines for different MCC accounts, though, that's not currently possible.

Ah, I see, perfect! Then Meltano is a great fit for my use case. Thanks!!

@pedro_barri Great, let me know how it goes!