https://meltano.com/ logo
#announcements
Title
# announcements
f

full-planet-99276

12/17/2020, 8:31 PM
Is it possible to have two bigquery loaders in meltano?
1
r

ripe-musician-59933

12/17/2020, 8:40 PM
@full-planet-99276 Right now, the closest you can get is to configure the same loader differently at runtime using https://meltano.com/docs/integration.html#pipeline-specific-configuration, but having multiple instances of the same plugin with different configurations in your project at the same time will be made possible by https://gitlab.com/meltano/meltano/-/issues/2390, which I hope to still get out in a release this week!
🙌 1
If you can tell me what exactly you're trying to accomplish by having two bigquery loaders, I can help you find the best way to realize that right now
f

full-planet-99276

12/17/2020, 8:44 PM
I have two sources, one is gitlab, another is a PG database, I want each to go to their own dataset.
r

ripe-musician-59933

12/17/2020, 8:46 PM
All right! Did you see the default value under https://meltano.com/plugins/loaders/bigquery.html#dataset-id? If you leave it unset, it will automatically use a different dataset for each extractor
By default, that will be the namespace of the extractor:
tap_gitlab
and
tap_postgres
. If you'd like to change that, you can set a different load_schema on the extractors: https://meltano.com/docs/plugins.html#load-schema-extra
@full-planet-99276 Can I assume you figured this one out?
f

full-planet-99276

12/18/2020, 4:26 PM
Got it
thank you
r

ripe-musician-59933

12/18/2020, 4:27 PM
👍
https://gitlab.com/meltano/meltano/-/issues/2390 took me a little longer to get over the finish line than expected, but I promise it was worth the wait: https://meltano.slack.com/archives/CP8K1MXAN/p1608746095055200