Hey guys, how can i setup multiple profiles for ta...
# announcements
k
Hey guys, how can i setup multiple profiles for taps and targets using the cli and setting up those profiles config at meltano.yml?
m
You should first create the
profile
in the
meltano.yml
Copy code
plugins:
  extractors:
    - name: …
      pip_url: …
      profiles:
        - name: test
          label: Test
          config: {}   # this is optional, you can set variables using ENV or the UI/CLI
At the CLI level, you can use
meltano config <extractor_name>@<profile_name>
to set/view/update the configuration values
As such, you can run an ELT using:
Copy code
meltano elt <extractor>@<profile> <loader>
It also applies to Loaders.
k
Thank you!
Im running into a problem that multiple taps are using the same target (postgres) and when i look into my database, the raw data gets crazy and the data extracted goes to the wrong schemas, this is expected to happen if i use the same target for multiple taps at the same time?
m
We are currently fixing that with the following issue, which will bring isolation to each profile
Basically the current approach doesn't isolate data sets, which might or might not cause what you are seeing, depending on the data set. With this issue resolved, each Pipeline will be isolated from each other.
You can workaround that by using multiple loaders, on different databases, for now.
k
Thank you! That was clarifying! We will probably use that approach until the issue get resolved, if we find something useful we will bring it to you guys