Meltano

When forking a branch, what's the best way to test in a Meltano pipeline before raising a PR?

Set a local editable `pip_url` to your package

```pip_url: -e <path>```
and then

```meltano install <type> <name>```
e.g.

`meltano.yml`
```version: 1
default_environment: dev
project_id: 70b74e31-2644-4c45-ba10-c064290af478
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-spotify
    variant: matatika
    pip_url: -e /home/reuben/Documents/taps/tap-spotify```
```meltano install extractor tap-spotify```
To test a loader, run any tap and capture the output

```meltano invoke <tap> > tap.out```
and then run

```cat tap.out | meltano invoke <target>```

Perfect! Thank you very much this is exactly what I was looking for.

Hi <@U06D0USC2CQ>, when I add the local editable pip url to my package and run `meltano install` I get the following error:

```Loader 'target-bigquery' is not known to Meltano. Try running `meltano lock --update --all` to ensure your plugins are up to date.` ```


```loaders:
  - name: target-bigquery
    variant: myname
    pip_url: -e path_to_the_forked_repo_on_my_machine```

Maybe try

```rm -rf .meltano/loaders/target-bigquery/
```
and then installing again?

No luck unfortunately. I'm wondering if I should be installing this like a custom plugin?

That `tap-spotify` example installed fine for me and I was able to `meltano invoke` without any issues... I'll try with `target-bigquery`.

Is it okay if I name the variant whatever I like if I specific a local path for the pip url?

I'm not sure - try keeping it the same (I didn't change it in my example).

Thanks a lot. I can see the results of my contribution in the terminal :grinning:

Yeah, it's probably falling back to the `.lock` file for the base plugin definition - effectively you are just providing overrides in the `meltano.yml`. Maybe by changing the `variant` it loses its reference to the `.lock` file (since it's not a custom plugin).

FWIW, yes the lock file is resolved with the plugin and variant name. So, if you change the variant name, a different lock-file name is expected.