I wrote a plugin that SCPs a file I call it like this `poetr Meltano #meltano-plugin-development

I wrote a plugin that SCPs a file. I call it like ...

Tanner Wilcox

04/07/2025, 8:42 PM

I wrote a plugin that SCPs a file. I call it like this

poetry run scp get-file tanner password url /remote/path ./

I'm struggling to add this to my meltano project. Here's the utility section of my meltano.yml

Copy code

utilities:
    - name: dbt-postgres
      variant: dbt-labs
      pip_url: dbt-core dbt-postgres meltano-dbt-ext~=0.3.0
    - name: scp-ext
      pip_url: '../scp-ext/scp_ext'
      executable: scp

I get this error:

Copy code

[tanner@sato ubb-meltano]$ meltano lock --update --all
Utility 'scp-ext' is not known to Meltano. Check <https://hub.meltano.com/> for available plugins.

Tanner Wilcox

04/07/2025, 8:47 PM

Copy code

[tanner@sato ubb-meltano]$ meltano add --custom utility scp-ext
Adding new custom utility with name 'scp-ext'...

Specify the plugin's namespace, which will serve as the:
- identifier to find related/compatible plugins

Hit Return to accept the default: plugin name with underscores instead of dashes

(namespace) [scp_ext]: 

Specify the plugin's `pip install` argument, for example:- PyPI package name:
	scp-ext
- Git repository URL:
	git+https://<PLUGIN REPO URL>.git
- local directory, in editable/development mode:
	-e extract/scp-ext
- 'n' if using a local executable (nothing to install)

Default: plugin name as PyPI package name

(pip_url) [scp-ext]: ../scp-ext

Specify the plugin's executable name

Default: name derived from `pip_url`

(executable) [scp-ext]: scp
Utility 'scp-ext' is not known to Meltano. Check <https://hub.meltano.com/> for available plugins.

Tanner Wilcox

04/07/2025, 8:58 PM

I was missing a namespace

Edgar Ramírez (Arch.dev)

04/07/2025, 10:41 PM

Glad you figured it out @Tanner Wilcox! We should make the first error message more useful: https://github.com/meltano/meltano/issues/9199.

Tanner Wilcox

04/08/2025, 4:17 PM

I'm getting a similar error when trying to run a script from meltano.yml:

Copy code

plugins:
  extractors:
    - name: tap-spreadsheets-anywhere
      variant: ets
      pip_url: git+<https://github.com/ets/tap-spreadsheets-anywhere.git>
  loaders:
    - name: warehouse
      inherit_from: target-postgres
      variant: meltanolabs
      pip_url: meltanolabs-target-postgres
  utilities:
    - name: dbt-postgres
      variant: dbt-labs
      pip_url: dbt-core dbt-postgres meltano-dbt-ext~=0.3.0
    - name: scp
      namescape: scp
      commands:
        get_file:
          executable: scp
          args: -h

Copy code

[tanner@sato ubb-meltano]$ mel run scp
2025-04-08T16:16:54.011729Z [info     ] Environment 'dev' is active   
Utility 'scp' is not known to Meltano. Try running `meltano lock --update --all` to ensure your plugins are up to date.
[tanner@sato ubb-meltano]$ mel lock --update --all
Utility 'scp' is not known to Meltano. Check <https://hub.meltano.com/> for available plugins.

I'm following the syntax described here: https://docs.meltano.com/guide/advanced-topics/#running-custom-scripts

Tanner Wilcox

04/08/2025, 4:22 PM

I typed 'namescape' instead of 'namespace'

9 Views

Open in Slack

Previous Next