I'm trying to test some changes I had made to wintersrd variant of tap-mssql. It's not built off of the SDK, and I think I may be running some issues with configuration. I have tested discover separately and it seems to work. However, after adding it to my meltano project, I get

Could not find catalog.

as an error.

2024-09-12T22:45:13.638073Z [debug    ] Failed to submit unstruct event to Snowplow, error err=Traceback (most recent call last):
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\invoke.py", line 226, in dump_file
    content = await invoker.dump(file_id)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\core\plugin_invoker.py", line 520, in dump
    raise err.__cause__ from None  # noqa: WPS469, WPS609
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\core\plugin_invoker.py", line 435, in _invoke
    yield (popen_args, popen_options, popen_env)
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\core\plugin_invoker.py", line 515, in dump
    return self.files[file_id].read_text()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\py\cpython@3.12.4\Lib\pathlib.py", line 1027, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\py\cpython@3.12.4\Lib\pathlib.py", line 1013, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\git\\dagster-hybrid\\src\\elt_projects\\meltano\\.meltano\\run\\tap-mssql\\tap.properties.json'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\invoke.py", line 137, in invoke
    exit_code = await _invoke(
                ^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\invoke.py", line 177, in _invoke
    await dump_file(invoker, dump)
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\invoke.py", line 228, in dump_file
    raise CliError(f"Could not find {file_id}") from err  # noqa: EM102
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
meltano.cli.utils.CliError: Could not find catalog

My meltano.yml

plugins:
  extractors:
  - name: tap-mssql
    namespace: tap-mssql
    pip_url: -e custom-plugins/taps/tap-mssql
    executable: tap-mssql

Initially, I thought it was because of the properties capability being required due to the readme having a properties example for debugging, but I do see that catalog is listed on meltano hub, and it's handled in the code, and that the meltano docs say that it's a default capability. Any ideas on the error or how would I continue investigating this? The command I was trying to run

meltano invoke --dump=catalog tap-mssql > catalog.json

You should definitely add at least these capabilities:

capabilities:
- catalog
- discover
- state

Is this a custom plugin, or did you add the wintersrd one and then changed the pip_url?

I cloned wintersrd, made some changes, then copied it to that directory specified in pip_url and added the meltano.yml entry manually

Yup

Hmm, interesting. That might've been it. I get a different error now. I have another extractor defined with inherit_from: tap-mssql, and that seems to not inherit the env vars. So I tried copying the settings as well.

plugins:
  extractors:
  - name: tap-mssql
    namespace: tap-mssql
    pip_url: -e custom-plugins/taps/tap-mssql
    executable: tap-mssql
    capabilities:
    - catalog
    - discover
    - state
    settings:
    - characterset: test
    - cursor_array_size: test
    - conn_properties: test
    - database: test
    - enable_tds_logging: test
    - filter_dbs: test
    - host: test
    - password: test
    - port: test
    - tds_version: test
    - use_date_datatype: test
    - use_singer_decimal: test
    - user: test
    - filter_objects: test

Then got this one:

is_extra
Traceback (most recent call last):
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\core\behavior\canonical.py", line 271, in __getattr__
    value = self._dict[attr]
            ~~~~~~~~~~^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\ruamel\yaml\comments.py", line 853, in __getitem__
    return ordereddict.__getitem__(self, key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'is_extra'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\__init__.py", line 103, in _run_cli
    cli(obj={"project": None})
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\cli.py", line 46, in main
    return super().main(*args, windows_expand_args=False, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\utils.py", line 662, in invoke
    super().invoke(ctx)  # type: ignore[misc]
    ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\utils.py", line 662, in invoke
    super().invoke(ctx)  # type: ignore[misc]
    ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\utils.py", line 711, in invoke
    super().invoke(ctx)
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\config.py", line 259, in list_settings
    full_config = settings.config_with_metadata(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\core\settings_service.py", line 233, in config_with_metadata
    value, metadata = self.get_with_metadata(
                      ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\core\settings_service.py", line 342, in get_with_metadata
    if setting_def and setting_def.is_extra:
                       ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\core\behavior\canonical.py", line 276, in __getattr__
    raise AttributeError(attr) from err
AttributeError: is_extra

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\daniell\.rye\tools\meltano\Lib\site-packages\meltano\cli\__init__.py", line 113, in _run_cli
    raise CliError(f"{troubleshooting_message}\n{err}") from err  # noqa: EM102
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
meltano.cli.utils.CliError: Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

is_extra
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

is_extra