Hey 👋 Is anyone using the SDK with Meltano? I am getting:

FileExistsError: Could not find schema file 'tap_tableau_wrangler/schemas/workbook.json'

even after adding to the

[tool.poetry]

section in `pyproject.toml`:

include = ["tap_tableau_wrangler/schemas/*.json*"]

Apparently poetry's support for package data is a bit patchy 😔 Is there a reason why the SDK cookiecutter defaults to poetry? Might be worth having a

setup.py

option too, especially as Meltano doesn't seem to support editable poetry plugins yet either 🤦‍♂️

@ken_payne - We recently logged https://gitlab.com/meltano/singer-sdk/-/issues/47 which talks about the challenges of wrapping a tap in meltano prior to publishing into pypi or another package repository.

Ah interesting. I have actually just had a poke around the site-packages of my local meltano virtualenv and the schema jsons are now showing up. I also saw somewhere that poetry includes package data by default (though it uses .gitignore to seed exclusions), so they may have been there all along. I started to wonder if the Stream class could be made to at least try and discover package files, rather than just using the relative filepath 🤔

SCHEMAS_DIR = Path(pkg_resources.resource_filename('tap_tableau_wrangler', 'schemas/'))

So I just modified the base path from the cookiecutters

streams.py

to point at the package and it works 🚀 This doesn't solve the editable problem, but it isn't too painful to run

meltano install

after changes, as most testing of the tap is already done at the point I want to add it to meltano and run locally to sense-check.

@ken_payne - I haven’t tested the packaging of the extra json files but I’m glad that is working for you now. Any feedback, tips, or lessons learned which you might want to memorialize into those related issues would be much appreciated. Thanks!

I too had to fix the path so it points to the installed package directory with

SCHEMAS_DIR = Path(__file__).parent / Path("./schemas")

https://github.com/edgarrmondragon/tap-confluence/blob/master/tap_confluence/streams.py#L11

Thanks for sharing this back. I’ve incorporated the fix in the commit: https://gitlab.com/meltano/singer-sdk/-/commit/e91724fd61120d3dd322b0e8885b7d95c14d2237 This is part of a large MR which add automated testing to several predefined cookiecutter scenarios: Add automated tests for cookiecutter generation (!23) · Merge Requests · meltano / Singer SDK · GitLab