https://linen.dev logo
Join Slack
Powered by
I’m struggling with my poetry installs. I’m runnin...
# singer-tap-development
j
I’m struggling with my poetry installs. I’m running 
meltano invoke tap-rest-api-msdk --help
and the resulting stack trace looks like poetry tries to install 
genson
but fails with
Copy code
ValueError: ("EntryPoint must be in 'name=module:attrs [extras]' format", 'tap-rest-api-msdk=tap_rest_api_msdk.tap:Taprest-api-msdk.cli')
I have no idea what that means. After the failed install, of course the actual invocation fails with
Copy code
File "/Users/jlloyd/Documents/dev/tap-rest-api-msdk/tap_rest_api_msdk/tap.py", line 6, in <module>
    from genson import SchemaBuilder
v
can you share your 
pyproject.toml
j
Copy code
[tool.poetry]
name = "tap-rest-api-msdk"
version = "0.0.5"
description = "`tap-rest-api-msdk` is a Singer tap for REST APIs, built with the Meltano SDK for Singer Taps."
authors = ["Josh Lloyd"]
keywords = [
    "ELT",
    "rest-api-msdk",
    "Meltano",
    "Singer",
    "REST",
    "API",
    "tap"
]
license = "Apache 2.0"
homepage = "<https://github.com/Widen/tap-rest-api-msdk>"
repository = "<https://github.com/Widen/tap-rest-api-msdk>"

[tool.poetry.dependencies]
python = "<3.10,>=3.6.2"
requests = "^2.25.1"
singer-sdk = "^0.3.11"
genson = "^1.2.2"
atomicwrites = "^1.4.0"

[tool.poetry.dev-dependencies]
pytest = "^6.2.5"
tox = "^3.24.4"
flake8 = "^3.9.2"
black = "^21.9b0"
pydocstyle = "^6.1.1"
mypy = "^0.910"
types-requests = "^2.25.8"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.scripts]
# CLI declaration
tap-rest-api-msdk = 'tap_rest_api_msdk.tap:Taprest-api-msdk.cli'
@visch
v
I'm staring at 
tap-rest-api-msdk = 'tap_rest_api_msdk.tap:Taprest-api-msdk.cli
j
me too
e
Taprest-api-msdk
this is not a valid Python class name so I'm guessing the class has a different name
j
I tried to change it to 
tap-rest-api-msdk = 'tap_rest_api_msdk.tap:TapRestApiMsdk.cli'
but when I rerun the 
--help
command I get the same error like I didn’t change anything: 
ValueError: ("EntryPoint must be in 'name=module:attrs [extras]' format", 'tap-rest-api-msdk=tap_rest_api_msdk.tap:Taprest-api-msdk.cli')
How do I get it to register the new name of the class?
e
can you do 
poetry install
?
j
again? sure
e
so it'll install the new entrypoint
j
no dice: still getting 
ValueError: ("EntryPoint must be in 'name=module:attrs [extras]' format", 'tap-rest-api-msdk=tap_rest_api_msdk.tap:Taprest-api-msdk.cli')
e
oh, is this the tap installed from PyPI???
j
no, I’m in the repo where the I built the tap with the SDK
e
so does 
poetry run tap-rest-api-msdk --about
work?
j
looks like it
e
so the command that's showing you that error is not a poetry one but a meltano one? How does the plugin definition look like in 
meltano.yml
?
j
Copy code
- name: tap-rest-api-msdk
      namespace: tap_rest_api_msdk
      executable: ./tap-rest-api-msdk.sh
      capabilities:
        - state
        - catalog
        - discover
      settings:
        - name: api_url
        - name: auth_method
        - name: auth_token
        - name: name
        - name: path
        - name: params
        - name: headers
        - name: records_path
        - name: primary_keys
        - name: replication_key
        - name: except_keys
        - name: num_inference_records
      config:
        name: us_earthquakes
        api_url: <https://earthquake.usgs.gov/fdsnws>
        path: /event/1/query
        params:
          format: geojson
          starttime: "2014-01-01"
          endtime: "2014-01-02"
          minmagnitude: 1
        primary_keys:
          - id
        records_path: "$.features[*]"
      select:
        - '*.*'
v
Did you change the name of your class after the cookie cutter ran? I think the cookier cutter makes the .sh file for you. Ignore this video I was trying to replicate your setup
Had to give slack video clips a shot as well
j
🙂 yes I did change it after the fact
v
Is that the winner? @edgar_ramirez_mondragon ftw
j
When I run 
poetry install
nothing actually updates. I think that’s part of the problem, it’s using a cache of some sort. Is there a way to force it to install and overwrite the cache?
yeah, that was the issue, had to go manually delete the cache in the 
pypoetry
dir (no 
--no-cache
option). Now I’ve got a new error
When I run 
meltano elt tap-rest-api-msdk target-jsonl
I get:
Copy code
meltano           | Running extract & load...
meltano           | No state was found, complete import.
meltano           | ELT could not be completed: Cannot start extractor: Catalog discovery failed: command ['/Users/jlloyd/Documents/dev/tap-rest-api-msdk/tap-rest-api-msdk.sh', '--config', '/Users/jlloyd/Documents/dev/tap-rest-api-msdk/.meltano/run/elt/2021-10-20T194403--tap-rest-api-msdk--target-jsonl/0e4a888c-4d58-4142-a6f2-3b7dfbc46332/tap.91cd9480-d8cd-4df8-ab89-6b9e740ea393.config.json', '--discover'] returned 1
A 
--log-level=debug
will produce this additional error:
Copy code
meltano.cli.utils.CliError: ELT could not be completed: Cannot start extractor: Separator is not found, and chunk exceed the limit
ha. nvm. user error, forgot to take out a extra test line
Open in Slack
PreviousNext
Powered by