https://linen.dev logo
Join Slack
Powered by
anybody here able to get `tap-clickhouse` to work?...
# singer-taps
k
anybody here able to get 
tap-clickhouse
to work? i feel like i set up the 
meltano.yml
config correctly with username and password, but still getting this error about a missing port (port should have a default set, and even if i set it explicitly in the yml i still get this error)
Copy code
File "/meltano_project/.meltano/extractors/tap-clickhouse/venv/lib/python3.11/site-packages/sqlalchemy/engine/url.py", line 221, in _assert_port
    return int(port)
           ^^^^^^^^^
ValueError: invalid literal for int() with base 10: ''
nm i think the issue was i was passing the port along with the host
actually, still getting it after all 😓 has anybody gotten this tap to work?
e
Hey @Kenny Ning! • Are you using https://hub.meltano.com/extractors/tap-clickhouse--akurdyukov/? • Can you share your (redacted) 
meltano.yml
? At least the specific section for 
tap-clickhouse
.
k
yes
Copy code
- name: tap-clickhouse
    variant: akurdyukov
    pip_url: git+<https://github.com/akurdyukov/tap-clickhouse.git>
    config:
      host: $CLICKHOUSE_URL
      port: 8443
      password: $CLICKHOUSE_PASSWORD
maybe related my host is of the form https://xyz.us-east-1.aws.clickhouse.cloud is that the right format?
ah maybe i do need to strip the header, think it's getting confused building the url: https://github.com/akurdyukov/tap-clickhouse/blob/main/tap_clickhouse/client.py#L40
e
I think you need to strip the scheme EDIT: you beat me to it, yeah!
k
okay new error! now im getting
Copy code
OSError: Could not find a suitable TLS CA certificate bundle, invalid path: 0
tried setting 
secure: 1
and 
verify: 0
e
can you try instead 
secure: true
and 
verify: false
?
k
new error with that config! fwiw i'm also running this on modal.com so might be a networking issue with the container and the database i need to dig into
Copy code
Plugin configuration is invalid
ValueError: unconverted data remains: 080
👀 1
e
Well that's a cryptic error message 😅. Is there a larger traceback or more context attached to it?
k
this is the whole message
Copy code
2024-03-19T19:53:47.851282Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
2024-03-19T19:53:47.937064Z [warning  ] `kind: password` is deprecated for setting definitions in favour of `sensitive: true`, and is currently in use by the following settings of extractor 'tap-clickhouse': 'password'. Please open an issue or pull request to update the plugin definition on Meltano Hub at <https://github.com/meltano/hub/blob/main/_data/meltano/extractors/tap-clickhouse/akurdyukov.yml>.
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
ValueError: unconverted data remains: 112
e
Ok, I think it's coming from 
datetime.datetime.strptime
in Python's stdlib:
Copy code
>>> import datetime
>>> datetime.datetime.strptime('2024-01', '%Y')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/python3.11/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/_strptime.py", line 352, in _strptime
    raise ValueError("unconverted data remains: %s" %
ValueError: unconverted data remains: -01
k
where would it be calling that function?
e
I'm thinking Meltano. What command triggered that error?
k
meltano config tap-clickhouse test
e
Do you get more details if you run on debug level?
Copy code
meltano --log-level=debug config tap-clickhouse test
👀 1
k
nope same logs
this is my config:
Copy code
- name: tap-clickhouse
    variant: akurdyukov
    pip_url: git+<https://github.com/akurdyukov/tap-clickhouse.git>
    config:
      host: xyz-123.us-east-1.aws.clickhouse.cloud
      secure: true
      verify: false
      port: 8443
      password: $CLICKHOUSE_PASSWORD
e
Are you setting any environment variables?
k
yes, that's what the $CLICKHOUSE_PASSWORD should be reading
e
Ok, let me give the tap a try myself
🙌 1
6 Views
Open in Slack
PreviousNext
Powered by