https://linen.dev logo
Join Slack
Powered by
Hi, there. I am trying to set-up the clickhouse lo...
# singer-targets
a
Hi, there. I am trying to set-up the clickhouse loader and get error 
ELT could not be completed: Loader failed
. Details inside.
I have pretty the following settings:
Copy code
loaders:
  - name: target-clickhouse
    variant: shaped-ai
    pip_url: git+<https://github.com/shaped-ai/target-clickhouse.git>
    config:
      database: holding_test
      default_target_schema: holding_test
      host: <http://rc1a-hqgn203ff9jog6cr.mdb.yandexcloud.net|rc1a-hqgn203ff9jog6cr.mdb.yandexcloud.net>
      load_method: upsert
      port: 8443
      table_name: test
      username: user1
      verify: true
      password: $TARGET_CLICKHOUSE_PASSWORD
env:
  SSL_CERT_FILE: "/home/andrew_petrukhin_vlad/.dbt/RootCA.pem"
My thoughts more about incorrect 
SSL
definition. But no ideas how to fix that.
r
If the error message is not clear, you can
Copy code
meltano invoke <tap> > tap.out
cat tap.out | meltano invoke target-clickhouse
a
Got the following error
Untitled
r
So it's pretty clear the connection is failing - you can see it tries to make a 
POST
request:
Copy code
File "/home/andrew_petrukhin_vlad/meltano-projects/dbt-gbq-clickhouse/.meltano/loaders/target-clickhouse/venv/lib/python3.9/site-packages/clickhouse_sqlalchemy/drivers/http/transport.py", line 171, in _send
    r = <http://self.http.post|self.http.post>(
  File "/home/andrew_petrukhin_vlad/meltano-projects/dbt-gbq-clickhouse/.meltano/loaders/target-clickhouse/venv/lib/python3.9/site-packages/requests/sessions.py", line 637, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
a
any recommendations how to fix that? I have tried the same creds using Dbeaver and it works well. So, my understanding it is about definition in 
meltano.yml
r
I don't know if it's treating the password as a string literal here, but in any case you don't need to define
Copy code
password: $TARGET_CLICKHOUSE_PASSWORD
as Meltano will pick up a value set in the shell env or 
.env
file for all plugin settings automatically: https://docs.meltano.com/guide/configuration/#configuring-settings So try removing that line and see if it works.
a
yep, actually I added that manually (as debugging measure). By 
target-clickhouse set --interactive
this param was added to 
.env
. if I delete 
password: $TARGET_CLICKHOUSE_PASSWORD
from 
meltano.yml
the same error occurs.
👍 1
From my side there is the main point to understand how to define ssl certificate correctly due to absence of real examples in the IFU.
r
Nothing else jumps out at me from your 
meltano.yml
, and I don't see anything about an SSL certificate in the repo - did you have to do something special for DBeaver? Also, have you tried with 
verify: false
, or is that not an option?
a
Actually, nothing special. Standards creds + activation of SSL (see picture attached)
Also, have you tried with 
verify: false
, or is that not an option?
I have not tried. Let me
Also, have you tried with 
verify: false
, or is that not an option?
didn't work either
r
If your server requires a certificate, then it doesn't look like there is currently a way to provide that as a parameter to the target: https://github.com/shaped-ai/target-clickhouse/blob/c0f71fce74f708d38368238c921e49d3e4ab4fdf/target_clickhouse/connectors.py#L47-L63
🫠 1
There is also a 
secure
setting https://hub.meltano.com/loaders/target-clickhouse#secure-setting but I don't think that helps you (only applied when not using HTTP). My understanding of SSL is not great, so I could be missing something fundamental here...
a
from my point of view this field more accurate here: verify False 1 Should secure connection need to verify SSL/TLS. But there is no way to upload SSL certificate itself.
are there any ways to ask responsible developer?
r
Perhaps you need to be able to configure the tap to pass something like this to SQLAlchemy internally: https://stackoverflow.com/a/48851490/19106124
a
yep, for sure
5 Views
Open in Slack
PreviousNext
Powered by