https://github.com/shaped-ai/target-clickhouse I have been working on a Clickhouse Target on and off for the past few weeks and am having an issue with adding a new config value. I added it in both the Target setup, and in the Meltano.yml, but I am not seeing it appear on Meltano Hub, or be recognized when I attempt to pass the config variable with environment variables. https://hub.meltano.com/loaders/target-clickhouse--shaped-ai/ The config value is

table_name

, so I expect the environment variable to be

TARGET_CLICKHOUSE_TABLE_NAME=foo

, but the following test of the target doesn't have it show up in the output which is strange.

TARGET_CLICKHOUSE_SQLALCHEMY_URL=localhost:8080 TARGET_CLICKHOUSE_TABLE_NAME=foobar meltano config target-clickhouse
2023-08-01T17:57:22.482666Z [info     ] The default environment 'sandbox' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
{
  "sqlalchemy_url": "localhost:8080"
}

Works for me

visch@DESKTOP-9BDPA9T:~/git/meltano-projects/test-clickhouse-config$ meltano install
Installing 1 plugins...
Installing loader 'target-clickhouse'...
Installed loader 'target-clickhouse'
visch@DESKTOP-9BDPA9T:~/git/meltano-projects/test-clickhouse-config$ TARGET_CLICKHOUSE_SQLALCHEMY_URL=localhost:8080 TARGET_CLICKHOUSE_TABLE_NAME=foobar meltano config target-clickhouse
2023-08-01T18:10:28.306296Z [info     ] The default environment 'test' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
{
  "sqlalchemy_url": "localhost:8080",
  "table_name": "foobar"
}
visch@DESKTOP-9BDPA9T:~/git/meltano-projects/test-clickhouse-config$ cat meltano.yml
version: 1
send_anonymous_usage_stats: true
project_id: "target-clickhouse"
default_environment: test
plugins:
  extractors: []
  loaders:
  - name: "target-clickhouse"
    namespace: "target_clickhouse"
    executable: "cat -"
    pip_url: cowsay
    capabilities:
    - about
    - stream-maps
    - record-flattening
    settings:
    - name: sqlalchemy_url
      kind: string
      required: true
      secret: true
      description: "The SQLAlchemy connection string for the ClickHouse database."
    - name: table_name
      kind: string
      required: false
      description: "The name of the table to load data into. Defaults to the stream name."

I think I may have an older version of the artifact then perhaps, thanks this is super helpful to debug

I wonder if it's a lock file thing on your side

What is the best way to promote cleaning and blowing away the lock files? I was guessing that

meltano install --clean

would be good enough but still seeing it build suspiciously fast that implies it is grabbing it from a cache somewhere

check out

./plugins

to see if it's a lock file thing. Clean is good for

pip

related stuff (dependencies and things) but meltano settings may have been set in

./plugins

default_target_schema [env: TARGET_CLICKHOUSE_DEFAULT_TARGET_SCHEMA] current value: None (default)
	Default Target Schema: The default target database schema name to use for all streams.
flattening_enabled [env: TARGET_CLICKHOUSE_FLATTENING_ENABLED] current value: None (default)
	Flattening Enabled: 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth [env: TARGET_CLICKHOUSE_FLATTENING_MAX_DEPTH] current value: None (default)
	Flattening Max Depth: The max depth to flatten schemas.
sqlalchemy_url [env: TARGET_CLICKHOUSE_SQLALCHEMY_URL] current value: 'localhost:8080' (from the TARGET_CLICKHOUSE_SQLALCHEMY_URL variable in the environment)
	SQLAlchemy URL: SQLAlchemy connection string
stream_map_config [env: TARGET_CLICKHOUSE_STREAM_MAP_CONFIG] current value: None (default)
	Stream Map Config: User-defined config values to be used within map expressions.
stream_maps [env: TARGET_CLICKHOUSE_STREAM_MAPS] current value: None (default)
	Stream Maps: Config object for stream maps capability. For more information check out [Stream Maps](<https://sdk.meltano.com/en/latest/stream_maps.html>).

Custom, possibly unsupported by the plugin:
table_name [env: TARGET_CLICKHOUSE_TABLE_NAME] current value: 'foobar' (from the TARGET_CLICKHOUSE_TABLE_NAME variable in the environment)

Getting the following output with the config line still being listed as unsupported after trying a few things such as reinstall Meltano, deleting the

./plugins

directory and deleting all lock files, this is confounding me a little bit.