bland-musician-6817803/09/2021, 1:45 PM
command? My env var
meltano config <extractor> set _schema <entity> <attribute> <key> <value>
is only considered if I have the following schema section in `meltano.yml`:
plugins: extractors: - name: tap-spreadsheets-anywhere namespace: tap_spreadsheets_anywhere pip_url: git+<https://github.com/ets/tap-spreadsheets-anywhere.git> capabilities: - catalog - discover - state schema: list_of_products: product_number: type: - string - 'null'
in the Docker container. I do not change this configuration file, ELT contex (data source, data target) is set using env vars. However I didn't find a way to set the data type using env var only. For now, I'm using the
option (https://github.com/ets/tap-spreadsheets-anywhere#configurationway) in
env var, but this is a way to set the data type to tap-spreadsheets-anywhere only. It is not a universal solution.
ripe-musician-5993303/09/2021, 3:37 PM
. This is because Meltano only defines the top-level
), and doesn't know about any of the nested settings until they're defined as custom settings and get an env var: https://meltano.com/docs/configuration.html#custom-settings Meltano doesn't currently parse the "original" schema from the catalog to proactively detect the nested settings and determine their env vars, nor does it eagerly read from the environment and attempt to parse
, since it's impossible to know just from that string which underscores represent nesting, and which represent actual underscores.