francis_niu
09/16/2021, 9:35 AMmeltano.yml plugins extras. Does it only work in plugins config now?douwe_maan
09/16/2021, 2:21 PMmeltano.yml snippet that isn’t working as expected?francis_niu
09/17/2021, 2:34 AM# .env
TAP_MONGODB_DATABASE=my_db_name
# meltano.yml
plugins:
extractors:
- name: tap-mongodb
variant: singer-io
pip_url: tap-mongodb
select:
- ${TAP_MONGODB_DATABASE}-user.*
metadata:
'*':
replication-method: LOG_BASED
schema:
${TAP_MONGODB_DATABASE}-user:
_id:
type: string
name:
type: string
Then I run meltano select tap-mongodb --list --all, it outputs as below and cannot select expected collections:
Enabled patterns:
${TAP_MONGODB_DATABASE}-user.*francis_niu
09/18/2021, 7:15 AMmetadata extra works fine, like this:
metadata:
'*':
replication-method: $REPLICATION_METHOD
But in select and schema extras doesn’t work.
My current workaround is putting all select into .env and use wildcard to replace db name in schema (because schema is too long for env var). It’s weird but works fine.douwe_maan
09/20/2021, 1:19 PM${TAP_MONGODB_DATABASE}-user under schema ), just in values.douwe_maan
09/20/2021, 1:20 PMdouwe_maan
09/20/2021, 1:20 PMselect and the keys in schema are patterns, would *-user.* work, or would that pick up unrelated *-user tables?francis_niu
09/27/2021, 2:21 AMselect into .env to specify tables precisely and use wildcard pattern in schema.