francis_niu
09/16/2021, 9:35 AMmeltano.yml
plugins extras
. Does it only work in plugins config
now?douwe_maan
09/16/2021, 2:21 PMmeltano.yml
snippet that isn’t working as expected?francis_niu
09/17/2021, 2:34 AM# .env
TAP_MONGODB_DATABASE=my_db_name
# meltano.yml
plugins:
extractors:
- name: tap-mongodb
variant: singer-io
pip_url: tap-mongodb
select:
- ${TAP_MONGODB_DATABASE}-user.*
metadata:
'*':
replication-method: LOG_BASED
schema:
${TAP_MONGODB_DATABASE}-user:
_id:
type: string
name:
type: string
Then I run meltano select tap-mongodb --list --all
, it outputs as below and cannot select expected collections:
Enabled patterns:
${TAP_MONGODB_DATABASE}-user.*
francis_niu
09/18/2021, 7:15 AMmetadata
extra works fine, like this:
metadata:
'*':
replication-method: $REPLICATION_METHOD
But in select
and schema
extras doesn’t work.
My current workaround is putting all select
into .env
and use wildcard to replace db name in schema
(because schema is too long for env var). It’s weird but works fine.douwe_maan
09/20/2021, 1:19 PM${TAP_MONGODB_DATABASE}-user
under schema
), just in values.douwe_maan
09/20/2021, 1:20 PMdouwe_maan
09/20/2021, 1:20 PMselect
and the keys in schema
are patterns, would *-user.*
work, or would that pick up unrelated *-user
tables?francis_niu
09/27/2021, 2:21 AMselect
into .env to specify tables precisely and use wildcard pattern in schema
.