magnus_avitsland
09/13/2022, 7:26 AMTARGET_BIGQUERY__EXCHANGERATE_TABLE_SUFFIX works perfectly - thx @edgar_ramirez_mondragon for solution
-------------------------------------------
Hi, I have my base plugin loader “target-bigquery”, and one inherited called “target-bigquery--exchangerate”, and I need to set table suffix for extracted data.
I can do it through config, but I need to set it on the load date through Airflow, but environment variables cannot contain dashes (“-”) in their name, therefor I am a bit unsure how to set it.
In my Airflow DAG, this is not picked up (obviously):
env_vars={
'TARGET_BIGQUERY__EXCHANGERATE_TABLE_SUFFIX': '_{{ ds_nodash }}'
,'TARGET_BIGQUERY--EXCHANGERATE_TABLE_SUFFIX': '_{{ ds_nodash }}'
},
The first envvar is created, but not picked up as I suspect Meltano cannot find it as the plugin is called target_bigquery--exchangerate (inherited plugin).
• Please advise 🙂magnus_avitsland
09/13/2022, 7:27 AMedgar_ramirez_mondragon
09/13/2022, 3:35 PMTARGET_BIGQUERY__EXCHANGERATE_TABLE_SUFFIX should be picked up by Meltano. You can confirm that’s the expected env var name by running
meltano config target-bigquery--exchangerate listmagnus_avitsland
09/14/2022, 7:03 AMmagnus_avitsland
09/14/2022, 7:06 AMtarget_bigquery__exchangerate_table_suffix?
Or target-bigquery__exchangerate_table_suffix?
First one, right (as env vars cannot contain dashes “-”)?magnus_avitsland
09/14/2022, 12:02 PMtarget_bigquery__exchangerate_table_suffix worked perfectly 🙂 gg!