How to use Meltano variables in a python file I can run a py Meltano #troubleshooting

How to use Meltano variables in a python file? I c...

rene_czepluch

10/04/2023, 7:45 AM

How to use Meltano variables in a python file? I can run a python script inside meltano by running

Copy code

$ meltano run utils:run_script

This is because my

meltano.yml

file has this:

Copy code

utilities:
  - name: utils
    namespace: utils
    commands:
      run_script:
        executable: python
        args: utilities/my_script.py

But meltano has a lot if nice variables, such as

${MELTANO_PROJECT_ROOT}

. How can I send that variable to the python file?

rene_czepluch

10/04/2023, 7:46 AM

my python file

my_script.py

looks like this:

Copy code

print("yaaaaas")
print("${MELTANO_PROJECT_ROOT}")

I get this out. (the variable is not passed)

rene_czepluch

10/04/2023, 7:54 AM

I see I can do:

Copy code

utilities:
  - name: utils
    namespace: utils
    commands:
      run_script:
        executable: python
        args: utilities/my_script.py --${MELTANO_PROJECT_ROOT}

and in the python file:

Copy code

print("yaaaaas")
print("${MELTANO_PROJECT_ROOT}")
import sys
print(sys.argv)

output:

Copy code

>>> meltano run utils:run_script
2023-10-04T07:53:17.643935Z [info     ] Environment 'dev' is active
2023-10-04T07:53:17.728957Z [info     ] yaaaaas                        cmd_type=command name=utils stdio=stderr
2023-10-04T07:53:17.729248Z [info     ] ${MELTANO_PROJECT_ROOT}        cmd_type=command name=utils stdio=stderr
2023-10-04T07:53:17.729322Z [info     ] ['utilities/my_script.py', '--/user/reth/Documents/et2ai-etl/meltano_etl'] cmd_type=command name=utils stdio=stderr
2023-10-04T07:53:17.731068Z [info     ] Block run completed.           block_type=InvokerCommand err=None set_number=0 success=True

Reuben (Matatika)

10/04/2023, 8:23 AM

MELTANO_PROJECT_ROOT

is an environment variable - you need to use the

os

module in your Python script to access those: https://stackoverflow.com/a/4907053/21047147

rene_czepluch

10/04/2023, 11:15 AM

Thank you @Reuben (Matatika) 😄

Open in Slack

Previous Next