I need to drop my raw schema every time before running running a specific tap. In my research I learned about macros. I wrote this one based on a macro I saw from a blog post:

{%- macro drop_schema() -%}
    {%- set drop_query -%}
        drop schema {{ target.schema }}
    {%- endset -%}
    {% do run_query(drop_query) %}
{%- endmacro -%}

I'm assuming I should be able to do something like this:

mel run dbt:run-operation:drop_schema sonar warehouse

but I get an error saying it can't find drop_schema. I have it in

./macros/

. I'm assuming I need to put it in my meltano.yml under my dbt transfromer section. Maybe it should go under utilities? I'm at a loss

The

:

syntax is for plugin commands: • https://docs.meltano.com/reference/command-line-interface/#commands • https://docs.meltano.com/concepts/project#plugin-commands You could add a custom command which runs your

drop_schema

macro:

commands:
    drop_schema:
      args: run-operation drop_schema

meltano run dbt:drop_schema sonar warehouse

Is there a way to make the drop_schema macro run before running the sonar extractor without having to specify it? It doesn't even need to be a macro. If not, how does the macro know which schema it's dropping? Is it informed by the

sonar

part of my

meltano run dbt:drop_schema sonar warehouse

?

Is there a way to make the drop_schema macro run before running the sonar extractor without having to specify it?

Not that I am aware of.

It doesn't even need to be a macro

If the target supported a

drop_schema

(or similar) setting, I could see something like

WAREHOUSE_DROP_SCHEMA=true meltano run sonar warehouse

working. Don't know if this is a good idea though, and certainly haven't seen it on any targets I have used.

If not, how does the macro know which schema it's dropping?

You might be able to do it with the environment variables Meltano exposes at runtime. I haven't tried myself, but as far as I understand in your case Meltano should set

MELTANO_EXTRACTOR_NAME

at runtime, so you could do something conditional in the macro when this is set to

sonar

. These might be helpful: • https://discuss.meltano.com/t/16289047/is-there-a-good-primer-on-how-to-integrate-an-existing-dbt-p • https://docs.meltano.com/guide/configuration/#available-plugin-environment-variables • https://docs.meltano.com/guide/configuration/#accessing-from-plugins

utilities:
  - name: drop-schema
    namespace: drop-schema
    commands:
      sonar:
        args: postgresql://${WAREHOUSE_USER}:${WAREHOUSE_PASSWORD}@${WAREHOUSE_HOST}/${WAREHOUSE_DATABASE} -c "DROP SCHEMA raw_sonar CASCADE"
        executable: /usr/bin/psql

There's my solution

Does that work better for you than the dbt macro?