Quick question - if I wanted to clear the state for a project how would I do so? Just testing things out with some of the examples, and I want to change the database, schema, etc, that meltano uses. I've already deleted the database on the DB server (PG). Do I just delete the .meltano directory in the project?

Lot to read through here, I'll take quick stab at it. Here's the questions as I see them so I can respond 1. if I wanted to clear the state for a project how would I do so? 2. I want to change the database, schema, etc, that meltano uses 3. Do I just delete the .meltano directory in the project? 4. I saved .meltano away and had to re-install the loader and extractor, and expect the DBT transform when I get to that point. 5. Failed as it looks like it expects the schema to already exist. Are there any options to have the loader create the schema if it does not exist? 6. Related question, I will eventually need to get this working with ephemeral users/roles.  In other words, we created a role (nologin) that the database is owned by, and the user meltano uses is a member of that role.  It looks like DBT has the capability to run post-hooks and on-run-end SQL as documented here: https://discourse.getdbt.com/t/the-exact-grant-statements-we-use-in-a-dbt-project/430.  Does meltano have similar, so we can change ownership of the objects created to the role instead of the user when the loader creates them?

Looks like there is a before_run_sql for the postgres loader that I'm testing, as far as creating the schema if not exists...

1. When you say State we normally refer to Singer state. So https://hub.meltano.com/singer/spec#:~:text=potentially%20unexpected%20manner.-,State,-State%20messages%20contain . I don't think that's what you're referring to here? If it is, no need to refresh anything just pass in

--full-refresh

https://meltano.com/docs/command-line-interface.html#how-to-use-4 into your ELT job 2. Are you talking about Taps, Targets, or Meltano itself? 3. Do I just delete the

.meltano

directory in the project? You certainly can, but it depends. You really don't need to delete

.meltano

almost ever. There's a local Database Meltano uses, is that what you're talking about? Or are you talking about your tap/target the connects to Postgres? 4. To reinstall everything just run

meltano install

it'll recreate the stuff you need in

.meltano

No need to do it yourself 5. For

targets

it depends on the loader. Some create the schema for you in the target some of them do not. I know target-postgres transfer wise varient does not create it for you. It'd be a good feature add (for all I know it could be a config option on the target) 6. Going to skip this for now as we should focus on #1 - #5

1. Yes, the singer state, for both the tap and target. For the tap, a full refresh would do it, but for the target does that apply also? Rhetorical question, I can test to see behavior if the DB is just wiped outside of meltano's knowledge. I think all the other questions are answered or I figured them out.

Target

doesn't do anything with

State

. The target takes the

state

message from the tap, and re-emits the state message the tap gave it after it syncs all data up to the point the tap gave it

state

or at least that's the idea behind

state

. Hopefully that helps?

Yea, I'll have to refresh my memory on the Singer Spec. It's been a while since I read it, and I've been busy with other projects.