I want to download the state.json file from s3 inside the tap when it start to run. How can I do that in order to meltano to read from it before the tap start?

If you run “meltano elt” with a “--job_id=my-pipeline” flag, it will automatically save state in the system database and use it on the next run with the same job ID, so there’s no need to manage it yourself or store in S3! Would that work for your use case?

I see what you mean, but, I want that the the tap will work on different keys that I want to pass, and each key has it's own state

Could you vary the job ID when you vary the keys you pass, so that a different state will be stored and used? Otherwise I’m not sure how Meltano would be able to figure out which state to use. Note that I’m not opposed to adding a “state” flag to “meltano elt” to allow the state JSON file to be specified directly, but that’d be a power user feature and not the default

I thought to add the state json file on the init of the tap and then call to singer.parse_args, and it will get the state from the file I just downloded

I’m not quite sure what you’re looking for... if your tap will download the state file itself, it should be able to use that one, but then you’re intentionally skipping Meltano’s own state management mechanism and you’ll want to remove the “state” capability from your custom tap’s definition in meltano.yml, to prevent Meltano from storing the state itself and passing the “state” flag on invocation of your tap.

I am going to use the state mechanizm, but, save it to s3 and download it from there, but, I guess, I can use the job id parameter

Yeah, Meltano will store and load state based on the job ID, so as long as you use different job IDs for “meltano elt” runs that should use their own distinct state, you should be good

where the state files are stored, you know, I want to run on different machines, and I need meltano to get the correct state file?

See https://meltano.com/docs/production.html#storing-metadata!

ok

You can set up a Postgres system database to be shared by environments if that’s desirable

great.