Meltano

Hi there! What is the best way to get metadata from a Meltano EL run?
I mainly want to get the count of records that were synced for any target.

I found this thread from last year but it is not clear how to proceed.
<https://meltano.slack.com/archives/CMN8HELB0/p1661958024854959>

I’m trying to run a custom python script after the pipeline is run, but I’m not sure how to pass data to it. Is it possible?

I have also considered creating a Meltano extension using the EDK but I want to make sure I’m not overengineering this :sweat_smile:

Most of this is handled by metadata information sent via the target like these <https://sdk.meltano.com/en/latest/implementation/record_metadata.html> which I think is why most folks don't really bother with it much

You could also redirect the metrics that are emitted by the tap into a `metrics.log` file and scrape that with an existing logger ingestor that works with JSON: <https://sdk.meltano.com/en/latest/implementation/logging.html>

But the simplest method is definitely a `COUNT GROUP BY _sdc_table_version` to get the amount of records per run, if your target is an OLAP database