Running into an issue with dbt I was hoping to get some help

Question

Running into an issue with dbt I was hoping to get some help with I have a dbt process that I m running with `meltano elt` When I run into an error during the transformation I m still getting back a success exit status from Meltano

important-island-7384 · Answer

Is this by design

important-island-7384 · Answer

< echoing needle 59281> < fierce honey 5880> Any thoughts

fierce-honey-5880 · Answer

You are right < important island 7384> we do not check the exit code of dbt when `meltano elt` runs That means that as long as the Tap|Target pipeline successfully completes and dbt runs we always complete successfully even when dbt returns 1 at least one handled error by dbt i e some models may have been skipped or 2 unhandled error eg ctrl c network interruption etc

fierce-honey-5880 · Answer

With `meltano elt` not running the transforms by default for a couple versions now that means that dbt only runs intentionally So I think that we should update how we treat dbt exiting with an error code return an error exit code and report back to the user that the ELT was unsuccessful The dbt transforms are a core part of the pipeline and if they fail something will be missing or will not be properly updated in the `analytics` schema So this seems like something we should communicate to whoever is calling `meltano elt` either the user a script or airflow and not exit with 0 success What do you think < echoing needle 59281>

echoing-needle-59281 · Answer

< fierce honey 5880> < important island 7384> if the user runs triggers dbt we should respect the error code and return 0 if and only if everything is alright

fierce-honey-5880 · Answer

I agree

echoing-needle-59281 · Answer

I created to track this thanks for the report < important island 7384>

important-island-7384 · Answer

Thanks gentlemen

important-island-7384 · Answer

I m curious about the case of `meltano invoke` when using dbt as well

important-island-7384 · Answer

I haven t tested whether a failure is passed back with that approach but I would assume it is