is there a plugin or utilities that can be integrated for dbt-mysql and dbt-mssql ? or do we have to use it as a separate pipeline step?

pretty easy to fork and make one, you can also override the standard dbt / postgres-dbt to include the pip url dependenices you're after and they will "just work" which is probably why no one has upstreamed the mssql / mysql ones