hello I was wondering if there's anyone who have tried migrating into meltano a dbt project using dbt-sqlserver - it looks like dbt-sqlserver is missing from MeltanoHub

oh dear. are you using meltano to pull data from sources into a sql-server target, and the doing dbt subsequent?

thanks @jacob_matson - I have noticed that already. I still need an orchestrator so I think I will jest use Airflow with mssql operator, eventually bash operator and invoke bcp command

@visch is the resident sql server + meltano expert I think, curious as to his thoughts too

Looks like I cheated a bit on my side and didn't use dbt through meltano in my project (Things keep chugging along so I haven't messed with it, I really should migrate to using Meltano as life's easier that way for me). This should definitely be doable even without it on the hub though. Something like

- name: dbt
    namespace: dbt-sqlserver
    pip_url: dbt-sqlserver==1.3.1
    commands:
      clean:
        args: clean
        description: Delete all folders in the clean-targets list (usually the dbt_modules
          and target directories.)
      compile:
        args: compile --models $DBT_MODELS
        description: Generates executable SQL from source model, test, and analysis
          files. Compiled SQL files are written to the target/ directory.
      deps:
        args: deps
        description: Pull the most recent version of the dependencies listed in packages.yml
      run:
        args: run --models $DBT_MODELS
        description: Compile SQL and execute against the current target database.
      seed:
        args: seed
        description: Load data from csv files into your data warehouse.
      snapshot:
        args: snapshot
        description: Execute snapshots defined in your project.
      test:
        args: test
        description: Runs tests on data in deployed models.
      docs generate:
        args: docs generate
        description: Generate docs
      docs serve:
        args: docs serve
        description: Serve docs
      create_db: run-operation autoidm_create_db
    config:
      target: postgres
      host: localhost
      port: 5432
      user: postgres
      dbname: postgres
      project_dir: $MELTANO_PROJECT_ROOT/transform
      PROFILES_DIR: $MELTANO_PROJECT_ROOT/transform/profile
      source_schema: $MELTANO_LOAD__TARGET_SCHEMA
      target_schema: autoidm
    settings:
    - name: password
      kind: password

Sorry to come into this thread out of context… but I’ve tried implementing this into Meltano and I haven’t had much success. I’m fairly new to sqlserver, but its throwing this error:

Database Error
  connection to server at "<ip address>", port 1433 failed: server closed the connection unexpectedly
  	This probably means the server terminated abnormally
  	before or while processing the request.

---- This is what I’ve got so far (heavily influence by your post above):

- name: dbt
    namespace: dbt-sqlserver
    pip_url: dbt-sqlserver==1.3.1 dbt-postgres~=1.3.0
    settings:
    - name: password
      kind: password
    commands:
      clean:
        args: clean
        description: Delete all folders in the clean-targets list (usually the dbt_modules
          and target directories.)
      compile:
        args: compile --models $DBT_MODELS
        description: Generates executable SQL from source model, test, and analysis
          files. Compiled SQL files are written to the target/ directory.
      deps:
        args: deps
        description: Pull the most recent version of the dependencies listed in packages.yml
      run:
        args: run
        description: Compile SQL and execute against the current target database.
      seed:
        args: seed
        description: Load data from csv files into your data warehouse.
      snapshot:
        args: snapshot
        description: Execute snapshots defined in your project.
      test:
        args: test
        description: Runs tests on data in deployed models.
      docs-generate:
        args: docs generate
        description: Generate docs
      docs-serve:
        args: docs serve
        description: Serve docs
    config:
      target: postgres
      host: <ip address>
      port: 1433
      user: <user>
      dbname: <db>
      project_dir: $MELTANO_PROJECT_ROOT/transform
      PROFILES_DIR: $MELTANO_PROJECT_ROOT/transform/profile
      source_schema: $MELTANO_LOAD__TARGET_SCHEMA
      target_schema: dbo
      driver: ODBC Driver 18 for SQL Server

Any direction or advice would be greatly appreciated

Separate thread gives you a much better chance, and the command you are running, and if it's meltano specific the meltano slack!