@Bruno Arnabar let's start a thread here

Hi, thank you Edgar. Sure thing - so the yml is being built with CLI commands when the project runs. Im running the script now to output the yml

So the first one will not work because that variant of tap-mssql is not based on the Meltano singer-sdk, and the second one is not using the right syntax (see the docs). Should be something like

mappers:
    - name: meltano-map-transformer
      variant: meltano
      pip_url: meltano-map-transform
      mappings:
      - name: remove-fields
        config:
          stream_maps:
            dbo-section:
              time_category_code: __NULL__
              receipt_notes:     __NULL__
              activate_wait_list: __NULL__

Then run it as

meltano run tap-mssql remove-fields target-csv

Thank you - I believe the error that I'm having is from trying to create the yml with CLI commands. When I use

meltano add mapper meltano-map-transformer
meltano config meltano-map-transformer set mappings '[{
  "name": "remove-fields",
  "config": {
    "stream_maps": {
      "dbo-section": {
        "time_category_code": "__NULL__",
        "receipt_notes": "__NULL__",
        "activate_wait_list": "__NULL__"
      }
    }
  }
}]'

it sets the

mappings

nested within a

config

block, instead of being a direct child of the

mappers

plugin. Creating a yml like this:

mappers:
    - name: meltano-map-transformer
      variant: meltano
      pip_url: meltano-map-transform
      config:
        mappings:
          - name: remove-fields
            config:
              stream_maps:
                dbo-section:
                  time_category_code: __NULL__
                  receipt_notes: __NULL__
                  activate_wait_list: __NULL__

namely:

config:
        mappings:
          - name: remove-fields
            config:

For now I have hardcoded the solution as you pointed out and that is functioning but if anyone is aware of how to modify the

mappings

through CLI without nesting it into the

config

- that would be greatly appreciated! @robin_spilner @kenny_morris

Might be worth improving the docs if this is the case, but I think

meltano config meltano-map-transformer set _mappings ...

should work (notice the leading

_

in the setting name)

Thanks Edgar let me try that out!

@Bruno Arnabar @Edgar Ramírez (Arch.dev) it looks like the Dan Norman variant is better rated and has full capabilities. Is there a reason that the Rob Winters variant is default? https://hub.meltano.com/extractors/tap-mssql--buzzcutnorman/

No good reason really, it's always been the default forever AFAICT. The default variant can be changing by submitting a PR to update this file: https://github.com/meltano/hub/blob/8cb575905cb81cd61a0fb504fbf01ac035c2ff62/_data/default_variants.yml#L337.

I began doing so but I did notice that the buzzcutnorman variant has python dependencies that are not required in wintersrd - not sure if thats worth noting here

Yeah, I think @BuzzCutNorman's variant is based on the Meltano singer-sdk, so it's bound to have different (and probably a few more) dependencies

I believe the BuzzCutNorman version doesn't support LOG_BASED sync, at least it didn't when I started with Meltano late last year.