Hi everyone not sure if that’s an issue or i’m doping something wrong i have a tap with 3 streams which i run separately using

--select <stream_name>

and i also have a

meltano-map-transformer

with mapping for all 3 streams in yaml but it looks like when i’m running only one stream it still tries to map all others, and fails because of schema mismatch is there a way to pass

--select

to mapper, or I’m missing something ? thanks

@pavlo what commands are you using? The

--select

flag is only available with

elt

and

elt

doesnt support mappers. If you want to use meltano-map-transformer you should run with with

meltano run tap-x mapper_name target-y

. Another option is if either of your tap/target are SDK based then you can configure the mapping using SDK stream maps and elt should work fine

yeah i’m using

elt

and stream_maps but for some reason when i have stream maps for all streams while running 1 with

--select

meltano crashes complaining that schema for another stream (not the one i selected) is not correct and it’s missing values so i found when i leave stream_maps only for stream i run it works (for remove i mean commenting them in yaml) currently trying to filter them there but was thinkining maybe there’s another solution

oh ok this starts to sound like a bug in the SDK then. Ideally we'd want unused mappings to have no affect on the sync. Would you be able to open an issue in the SDK explaining your use case and the bug? As a workaround depending on how many streams you have you could try inheriting the base plugin https://docs.meltano.com/concepts/project#inheriting-plugin-definitions and configure each stream_map in its own child plugin so it would be

meltano elt tap-foo-stream1 target-y

and

meltano elt tap-foo-stream2 target-y

. That way you wont need the select flag anyways, you'd have to test it though.

thank you will try that ! and will open an issue