Hello everyone! I run a pipeline and get this mess...
# getting-started
p
Hello everyone! I run a pipeline and get this messages in the terminal: • Removed paths list: ['Column1', 'Column2'] • Removed 2 paths during transforms: Column1, Column2. I don't get an error, so what is means? Something is wrong or why I get this messages?
e
Which extractor/loader are you using? I can see that message's coming from the singer library: • https://github.com/transferwise/pipelinewise-singer-python/blob/da64a10cdbcad48ab373d4dab3d9e6dd6f58556b/singer/transform.py#L100-L104https://github.com/singer-io/singer-python/blob/2c053f4c5f468235bba85a203a252438b1b3c704/singer/transform.py#L112-L116 Perhaps you have to update your select rules and check the output of
meltano select <extractor> --list --all
p
I use a local repo of this tap-intacct (https://github.com/singer-io/tap-intacct/tree/master) and target-csv
e
Ok. Yeah, it's definitely filtering out fields based on selection/metadata: https://github.com/singer-io/tap-intacct/blob/8ea618832ba367e00daf41eacd476e046da2ca7c/tap_intacct/sync.py#L57
p
So is everything Ok or does it mean that at some point something is wrong with what I do?
e
If you're seeing data flowing, it may just mean that you won't see whatever fields you are seeing logged after
Removed paths
won't land in your target.
p
Ok I get it. Thanks !