Hi I’m noticing that the

exclude

clause doesn’t seem to be working for

tap-salesforce

. For example:

select:
- Task.*
exclude:
- Task.description

Doesn’t exclude

Task.description

. Interestingly, the “exclusion pattern” in the

select

clause does work:

select:
- Task.*
- '!Task.Description

Does work…kinda. It just

nulls

the column, instead of excluding it. Not sure if I’m missing something obvious or not

Hi, @matt_cooley - You've actually stumbled on a common troubleshooting point. Not all taps filter the SCHEMA messages they send to the target, which means the target will often create columns for those even if the data itself (sent in RECORD messages) are being correctly filtered. SDK-based taps will always filter both RECORD and SCHEMA messages, but others may or may not, depending on the implementation.

Ok, cool. Does that mean that

select: !Task.Description

filters the records and

exclude: Task.Description

filters the schema, but the tap I’m using doesn’t respect schema filters?

Unfortunately, no,

exclude

and

select

in this context both are meltano-implementation that feed into the

selection

feature in the catalog. If the tap doesn't filter the schema messages as part of that, there's not much we can do about it.

Hmmm…what’s weird is that the

select: !Task.Description

works, it doesn’t EL the data (though it does add the column).

exclude: Task.Description

doesn’t work at all. That’s why I was making that assumption.

I don't know the reason why the exclude wouldn't work, but the other stuff makes sense.