Hello, I am using tap-mysql and I have employees table having following columns Employee ID | Employee Name | Department | Address 1 | Emp 1 | Account | Address 1 2 | Emp 2 | Account | Address 2 3 | Emp 3 | IT | Address 3 4. | Emp 4 | IT | Address 4 Now I want to filter data so that it extracts data having department Account and that data should be loaded in target-postgres. I tried to find out the way using pipelinewise-transform-field and meltano-map-transform but I couldn't find the example of it. Can anyone please provide me example how can I achieve this ? Thanks

https://sdk.meltano.com/en/latest/stream_maps.html#filtering-out-records-from-a-stream-using-filter-operation stream_maps: customers: __filter__: department == 'Account'

@mark_johnston thanks this worked.

you're not too far off - the filters are Python syntax, so the following example shows you how to construct your filter and avoid a syntax error, (hint:

AND

should be

and

):

$ python
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> id = 2
>>> department = 'Account'
>>> department == 'Account'
True
>>> id == 2
True
>>> department == 'Account' AND id == 2
  File "<stdin>", line 1
    department == 'Account' AND id == 2
                            ^^^
SyntaxError: invalid syntax
>>> department == 'Account' and id == 2 
True

I would recommend learning Python if you can!

Thanks @mark_johnston yeah I am learning that too.