Heylo peeps! I’m trying to figure it out a way to ...
# getting-started
g
Heylo peeps! I’m trying to figure it out a way to “pre-filter” a table to do the ingestion as INCREMENTAL. The tricky part here is that the ingestion key should be a function -> greatest() . I know that this setup won’t work but i’m trying to brainstorm some ideas on how i could solve this without adding an extra column in the prod-db. Is there a way to filter queries in the moment of extraction?
t
If you can modify the source DB you can create a view that implements the restrictions you want, then replicate the view instead of the actual table. Otherwise... I dunno.
j
I am also interested in figuring out if there is a way to only ingest the results of a query on a table, rather than the whole table
g
Yeah, i followed Thomas hint and it was the “feasible” way to do it. But it would be interesting to have that kind of option too.