Question, I have a tap.. it 1. queries a api endp...
# singer-tap-development
e
Question, I have a tap.. it 1. queries a api endpoint.. pushes out some relational data result.. lets call it pipeline A, this would run every weekend and right now can take anywhere from 45 minutes to 12 hours 2. Pipeline B .. would run daily and.. use the last known fresh results stored from pipeline A ... which in my case is being pushed to Postgresql... How do I write the 2nd pipeline B tap. logic.. I guess I can at first hardcode it to... go back into the postgresDB configured by pipeline A ... (the metadata of each is in the same tap in the yaml file) so I would use SQLAlchemy pypi library.. to pull that last known run out of DB X Table Y.. result from pipeline A .. in order to then query and fill up results for pipeline B Am I tapping things correct or?