question on contexts in taps.. I saw the suggestio...
# singer-tap-development
question on contexts in taps.. I saw the suggestion that it is for when you have like.. shards of a dataset.. I think I have that situation now.. I have an endpoint which I have to query with all permutations of the alphabet from 1 to 5 letters.. it then compiles a response for that set of letters and this takes about 24 hours to run... I see it running now but I have coded to run the calculation of the permutations and the entire set of multiple queries within the single "Tap" call>> Im wondering are there good examples of this use of context for queries? should I perhaps lean towards splitting up my tap by using these alphabet permutations as the "slice" of the datasets response? Basically I either hand squash.. or push to postgresql via python by hand (until just now when I started using my tap) and I get rid of duplicates myself.. but maybe I can start to just let the context calls send the raw response and postgresql handles the rest? Please excuse any ignorance.. I am sort of a noob on postgresql and the data engineering side but super happy to see my tap going along.. will wait and see how it goes , it takes about 24 hours to finish