https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
p

prratek_ramchandani

02/22/2022, 2:45 PM
I'm trying to make logging less verbose for this SDK based Instagram tap. It uses parent-child streams in a few places and one in particular has a large number of parent objects. To fetch insights for a media object you have to pass in the media ID, making the media stream the parent and you could have potentially thousands of those. • I can reduce metric logging by just setting
metrics_log_level
to DEBUG or NONE • I can log state less frequently by changing
state_partitioning_keys
to track less state bookmarks in the first place • What I'm left with is the SDK still logging
Beginning full table sync of ... with context ...
after each media object, leading to in our case tens of thousands of lines of logs. Is there anything we can do to not log that for child streams in particular?