I am developing a tap right now, and I am not sure...
# singer-tap-development
a
I am developing a tap right now, and I am not sure how exactly to model a many-to-many relationship appropriately. I have Teams <-> Members . The api has queries for Teams and Team Members, but it does not have a way to list all Members so I need to collect all the member id's for all the teams. Right now, this is what I have. Team (Parent Stream) -> TeamMembers (Child Stream of Team, Parent Stream) -> Members (Child Stream of TeamMembers). Since Members can belong to multiple teams, TeamMembers
get_child_context
spits out duplicate contexts
return {"member_id": record['member_id']}
. Is there a way to collect all the member_ids from the TeamMembers stream then pass them along to a child stream?
e
Hi @Alex B! So to make sure I understand, each TeamMember record contains one or more member IDs, and these IDs are global so other team members requests may respond with overlapping sets of members?
Is there a way to collect all the member_ids from the TeamMembers stream then pass them along to a child stream?
There isn't, but you could try the overriding the
__init__
method of
TeamMembers
to record which member IDs you've seen before
Copy code
class TeamMembers(YourParentClass):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.seen_members = set()
then use that to return None from
get_child_context
if the team member has already been processed.
a
Ahhh, OK. The stream class TeamMembers is only created once? Not once per context? @Edgar Ramírez (Arch.dev)
e
Right, child streams are instantiated once and synced multiple times.