https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
n

niclas_roos

02/05/2023, 10:14 AM
Hi! I'm trying to figure out if there's meltanonic (or whatever the equivalent to pythonic is for meltano) way to create a child stream that doesn't do another request, but simply parses a specified part of the parent stream row? Let's say this is one row from the parent stream:
Copy code
row = {
    'invoice_id': 1234,
    'customer_id': 5678,
    'invoice_lines': [
        {'invoice_line_id': 1, 'value': 100},
        {'invoice_line_id': 2, 'value': 300}]}
And I want the parent stream to output the following to the target:
Copy code
invoice_stream_row = {
    'invoice_id': 1234,
    'customer_id': 5678}
Next, I want to send row['invoice_id'] and row['invoice_lines'] to get_child_context() and pick them up from context as if they were a list of records to be handle by the stream, like any response would've been handled, and the yield the following rows as output of the child stream:
Copy code
invoice_line_stream_rows = [
    {'invoice_id': 1234, 'invoice_line_id': 1, 'value': 100},
    {'invoice_id': 1234, 'invoice_line_id': 2, 'value': 300}]
Any thoughts?