For parent-child streams, I have this in the paren...
# troubleshooting
f
For parent-child streams, I have this in the parent:
Copy code
def get_child_context(self, record: dict, context: Optional[dict]) -> dict:
        # Create child context for each account
        return {"accountId": record["accountId"]}
and this in the child:
Copy code
class AccountUtilizationStream(BaseStream):
    """Account Utilization."""

    name = "account_utilization"
    parent_stream_type = AccountsStream
    ignore_parent_replication_keys = True
    path = "/v1/search?accountIds={accountId}"
    primary_keys = ["account_id"]
It is failing, because it is not replacing
{accountId}
in the path. I do see the following in the logs:
Beginning incremental sync of 'account_utilization' with context: {'accountId': 123456}
and the following METRIC:
METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 0.190973, 'tags': {'endpoint': '/v1/search?accountIds={accountId}', 'http_status_code': 404, 'status': 'failed', 'context': {'accountId': 123456}}}
Any ideas?
a
I can't say exactly if something fishy is going on here, but generally "path" should just be the part before
?
in the above, and then the account id can be passed in get_params, with assist from
context
.
f
I was just following the example here: https://sdk.meltano.com/en/latest/parent_streams.html I'll try with get_params
a
My mistake. Should be get_url_params()
I'm not sure why your attempt did not work, except to say that we probably do not fully handle url params when passed as part of
path
. (Open to an MR or issue though if you think this could be improved.)
f
I looked at the code, it should work. In rest.py for the RESTStream:
Copy code
def get_url(self, context: Optional[dict]) -> str:
        """Get stream entity URL.

        Developers override this method to perform dynamic URL generation.

        Args:
            context: Stream partition or context dictionary.

        Returns:
            A URL, optionally targeted to a specific partition or context.
        """
        url = "".join([self.url_base, self.path or ""])
        vals = copy.copy(dict(self.config))
        vals.update(context or {})
        for k, v in vals.items():
            search_text = "".join(["{", k, "}"])
            if search_text in url:
                url = url.replace(search_text, self._url_encode(v))
        return url
I found it. Not showing the params sidetracked me, it's a POST not a GET, so needed a rest_method for the class.
a
👍