Hello, Working on a tap based on a REST API. The tap hits several endpoints and I know that one of them will occasionally fail with a 403 error. However, this endpoint doesn’t have particularly important data and I want the tap to run and return data from all of the other endpoints even if the one fails. My current solution is to add an additional

optional

parameter and override the validate_response method in my stream class (which subclasses RestStream) with

def validate_response(self, response: requests.Response) -> None:
        """Updating validation to handle special case of 403"""
        if self.optional and response.status_code == 403:
            self.logger.warning(
                f"403 Forbidden Error at stream: {self.name}. "
                f"Continuing because stream is marked as optional."
            )
        else:
            super().validate_response(response)

If this is triggered, my target will receive no records and then write an empty file. Mainly curious if anyone else has encountered this and if it could/should be built in to the singer sdk.