#3554 feat: `OpenAPISchemaNormalizer` should strip `pattern` keyword, like it does `enum` Issue created by edgarrmondragon Summary

OpenAPISchemaNormalizer

(in

singer_sdk/schema/source.py

) already removes the

enum

keyword from OpenAPI schemas because it is a validation-only constraint that has no meaning in Singer's data-exchange model. The

pattern

keyword is in the same category — it is a string-validation constraint that Singer taps and targets do not use — but it is currently left in the schema. This causes a concrete failure when an OpenAPI spec contains a

pattern

value that is valid in the spec's intended regex dialect (e.g. Java/PCRE with named groups

(?<name>...)

) but is not valid ECMA 262 / Python regex. The jsonschema meta-schema validator then rejects the Singer schema at test time with:

jsonschema.exceptions.SchemaError: 'data:(?<mime>image\/(.+?));base64,(?<data>.*)' is not a 'regex'

Even when the pattern is syntactically valid Python regex, keeping it in Singer schemas produces false-negative validation failures for records whose values are perfectly acceptable for ELT purposes but happen not to match the constraint. Concrete example

WorkspaceResource

in a Spring Boot / Springdoc OpenAPI spec: "imageUrl": { "type": "string", "maxLength": 699052, "minLength": 0, "pattern": "data:(?<mime>image\\/(.+?));base64,(?<data>.*)" }

(?<name>...)

is a Java named-capture-group — valid PCRE, invalid ECMA 262 and invalid Python

re

. When

OpenAPISchema.fetch_schema

returns this schema and the SDK's built-in test suite validates it,

test_tap_stream_schema_is_valid

fails because

jsonschema

checks the

pattern

value against the

regex

format. The workaround currently required in tap code: class MyOpenAPISchema(OpenAPISchema): def fetch_schema(self, key): schema = super().fetch_schema(key) for prop in schema.get("properties", {}).values(): if isinstance(prop, dict) and prop.get("pattern"): try: re.compile(prop["pattern"]) except re.error: prop.pop("pattern") return schema Proposed fix Add

handle_pattern

to

OpenAPISchemaNormalizer

, mirroring the existing `handle_enum`: def handle_pattern(self, schema: Schema) -> Schema: """Handle pattern values in a JSON schema. Args: schema: A JSON schema. Returns: The schema with ``pattern`` removed. """ schema.pop("pattern", None) return schema And call it from

normalize_schema

, alongside `handle_enum`: # Remove 'pattern' keyword (validation-only, not used in Singer data exchange) if "pattern" in result: result = self.handle_pattern(result) Rationale Singer schemas describe the shape of data for transport between taps and targets. Validation-constraint keywords like

enum

,

pattern

,

minimum

,

maximum

,

minLength

,

maxLength

,

multipleOf

, etc. are meaningful in an OpenAPI context (to validate API request/response payloads) but have no role in Singer's ELT pipeline. Stripping

enum

is already established precedent;

pattern

should follow. The overridable

handle_pattern

hook keeps the same design as

handle_enum

, so tap authors who want to preserve patterns can override it. Environment • Singer SDK version:

0.54.0a5

• Reproduced while building a tap against a Spring Boot API whose OpenAPI spec uses Java-style named-capture-group patterns. • Related: #3552 (

additionalProperties

$ref not resolved — filed separately, now fixed in 0.54.0a5) meltano/sdk