Matt Menzenski
04/11/2023, 1:45 PMth.Property(
"operation_types",
th.ArrayType(th.StringType),
required=False,
description=(
"List of MongoDB change stream operation types to include in tap output. The default behavior is to "
"limit to document-level operation types. See full list of operation types at"
"<https://www.mongodb.com/docs/manual/reference/change-events/#operation-types>. Note that the list "
"of allowed_values for this property includes some values not available to all MongoDB versions."
),
default=[
"create",
"delete",
"insert",
"replace",
"update",
],
allowed_values=[
"create",
"createIndexes",
"delete",
"drop",
"dropDatabase",
"dropIndexes",
"insert",
"invalidate",
"modify",
"rename",
"replace",
"shardCollection",
"update",
],
),
Matt Menzenski
04/11/2023, 1:46 PM2023-04-11T13:43:37.069262Z [info ] raise ConfigValidationError(summary) cmd_type=elb consumer=False name=tap-mongodb producer=True stdio=stderr string_id=tap-mongodb
2023-04-11T13:43:37.069428Z [info ] singer_sdk.exceptions.ConfigValidationError: Config validation failed: ['create', 'delete', 'insert', 'replace', 'update'] is not one of ['create', 'createIndexes', 'delete', 'drop', 'dropDatabase', 'dropIndexes', 'insert', 'invalidate', 'modify', 'rename', 'replace', 'shardCollection', 'update'] cmd_type=elb consumer=False name=tap-mongodb producer=True stdio=stderr string_id=tap-mongodb
Denis I.
04/11/2023, 2:14 PMenum
for allowed values representation in the JSON schema and it doesn’t work with arrays unless this property is not moved to items
.
(https://github.com/meltano/sdk/blob/main/singer_sdk/typing.py#L479)
Now:
"operation_types": {
"type": ["array", "null"],
"items": { "type": ["string"] },
"enum": [
"create",
"createIndexes",
"delete",
"drop",
"dropDatabase",
"dropIndexes",
"insert",
"invalidate",
"modify",
"rename",
"replace",
"shardCollection",
"update",
],
}
Should be:
"operation_types": {
"type": ["array", "null"],
"items": {
"type": ["string"],
"enum": [
"create",
"createIndexes",
"delete",
"drop",
"dropDatabase",
"dropIndexes",
"insert",
"invalidate",
"modify",
"rename",
"replace",
"shardCollection",
"update",
],
},
}
Matt Menzenski
04/11/2023, 2:26 PMenum
property into the items
object to get the behavior I want?Matt Menzenski
04/11/2023, 2:26 PMDenis I.
04/11/2023, 2:34 PMallowed_values
feature of SDK, something like this:
if self.allowed_values:
if type is array: # pseudocode
type_dict["items"].update({"enum": self.allowed_values}) # just a concept
else:
type_dict.update({"enum": self.allowed_values})
Denis I.
04/11/2023, 3:01 PMMatt Menzenski
04/11/2023, 9:28 PMMatt Menzenski
04/11/2023, 9:29 PMoperation_types:
- testOp # not supported
- create
- delete
- insert
- replace
- update
fails with:
2023-04-11T21:27:44.470506Z [info ] raise ConfigValidationError(summary) cmd_type=elb consumer=False name=tap-mongodb producer=True stdio=stderr string_id=tap-mongodb
2023-04-11T21:27:44.470774Z [info ] singer_sdk.exceptions.ConfigValidationError: Config validation failed: 'testOp' is not one of ['create', 'createIndexes', 'delete', 'drop', 'dropDatabase', 'dropIndexes', 'insert', 'invalidate', 'modify', 'rename', 'replace', 'shardCollection', 'update'] cmd_type=elb consumer=False name=tap-mongodb producer=True stdio=stderr string_id=tap-mongodb
Denis I.
04/11/2023, 10:58 PM