#3803 bug: a full-table stream cannot write state after it reads a bookmark with `self.context` Issue created by ReubenFrankel Singer SDK Version

0.54.6

(also

main

at

aea8e43

) Is this a regression? • Yes Python Version 3.10 (also reproduced on 3.12) Bug scope Taps (catalog, state, etc.) Operating System Linux Description A full-table stream that reads a bookmark with

self.context

during a sync cannot write state. The sync stops with

TypeError: cannot pickle 'mappingproxy' object

. This is a regression in `0.50.0`:

0.49.1

writes the state without an error. from singer_sdk import Stream, Tap from singer_sdk import typing as th class FullTableStream(Stream): name = "full_table" schema = th.PropertiesList(th.Property("id", th.StringType)).to_dict() def get_records(self, context): # For example, to seed a date filter from a bookmark, if there is one self.get_starting_replication_key_value(self.context) yield {"id": "1"} class ReproTap(Tap): name = "tap-repro" config_jsonschema = th.PropertiesList().to_dict() def discover_streams(self): return [FullTableStream(self)] tap = ReproTap(config={}) tap.streams["full_table"].sync({"customer_id": "1"})

File ".../singer_sdk/singerlib/encoding/simple.py", line 46, in to_dict
    return asdict(self, dict_factory=exclude_null_dict)
  ...
  File ".../copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle 'mappingproxy' object

| singer-sdk | Result | | ---------- | --------------------- | | 0.49.1 | STATE message written | | 0.50.0 | TypeError | | 0.54.6 | TypeError | Cause 1.

sync()

stores

self.context

as a

MappingProxyType

(`streams/core.py#L1329`). 2.

get_starting_replication_value()

calls

get_context_state()

before it checks the replication method (`streams/_state.py#L281`). 3. No partition exists for the context, so

get_writeable_state_dict()

creates one and stores the

mappingproxy

in it as it is (`helpers/_state.py#L100`). 4. The next

STATE

message deep-copies the state, and a

mappingproxy

cannot be copied. Up to

0.49.1

,

_write_starting_replication_value()

created the partition from the plain

dict

context for every stream before

get_records()

ran. The later lookup with the

mappingproxy

compared equal to that partition, so it created no new one. #3286 added an early return for a stream that is not incremental (`streams/_state.py#L145-L154`). A full-table stream now has no partition when

get_records()

runs. Possible fixes • Store a

dict

copy of the context when a new partition is created in

get_writeable_state_dict()

. This also protects any other path that passes

self.context

to a state method. • Return from

get_starting_replication_value()

for a stream that is not incremental before it calls

get_context_state()

, so that a read creates no state. Downstream

tap-googleads

works around this in Matatika/tap-googleads#173: it reads a bookmark only for a stream with a replication key. Link to Slack/Linen No response --- 🤖 Generated with Claude Code meltano/sdk