Hi guys. I need help with a problem. I'm developing a Tap for Amazon SP API and when I try to save a state, it's not saving anything in the database (I connect it to postgresql as mentioned in the documentation) and I have configured my replication_key and STATE_MSG_FREQUENCY properties. Any idea what it could be? I do have the MELTANO_DATABASE_URI property in the .env file.

Hi @Julio Contreras! Did you define

capabilities

for your tap in

meltano.yml

?

Hi @Edgar Ramírez (Arch.dev) Yes! I do have this capabilities for both, the tap and meltano project

capabilities:
- discover
- state
- catalog

Ah, are you using Stream.get_starting_timestamp or Stream.get_starting_replication_key_value in your tap to make a filtered request? I'm not familiar with the amazon sp api, but that'd be a query parameter like

?after=...

or similar.

Hey @Edgar Ramírez (Arch.dev) I found the same example here Looks like that part was missing 😅 Still don't see anything saved in the state table (Don't know if that's normal) but it's printing state messages and looks like it's taking the last state when process is stopped. Thank you!!

It's great that it's working now! If you want to explore the saved states, you can use the

meltano state

command: https://docs.meltano.com/reference/command-line-interface#state

Sure 🙂 Thank you!

Hey @Julio Contreras 👋 That's a note added by the SDK here: https://github.com/meltano/sdk/blob/952ea676402556b58fda3f3b6f0755aa311c6f0b/singer_sdk/helpers/_state.py#L210-L212 It means the stream is assumed to be unsorted by the replication key, which means it's not safe to increment state because older records that would come at a later time could be lost if the sync is interrupted. You can set the

is_sorted = True

attribute in your stream class if you know records are sorted in ascending order by the replication key. It's a common enough question I got a PR to add a FAQ entry: https://github.com/meltano/sdk/pull/2168

Thank you very much, once again @Edgar Ramírez (Arch.dev)!! 🥳