Anyway to skip schema for Rest based APIs? Would be nice if its possible to auto generate schema

Yes you can do it, just create a property in the

client.py

that generates the schema, and in the

tap.py

dynamically create the objects for each stream, you can see an example here although it is not using the Rest base class the definition of the stream property and the creation of objects should be similar.

I just opened and issue in the SDK about this because it came up in a bunch of other threads recently and I'm still not clear what the recommended approach is. I think @josue_sehnem’s is a good one but I feel the docs should make a better best practice recommendation

I agree @pat_nadolny, there are also APIs where you need to paginate using a time window and page number at the same time, I implemented a dict that returned both parameters when working on that, but I think it would be better to use some kind of double pagination, I haven't explored the new paginations class implemented but it may be possible to do it there.

Also if either of you have any thoughts to add to that issue based on your experience it would be appreciated!

I am exploring soap APIs to try to create a base class for my netsuite tap that can be used in meltano SDK. About the other things that I worked on, they may be already solved though: • Replication keys needed to be selected in order for the taps to work, maybe it was solved, and maybe It should not be solved. I can see a usage where the user don't want the replication key value in their database, but at the same time the value can be persisted in the state, in other hand it can cause other side effects that I am not anticipating, but in this case the discover could set the selection to automatic; • The finalize_state_progress_markers can be slow in cases were you have too many child stream, like APIs that have one request for each child stream; • If I remember correctly once I had an issue where datetimes were accepted as date type, but I will need to test that. I am dealing with some other things and want to create a basic POC for a SOAPStream class first, but after that I can confirm that these issues still exist and even work on them, I am trying to create a portfolio and would be happy about helping the community 🙂