I built another personal project tap over the weekend, tap-lichess. A few thoughts: • I love the new Paginator interface! It made a common problem I've seen (fetch an explicit subset of items defined in config, e.g. Lichess usernames, GitLab project ids, etc) very easy: https://github.com/rabidaudio/tap-lichess/blob/main/tap_lichess/streams.py#L12 • I made an

OpenApiStream

class. It doesn't generate streams from a swagger spec, but it does use the base url and response schema from it, which still saves a lot of time over defining schemas by hand: https://github.com/rabidaudio/tap-lichess/blob/main/tap_lichess/openapi.py • If anyone ever needs to do streaming requests, there's an example in GamesStream. Basically enable stream mode in the session with

requests_session.stream = True

and then override

parse_response

• Posting a request body that isn't json wasn't hard but I did have to re-implement

prepare_request

.

RESTStream

seems to assume all bodies are json (which seems pretty reasonable to me)

I love lichess I'm not a great chess player but I'm down to play anytime @julian_knight! Cool tap

I'm not a great chess player

Me neither!

I dig that the combination of

requests_session.stream = True

plus simple custom

parse_response

just works 😄

@julian_knight I dove into the problem about nulls a bit ago, it's a difference in the OpenAPI spec and JSON Spec (iirc). Easiest thing to do is to run a transform on the schema, and add "null" to the type of everything.