I have a custom tap that has a single setup step to run, (a simple rest call) which writes multiple files out to a azure location, before I can return multiple streams, one for each file. Where should I place the code to run this setup step? Which method should I override? I need the

run_id

available to me, so I think

Tap.discover_streams

is too early. But it's not correct to do that at the stream level (as part of

Stream.get_records

) because it only needs to be carried out once per tap run.

Tap.sync_all

would be another option if it weren't a

@final

method. Is there anywhere I can jump in, a

setup

ish method in Tap? Or should I implement a singleton for all my streams to make use of in

get_records

?

I'd probably do it on init of the stream class so something like

def __init__(
        self,
        tap: Tap,
        name: str | None = None,
        schema: dict[str, t.Any] | Schema | None = None,
        path: str | None = None,
    ) -> None:
        self.setup()
        super().__init__(tap, name, schema, path)

I think that would run once for each stream, but I just need it once for each tap run.

Then on the init of the tap class

Think I will bodge it for now, but interested to see if there is a way... Hello

global

my old friend :)