Hi Singer-Tap Devs, I am about to embark on writi...
# singer-tap-development
e
Hi Singer-Tap Devs, I am about to embark on writing my first tap. The data source is a proprietary data source which requires me to install their code (python or C++) in order to make requests and get back what I want.. the questions I have are.. are taps always and only written in Python? Seems so.. which is fine, but I will likely want to call the C++ version of the client due to performance requirements.. this service is something I have to be logged into in order to query. Where or how do I signal to the tap that I am logged in.. or is this stuff that you reserve for the scheduler? I guess I can look into airflow (the scheduler I am using rn) and see how can I queue up things so when / IF I login.. I will begin querying what was requested… is anyone working with taps which are not always “on”? and have some sample ways to handle this ? thanks
a
Hi, @emcp. Welcome! There's no "hard requirement" that taps be written in Python, but doing so greatly expands the amount of code reuse you can take advantage of, including being able to leverage the Tap SDK, which is Python.
Where or how do I signal to the tap that I am logged in.
Generally, you can manage this internally within an Authenticator class, or in a Stream class method - either just in time with a check during get_records() or during initialization. The SDK supports different authenticator models like OAuth and JWT, but these might not apply if you are using a custom API and C++ library.
There's an article here about wrapping C++ libraries within Python - generally for performance/interop, as I think is also your use case also: Python Bindings: Calling C or C++ From Python – Real Python
e
my plan was to write a thrift based C++ server.. and “tap” it via python client
most of the logic will be in C++.. up until the part which singer, etc.. needs to be python
great insights on the SDK, thanks AJ