Hey all, trying my hand at the hackernews API over...
# singer-tap-development
m
Hey all, trying my hand at the hackernews API over the weekend. Bit of a noob when it comes to building production apps in python as opposed to analysis. https://github.com/mattarderne/tap-hackernews How should I iterate and develop? After making a cookiecutter, adding some obvious bits and then running it (it worked!), I'm wondering what the most effective workflowfor doing the more subtle refinements would look like? Currently I have just hardcoded a single HN endpoint into the Streams module but I need to do a few things: 1. I need to check the max_id from one endpoint and use that as an input for another endpoint 2. the URL needs a
.json
appended, ie the path needs to look something like
path = "/item/{item_id}.json"
Do I just keep running the whole package ie
poetry run tap-hackernews --config config.json
blindly until the outputs make sense? Is there a way to run single modules? How do I add and check logic as I go. I've been trying to setup poetry with VSCode debugger so that I can analyse each step, but I'm getting stuck with the debugger not actually executing the tap (it doesn't log the process nor show any outputs, just loads it)
p
Hey Matt - I was working on the same thing yesterday and I'm a pycharm user moving to VS code so I actually was referencing your debugger config here haha. I just change
"program": "${workspaceRoot}/tap_csv/tap.py",
and added a main to my tap.py.
Copy code
if __name__ == "__main__":
    TapCSV.cli()
Idk if theres a better way but that was fine for my debugging.
I think you also need to set your poetry virtualenv as your interpreter in VS code for that too
m
@pat_nadolny I could cry, thank you! 🍻 🍻 🍻 🍻 🍻 Turns out after turning myself round in circles (see my other thread haha), I was just missing the
()
on
.cli()
to get it working in a useful manner