Continuing my weekly trend (see: requirement) of blogging my progress in OSD600, this week’s post will be about my progress setting up Telescope and modifying a small program I wrote to work with it.
For the uninitiated: Telescope is an open source blog rss feed aggregator targeted specifically at students attending Seneca College and taking OSD600/DPS909.
This week we’re required to fork and build Telescope, and give our link checking programs the ability to check its blog post API. Luckily for me I actually set up Telescope in my spare time about a month ago in order to contribute to it. So today I essentially just had to remember how to get it up and running again. I remember setting up WSL2 which was surprisingly complex (wouldn’t MS want to make this easier to set up?) but relatively straight forward. It’s actually pretty neat but doesn’t hold my attention like running native Linux does, though I do appreciate having that layer of compatibility.
Nonetheless, I remember running into several hurdles setting up the environment- most notably getting docker and VS Code set up to communicate between Windows and WSL. Luckily the community at #telescope in Telescope’s slack was there to help me out. Learning the basics about the technologies used with Telescope was also particularly challenging, again, notably docker.
Since there have been numerous changes since I last set up Telescope, I just needed to fetch ->checkout->merge with the upstream, npm i, and run Elasticsearch, Redis, and npm start. Woohoo. Time to get my link checker program to communicate with it.
I always enjoy working with any sort of REST API. I remember the breakthrough I had when I first learned about communicating with a Mongo DB and using an API to execute CRUD commands on that DB. Something on that day clicked, in regards to both CRUD work and APIs. Because of this I actually wrote a small dad joke generator using an API (which was a big talking point during my co-op interviews actually) during my 3 week downtime between semesters 3 and 4 (well, I also wanted to learn Python to make myself more marketable.) Lots of fun, and this time was no different, JSON is pretty neat.
Anyway I started my journey testing the API using Postman, and wondering what the best approach to getting blog posts would be: using JSON, HTML, or plain text. Ultimately I settled on HTML as my link checker scraped the HTML when searching for hrefs.
Regarding implementation, since I previously modularized my code pretty efficiently, I had no real issues getting my link checker to communicate with the JSON output from my local Telescope build. I simply had to create a blog post link using a provided ID, visit the link, and grab all the hrefs… except I did spend a solid 45 minutes hunting down this damn bug that I just kept mindlessly copying and pasting without realizing between different implementations I did…
post = requests.get(post, headers={'Content-type': 'text/html'})
vs
post = requests.get(post, headers={'Accept': 'text/html'})
Oops.
You can see a GitHub commit compare here, or a git diff here.
That about does it for this one. I had a lot of fun with this week’s lab, and I’m really glad I already had Telescope set up and working, it saved me a lot of time.
Here’s a nice picture I took of the rare “blue moon” we had over Halloween, as seen from my backyard:
