Wednesday, December 3, 2014

hackernews2nntp

It has been almost 2 month since YC folks have announced their official Hacker News API & have threatened us w/ an imminent HN design change. (When I say 'us' I mean authors of various Chrome extensions or web scrapers.)

Writing YA interface on a top of a common backend is exciting only if you are 17 y.o. Instead of inventing a 'new' forum-like view I've decided to make a one-way HN to NNTP 'convertor', so that I can read HN in mutt. Like this:

https://raw.github.com/gromnitsky/hackernews2nntp/master/screenshot1.png

Why NNTP?

Because of a history of newsreaders UI, reading something that represents a newsgroup means:

  1. Being able not to read the same post (article) twice (the client software marks old articles).
  2. Local filtering. Highlighting favourite authors, hiding trolls, sorting by date, thread, etc.
  3. The offline mode (if you have a local NNTP server on your laptop).

Some time ago I've specifically wrote a Chrome extension for items 1-2, but have never impemented the custom thread sorting in it.

Moving your reading activities to mutt has its disadvantages:

  • No up-voting.

  • No score updates.

  • Once article is fetched & posted, it's very cumbersome to post it again if the content of it changes. You have to check w/ the server if it has the article w/ a particular message id, check for body differences, change the message id of a new article (otherwise the server will reject it as a duplicate), and possibly modify its References header to point it out to the old version.

    In short, I didn't do that. Once the article is posted it stays the same.

The original idea was to run some-gateway as a daemon that would have monitor for NH updates & would have immidiately convert new stories/comments. That turned out to be impractical because my laptop isn't on 24/365. Instead I took an old usenet path: donwload a bunch of articles & read them later.

The old way has 2 primary advantages:

  • There is no need to save the program state, because if we download an article twice (now & in the previous run), NNTP server will reject a duplicate.
  • It can help w/ HN addiction. You run some-convertor once a day & read all the interesting staff in your scheduled 'NH time'.

Then, if we use a decent article injector, it'll spool undelivered articles (for example if the NNTP server isn't responding) & post them in the next run automatically.

In the end, I run

  $ hackernews2nntp-get top100 -v | hackernews2nntp-convert -v | sudo rnews -N  

once a day & practically never visit the HN website.

You can read more about the convertor here: https://github.com/gromnitsky/hackernews2nntp

No comments:

Post a Comment