News Recommendation Algorithm

Done!  I threw together a quick recommendation algo over lunch and it worked out great so it sticks.  It’s on the Skimfeed front page right now.  There are no plans to roll the recommendation engine out to all individual category pages yet, but it is an option.  Server load is the only issue.

The news recommendation system is a dynamic element.  This means it doesn’t scale too well at traffic peaks and is currently taking 0.04 of a second to work it’s magic.  That’s nice, but not so great when you have more than 20 visits/second.

Some rearranging with the CSS was required to get the boxes to grid-up nicely.  Here’s the layout structure right now depending on how many boxes you can fit across:

• 2 boxes – Popular and Latest.
• 3 boxes – Popular, Latest and Recommendations if you have them, else Random.
• 4 Boxes – The full flavor.  Popular, Latest, Recommendations if you have them and Random.  The sweet spot.

Getting this layout just right was suprisingly challenging, yet satisfying in a way hackers will understand.

A tag cloud was also tried.  It didn’t work out too well because of the lack of keywords coming down the pipe.  I didn’t feel like building out a list of “words to ignore” like “the” and “this”.   A news tag cloud would be a nice touch to get an overview of popular news topics/words.  It would provide a quick way to see what’s hot.  Like zooming all the way out in Google maps.

Skimfeed is now feature complete (le tick).  New todos are:

• Rebuild the entire backend for optimal speed and flexibility.
• Squeeze out tweaks to the design and layout.
• Add more feeds.

Yeah that’s it.  I’m happy, I use Skimfeed.com  at least twice a day.  It’s my morning routine, followed by a little HN for comments.  I no longer use Reddit, Fark, Slashdot or any other news aggregaters at all.

• Mission accomplished.
• Time saved
• Knowledge acquisition up
• Knowledge discovery up
• Interaction with 12 year olds – down.

Comments on Skimfeed – like the last update, it’s a yes and no.  I’d like them, I just don’t want to deal with the moderation duties and I don’t want to waste your or my time on idle chat.  I do have an idea for where I’d fit them into the layout though…we’ll see.

Anyhoo, keep the feedback coming and news feed suggestions are ALWAYS WELCOME AND NEEDED.  Like seriously, we have 10 gaming feeds right now and one of them is Kotaku.

The end.