Self-hosted moderated chat room and liveblog project

A few months ago I contacted Brian Cook, proprietor of one of the best sports blogs on the Internet, MGoBlog. A year or so earlier I had created a userscript that improved some functionality for his site, and I thought he might be interested in building some of that JS directly into the site so it would benefit every user. Instead we ended up discussing a few other projects, including an open-source alternative for the relatively expensive moderated liveblog systems the site uses for Michigan football gamedays. These are the results of that project so far.

[Skip all this and go straight to the code on GitHub]

The system is divided into two sections, one for users and one for moderators. Users submit comments which are stored in a MySQL table as a “raw” feed. Moderators see this raw feed in real time thanks to a recursive JavaScript function making repetitive AJAX requests to a PHP service that acts as an endpoint for the raw MySQL table. If a moderator approves a comment out of the raw feed (or makes a comment themselves), that comment is posted to another MySQL table (the “filtered” feed) where it populates into users’ views by way of similar repetitive AJAX calls.

If you like flowcharts to visualize this sort of thing, here’s an attempt at one:

Moderated Liveblog Flowchart

If you want to see the entirety of the code (with lots of helpful comments), you can check out the project on GitHub, I’ll just go over a few of the more interesting points here.

Managing the AJAX requests

The first problem I tackled on this project was how to make the recurring AJAX requests. My first thought was to simply use callbacks to kick off a new AJAX call as soon as an old one finished. I decided against this because it left no room to add a delay, which might be necessary under high traffic loads or on underpowered servers.

The next option was to go strictly with a delay, make a new AJAX call after X milliseconds. The problem with this approach was the if the delay was set too low or if some users had very low connection speeds, AJAX requests could end up backing up, with new ones going out before the old ones had completed. I solved these issues by creating a series of cooperative functions that make recursive calls only after a specified delay time has elapsed AND the previous AJAX request has completed.

This starts by declaring two boolean values, one to track whether the most recent request has completed, and another to track whether the specified delay has elapsed since the last request was made.

The update function sets both of these values to true, and sets a callback for the successful completion of the AJAX request and also specifies a timeout function.

In the AJAX callback function we set the boolean tracking the progress of the request back to false, and check to see if the boolean marking the delay progress has also been set back to false. If it has, we call the update function again and make another AJAX request.

In the delay timeout function, we do something very similar. First the “delay” boolean is set back to false, then the “update” boolean is checked. If it’s also false the update is run again. This ensures that a new AJAX request will be made either after the minimum delay occurs or after the last request completes, whichever comes last.

Returning the data from the server

As you may have noticed in the update() function above, each AJAX update request contains a parameter named “id” that notifies the server of the most recent comment returned by the server on the last request, that way the server will return only new data.

When the server receives this information, it will connect to the database and make a query for all entries with an ID greater than the ID passed in the request. The ID column on the database is an auto-incremented primary-key indexed column, making queries against it very light and fast.

PHP’s mysqli extension is used for all interaction with the database, and all queries are made using prepared statements to maximize security.

Once the query is made an array is created to hold all the returned items. Each element within this array is itself an associative array containing individual column data from each item. This larger array is then converted into a JSON string using json_encode() and returned to the browser where JavaScript can parse the response and insert the content.

Handling videos, images, and links

One of the requirements of this project was some ability for users to embed videos and images in their posts and have them render appropriately. I accomplished this by having JavaScript parse through all content before it gets placed on the page and search for anything formatted like a URL. All matching strings are then checked to determine if they are YouTube videos, images, or plain links. YouTube video links are converted to iframe embed containers, images are converted to simple img tags, and all other URLs are simply turned into active links.

For simplicity (to avoid having to write some monstrous regular expressions), I wrote this tool to iterate over every word of content. Since most applications of this liveblog will involve only very short comments this shouldn’t have a noticeable effect on performance. However, if this tool were ever used to parse through longer blocks of content it’s very likely that it could become bogged-down rather quickly, and that the monstrous regular expressions mentioned above would be preferable.

And just in case this function encounters an error, a try ... catch block is used when inserting content.

Tagged with: , , , , ,
Posted in GitHub, HTML & CSS, JavaScript, jQuery, MySQL, PHP, Server Side Code

Leave a Reply

Your email address will not be published. Required fields are marked *