FeedWordPress works well if you want to syndicate content from various sources into a single Wordpress blog. Syndicating comments is, of course, more difficult. I’m finishing up a job for a client who wanted real-time synced comments, and suggested that Disqus might do the trick. I quickly discovered that Disqus is clearly not made to do what I wanted it to do. But, being the cool guy that I am, I hacked something together that is more or less functional.
Here were the requirements: Comments on a blog post needed to be synchronized between the source blogs and the hub blog. Readers had to be able to comment in both places and have the comments sync. While I’d be using Wordpress to create the hub blog, the source blogs would be hosted on various platforms: Tumblr, Typepad, Blogger, self-hosted Wordpress. (The distributed requirement is especially important. If the blogs were all on the same installation of WPMU, the job would be trivial and would not require a third-party solution like Disqus.) Because bloggers would be coming from different platforms, I not only had to be able to accomodate those platforms, but I also had to make sure that the system would work with the platforms’ stock configuration. That is, since I (and, generally speaking, the bloggers) don’t have access to the platform code, all custom modifications need to happen at the hub blog.
I don’t particularly recommend that anyone try to replicate what I’ve done here. But hopefully it will point the way toward what might be a viable third-party system for true comment syncing.
The details
Here’s my strategy with regard to Disqus. If all the source blogs were registered to the same Disqus Comments account (ie corresponding to a single shortname), then they’d all have the same forum_key, which is to say they’d be accessible by the same API request. Thus the strategy is to make Disqus unable to distinguish between API calls from the source blogs (which are, recall, making stock API calls to Disqus) and API calls from the corresponding posts on the hub blog.
I installed the Disqus Comment System plugin for the Wordpress hub blog and registered with the same credentials that would be given to the source blogs. When feeds starting syndicating to the hub blog, however, I found that the comment sections on the source post weren’t matching the comment section on the hub post. The URL for each comment thread’s RSS feed showed me why: Disqus indexes a forum’s comment thread based on some post information that it gets from the client platform, and each platform was formatting the information in a different way.
First problem: The Wordpress Disqus plugin uses a post variable called $thread_meta, which is set in disqus-comment-system/lib/api.php thus:
$thread_meta = $post->ID . ' ' . $post->guid;
Disqus would then create a comment thread based on this string. The problem is that $post->ID is the post ID number for the hub blog, and has nothing to do with the source blog (which, depending on platform, does not include post ids in its API request at all). So the source blog’s thread would be identified as test_post (for example) while the hub blog would be 34_test_post. I replaced the code above with
$thread_meta = $post->guid
which manages to stay pretty consistent across platforms. (NB: The same change has to be made on the source blog version of the Disqus plugin, if the source blog is running a self-hosted installation of Wordpress.)
Second problem: Getting a stable and unique identifier for each post thread is only the first step. You also need to make sure that the identifier is concatenated correctly when the actual API request is made. Disqus comment sections work by loading a piece of Javascript that is concatenated from an API request to disqus.com for the proper thread, then finds the comment section on the post page, and replaces the native comment code with the code returned from disqus.com. But I found (again, by looking at the URL for the RSS feeds) that each platform was making the request a little bit differently. At the end of disqus-comments-system/comments.php, the stock WP plugin reads
<script type="text/javascript" charset="utf-8" src="http://./disqus.js?v=2.0&slug=&pver=">
Through a fair degree of trial and error, I replaced it with a big block of code that figures out (via some metadata created by FeedWordPress) which platform that particular post came from, and then modifies the javascript accordingly:
ID, 'syndication_permalink'); ?> ID, 'syndication_source'); ?> <script type="text/javascript" charset="utf-8" src="http://./disqus.js?v=2.0&slug=&pname=wordpress&pver="> <script type="text/javascript" charset="utf-8" src="http://./disqus.js?v=2.0&slug=&pname=wordpress&pver="> <script type="text/javascript" charset="utf-8" src="http://./disqus.js?v=2.0&slug=&pver=">
In the first few lines, I do a bit of string manipulation to standardize the post title (ie the unique post identifier from Disqus’s point of view). Then I do some very ugly stuff. Wordpress was converting the em-dash (which was all over the client’s blog) into an ASCII code, which was screwing up the post identifier, so I just str_replaced it out. The next part (with the preg_match) is a bit tricky: in some cases, when Disqus receives requests from two blog posts with the same title (as is the case with the source blog post and the hub blog post), it differentiates between the two by assigning an apparently random two digit number to the second request it gets. Since the syndicated Disqus request will generally be sent after the source blog’s Disqus request (in virtue of its being syndicated), and therefore will be the one to be appended with the two-digit number, I figured I could just look for ‘_xx’ (where xx is a two digit number) at the end of the post title and strip it off. Ugly ugly ugly, but it works. The rest of the code just rearranges the javascript according to which platform the original post comes from, which in some cases requires the addition of source blog name.
With all this in place, I’ve got the following: A blogger posts on, say, his Tumblr blog, where Disqus is enabled. The post is fetched by FeedWordPress on the hub blog, where Disqus is also enabled, with the same user credentials. Then the hacks listed above trick Disqus into thinking that the syndicated post is the very same as the source post, so that the very same comments section is sent to each post. Kind of like magic, when it actually works.
Clearly, though, it would be much, much easier with a system that is built to do what I’m trying to do. That means, in part, having a single system for identifying posts across platforms (some appropriate htmlization of the post name, I presume) and then a single, unified system for making API requests.
I think Disqus is awesome with facebook great to go..
This is an interesting approach to a huge problem for us. The idea of syndicating comments cleanly is the finishing touch for FeedWordPress, I like how WordPress.com shows the number of comments on a post that is syndicated, and you can see those comments on the original post. One issue though, and an ongoing one, is archiving those comments on the original course page, or even people owning their own comments, which co-comment tried to get at, though not too successfully.
The only issue I have with Disqus and intense debate, etc, is that the are services, not standards, for getting these comments syndicated back and forth and giving people some control over those comments. And now that wp.com immediately gives you an ID account and immediately locks you in suggests a kind of battle for this space that wouldn’t integrate with Disqus that cleanly, but I may be wrong. Since about 2.7 on WP/WPMu the comments were associated directly with posts, which is new, and I don’t know enough code, and this wouldn’t solve larger cross-platforms issues, but it would possibly allow direct syndication on a post-by-post basis. But how TypePad, Blogger, and the like do that is a larger issue. Stephen Downes suggested that the way to solve this is to link commenting with a personal identity online through something like OpenID and the like, which is what ID and Disqus are approaching. But the thing about these two services is they are services not something you host and own yourself.
It remains an ongoing question, but given the roll you are on, it may be worth a shot 🙂
Thanks for the comment, Jim. I agree that the issue of comments is a big problem, in fact maybe the biggest problem with the syndication bus model you so brilliantly put forth.
The crux of the problem is not necessarily that syndicating comments is hard to do. It’s not. Even before WP stored comments in blog-specific DB tables, there was always enough information about the comment-post relationship to provide the infrastructure for creating comment feeds, etc. The problem is that comments are not unidirectional. When I write a blog post, WP acts as a hub that sends the immutable text out to my hundreds of thousands of millions of avid readers. Comments have to work in the other direction, with users sending information my way. Right now the only real way for that to happen is for them to actually direct a browser to my blog. There’s the problem: you can write a blog post via XML-RPC, via e-mail, or via some other API, but not comments – that’s platform-lockin, or UI-lockin, which is the opposite of platform-agnostic standards.
So comment writing is one part of the problem: coming up with a way that people can comment without visiting my blog. By extension, and more closely related to the FWP point, people should be able to comment on my blog by commenting on another (aggregator) blog. The Disqus solution I hacked together here does a good job of that, and I could easily write a plugin that would allow you to do that on a single installation of WPMU (btw, oh shit, I meant to do that last week), but you can’t do it in a truly distributed way.
The other conceptual distinction between comments and blog posts is that, to a certain extent, blog posts provide their own context. Thus when a post from the bava comes through my feed reader, I don’t necessarily need to leave the reader to know what’s going on. Comments are almost always very post-dependent for their content and context, so they don’t make much sense by themselves. This calls for some sort of new innovation in reader design – in other words, I think it’s a UI problem, not a standards problem – that pulls comments from places I’m interested in and allows me to read them in context without having to visit 25 different blog sites throughout the day. Pages like iGoogle are the start of a way to think about what such a UI might be (totally personalizable portals that bring together content in a context that you get to determine, rather than the linearity of the feed reader) but a lot more work remains to be done, by people smarter than I am 🙂