Monthly Archives: March 2011

Supermarketsthesia

· 2 Replies

The New York Times magazine recently ran a piece on the world of competitive memorization. The article was particularly interesting to me in its discussion of the apparently popular technique of “memory palaces”. I have some memory palaces of my own, though they are a far cry from the intentionally laid-out worlds used by crack memorizers. My “palaces” take the form of seemingly inexplicable connections between specific locations from my childhood and apparently unrelated topics from my modern life. I thought this was typical, but a recent chat with my wife suggested that maybe it’s not. So I thought I’d write about it here.

Here’s the first weird thing: my memory palaces are supermarkets.

When I was young – from a baby to, I guess, 9 or 10 years of age – there were two grocery stores in town where my family did most of its shopping. One was Denny’s, and the other was a Red Owl. I haven’t been in either one for 20 years; both of them ceased being grocery stores over a decade ago. But, for some reason, whenever I think of the reality show The Biggest Loser, I am immediately transported to the front-end of the frozen food aisle at Denny’s.

A mental stroll through the aisles of Denny’s evokes a number of similar associations. Thinking about the game show Jeopardy, and in particular Ken Jennings, makes me think of the produce aisle, especially heads of iceberg lettuce. A few years ago, I went on a kick where I read a bunch of books on fundamentalist Mormonism, and ever since, there’s been a connection in my mind between FLDS (especially prairie dresses) and the bread aisle. Some of the associations aren’t so vivid – when I think about moving along the back of the store, from the meat section toward the frozen food aisle, I think vaguely but consistently about whaling.

I am not making this up.

I have even more associations with the Red Owl. The strongest has to do with the magazine section, which was my favorite part of the store (I was super into sticker albums). I cannot think about Space Ghost (a cartoon I never heard of until college) without feeling like I’m in that magazine section. The bakery counter is the 2009 Star Trek movie. The deli meats section is the philosophical literature on artificial intelligence. The toiletries aisle is my 11th grade American history textbook. The cereal/candy aisle is a few things: towards the back, it’s Brazil; as I move towards the front, it’s conspiracy theories about the Kennedy assasination, and then about UFOs. There are a few places in the store that evoke something that I can’t quite put my finger on, such as the seasonal section (something about struggling to write a program that would combine or reconcile two data sets) and the front end of the bakery (something to do with airplanes).

I don’t think there’s anything monumental or deeply meaningful about these associations. I do find it kinda cool, though, that (a) they’re so persistent, (b) they’re limited pretty much exclusively to these two supermarkets, and (c) there are so few of them. Brains 4eva!

Redirect BuddyPress activity reply links to forum’s “Leave a Reply”

· 12 Replies

Activity stream replies in BuddyPress are pretty cool, but they have the potential to be confusing. On the CUNY Academic Commons, we have disabled activity replies for activity entries related to blogs and forums, because allowing replies in these cases has the potential to confuse users and fracture conversation.

There are a number of ways that this could (and should, and will!) be improved in future versions of BuddyPress. But, for now, here’s a trick. The following code will change the behavior of the Reply buttons for forum-related activity entries (new forum topics, and forum topic replies), so that instead of sliding down the inline activity comment box, it goes to the Reply form on the forum topic itself.

Side note: This seems like it’d be an easy thing to do, but it turns out to be somewhat complex. As I explain in the inline documentation, the issue of pagination means that there’s no predictable way to easily concatenate a URL for a topic’s reply box (this is one of the things I want to fix in BP core) – you have to fetch the number of total replies and figure out the last page from there. Also, in the case of topic replies, you have to do an additional query to get the id of the topic that the post belongs to, because that info is not stored in the activity table. The function cac_insert_comment_reply_links() below tries to consolidate these lookups to add as few queries as possible to the pageload.

Second side note: This code is not particularly beautiful. It makes direct queries to the bbPress database tables. So sue me.

OK, so the code itself. First, put this chunk into your bp-custom.php file.

[code language=”php”]
/**
* Gets accurate reply URLs for the activity stream
*
* Getting accurate Reply links for forum topics is tricky because of pagination – you need to know
* how many total posts are in the topic so that you can figure out what the last page should be.
* Moreover, the forum reply activity items don’t have the topic_id stored with them. This function
* attempts to minimize DB queries by looking up all topic_ids at once, then looking up all post
* counts at once – adding 2 queries for the activity loop is better than 20.
*
* Todo: Get a real redirecter into BuddyPress itself
*/
function cac_insert_comment_reply_links( $has_comments ) {
global $activities_template, $wpdb, $bbdb;

do_action( ‘bbpress_init’ );

$topics_data = array();
$posts_data = array();
foreach( $activities_template->activities as $key => $activity ) {
if ( $activity->type == ‘new_forum_topic’ ) {
$topic_id = $activity->secondary_item_id;
$topics_data[$topic_id][‘url’] = $activity->primary_link;
$topics_data[$topic_id][‘activity_key’] = $key;
}

if ( $activity->type == ‘new_forum_post’ ) {
$post_id = $activity->secondary_item_id;
$posts_data[$post_id][‘url’] = array_pop( array_reverse( explode( ‘#’, $activity->primary_link ) ) );
$posts_data[$post_id][‘activity_key’] = $key;
}
}

// In cases where we only have the post id, we must do an extra query to get topic ids
if ( !empty( $posts_data ) ) {
$post_ids = array_keys( $posts_data );
$post_ids_sql = implode( ‘,’, $post_ids );
$sql = $wpdb->prepare( “SELECT topic_id, post_id FROM {$bbdb->posts} WHERE post_id IN ({$post_ids_sql})” );
$post_topic_ids = $wpdb->get_results( $sql );

// Now that we have the topic IDs, we can add that info to $topics_data for the main query
foreach( $post_topic_ids as $post_topic ) {
$topics_data[$post_topic->topic_id] = $posts_data[$post_topic->post_id];
}
}

// Now for the main event
// First, make a topic list and get all the associated posts
$topic_ids = implode( ‘,’, array_keys( $topics_data ) );
$sql = $wpdb->prepare( “SELECT topic_id, post_id FROM {$bbdb->posts} WHERE topic_id IN ({$topic_ids})” );
$posts = $wpdb->get_results( $sql );

// Now we get counts. BTW it sucks to do it this way
$counter = array();
foreach( $posts as $post ) {
if ( empty( $counter[$post->topic_id] ) )
$counter[$post->topic_id] = 1;
else
$counter[$post->topic_id]++;
}

// Finally, concatenate the reply url and put it in the activities_template
foreach( $topics_data as $topic_id => $data ) {
$total_pages = ceil( $counter[$topic_id] / 15 );
$reply_url = cac_forum_reply_url( $data[‘url’], $total_pages, 15 );
$key = $data[‘activity_key’];
$activities_template->activities[$key]->reply_url = $reply_url;
}

return $has_comments;
}
add_action( ‘bp_has_activities’, ‘cac_insert_comment_reply_links’ );

/**
* Filters the url of the activity reply link to use reply_url, if present
*/
function cac_filter_activity_reply_link( $link ) {
global $activities_template;

if ( !empty( $activities_template->activity->reply_url ) )
return $activities_template->activity->reply_url;
else
return $link;
}
add_action( ‘bp_get_activity_comment_link’, ‘cac_filter_activity_reply_link’ );

/**
* Echoes the proper CSS class for the activity reply link. This is necessary to ensure that
* the JS slider does not appear when we have a custom reply_url.
*/
function cac_activity_reply_link_class() {
global $activities_template;

if ( !empty( $activities_template->activity->reply_url ) )
echo ‘class=”acomment-reply-nojs”‘;
else
echo ‘class=”acomment-reply”‘;
}

/**
* A replacement for bp_activity_can_comment(). Todo: deprecate into a filter when BP 1.3 comes out
*/
function cac_activity_can_comment() {
global $activities_template, $bp;

if ( false === $activities_template->disable_blogforum_replies || (int)$activities_template->disable_blogforum_replies ) {
// If we’ve got a manually created reply_url (see cac_insert_comment_reply_links(), return true
if ( !empty( $activities_template->activity->reply_url ) )
return true;

if ( ‘new_blog_post’ == bp_get_activity_action_name() || ‘new_blog_comment’ == bp_get_activity_action_name() || ‘new_forum_topic’ == bp_get_activity_action_name() || ‘new_forum_post’ == bp_get_activity_action_name() )
return false;
}

return true;
}
[/code]

You’ll note that there are a few places in that code where the number 15 is mentioned explicitly. I’m assuming that you’re using 15 posts-per-page for your single topic pagination. You can change this number accordingly if you want.

Next, you’ll have to make a few changes in your theme’s activity/entry.php to account for the changes. There are two relevant changes. First, you’ll be removing the activity reply button’s CSS class (hardcoded by default) and replacing it with the dynamically generated version in cac_activity_reply_link_class(). Second, you’ll be swapping out the checks for bp_activity_can_comment() with cac_activity_can_comment(), so that you can still block blog-activity comments. The code below is lines 27-29 of my activity/entry.php – you should be able to figure out which lines to replace with the following, as I haven’t changed much.

[code language=”html”]

()

[/code]

Finally, because you’ve changed the CSS selector on some of the reply buttons, you’ll want to add some styles to your stylesheet. These are borrowed right from bp-default.

[code language=”css”]
.activity-list div.activity-meta a.acomment-reply-nojs {
background: #fff9db;
border-bottom: 1px solid #ffe8c4;
border-right: 1px solid #ffe8c4;
color: #ffa200;
}

div.activity-meta a.acomment-reply-nojs:hover {
background: #f7740a;
color: #fff;
border-color: #f7740a;
}
[/code]

Good luck!

Making the Thunderbird interface more Gmail-y

· 5 Replies

As part of Project Reclaim, I’m gearing up to move my email off of Gmail and onto my own server. Email is, and long has been, central to my life online – it’s my main point of contact for so many personal and professional connections, and my email archives are the closest thing I’ve got to a record of my online activity. So I’m keen to make the move as smooth as I can.

For that reason, I’m handling the transition in stages. The first stage involves transitioning email client software away from the Gmail website.

Choosing a client

I’ve chosen Thunderbird as my alternative, for a couple reasons.

  1. It’s open-source.
  2. It’s highly extensible and customizable.
  3. It works across platforms. That’s important, because I’ll be migrating away from OSX.

Choosing Thunderbird is not without its sacrifices. For one thing, moving to an OS-native application, rather than an app that runs in a browser window like Gmail does, means that I’ll no longer be able to count on having a consistent UI and feature set between different devices. In some cases, this is not a huge loss. The UI for Gmail on my Android phone is really quite different from the normal web version, and it’s never bothered me very much. My biggest worry, though, is that I’ll have multiple workstations – a primary work machine and a netbook, for example – with different email setups. I’m hoping to mitigate the problem by coming up with some idiot-proof backup and syncing methods for the fairly small number of files that comprise Thunderbird’s configuration. (This’ll be necessary for other software transitions as well, like my gradual move to Vim. It’d be quite easy with Dropbox and some strategically-placed symbolic links, but I’m trying to break the Dropbox habit too 🙂 )

It’s a considerable comfort that, now that I have a smart phone (and am thus no longer reliant on public or borrowed computers for email access), the vast, vast majority of my email use is centered on a handful of devices, all of which I own. The last time I checked my email on a device other than my own was probably three years ago. Weird, now that I think of it.

The other sacrifice is related to UX. I happen to like Gmail’s interface. In particular, I’ve grown quite used to Gmail’s thoughtful keyboard shortcuts, which make it possible to do nearly navigation without touching the mouse. Coming up with a reasonable facsimile of these shortcuts in Thunderbird would be the biggest part of my configuration process.

Keyboard Shortcuts

Thunderbird has pretty good keyboard shortcuts out of the box. I didn’t feel like learning a whole new system, though, so I wanted a way to map Gmail-style shortcuts onto Thunderbird. There used to be a Thunderbird extension to do just that, called GmailUI. But the GmailUI website suggests that the extension is only compatible with Thunderbird versions 0.8-2.0 (Thunderbird’s currently in 3.1), which would explain why the extension doesn’t show up in a search on tho Mozilla repo (the “Expression Search” plugin does come up, which is a fork of a part of GmailUI that I’ll talk about in a minute – but it doesn’t do keyboard shortcuts).

So I looked for a more general method for customizing Thunderbird’s keyboard shortcuts, and found it with Keyconfig. It’s pretty straightforward to remap keystrokes using Keyconfig (Tools > Keyconfig), though it can be a bit of a pain because many of the standard Gmail shortcuts (like j and k for up/down navigation) are already in use by Thunderbird, so that changing one shortcut often means making two. You might find it helpful to borrow my configuration, which I’ve pasted below. Add these lines to your user.js config file (create if it doesn’t exist):

[code language=”javascript”]
user_pref(“keyconfig.main.key_killThread”, “][I][“);
user_pref(“keyconfig.main.key_markJunk”, “meta shift][J][“);
user_pref(“keyconfig.main.key_markReadByDate”, “meta shift][D][“);
user_pref(“keyconfig.main.key_markThreadAsRead”, “meta shift][R][“);
user_pref(“keyconfig.main.key_newMessage”, “meta shift][M][“);
user_pref(“keyconfig.main.key_newMessage2”, “meta shift][N][“);
user_pref(“keyconfig.main.key_nextMsg”, “][J][“);
user_pref(“keyconfig.main.key_previousMsg”, “][K][“);
user_pref(“keyconfig.main.key_reply”, “][R][“);
user_pref(“keyconfig.main.key_replyall”, “shift][R][“);
user_pref(“keyconfig.main.key_toggleMessagePane”, “][V][“);
user_pref(“keyconfig.main.xxx_key74_SwitchPaneFocus(event);”, “][D][“);
[/code]

Briefly, this does the following. First, it maps some familiar keystroke combos from Gmail to Thunderbird: j and k to up and down, and r and R to Reply and Reply To All. Second, because two-stroke codes from Gmail (like gi for Go To > Inbox) don’t seem to be supported natively by Thunderbird, I’ve mapped D to SwitchPaneFocus, which lets me get back and forth between the folders pane, the message list pane, and the single message pane, for easier navigation. As I get more comfortable with it, I might write my own extension that ports over some of the other most convenient Gmail shortcuts, but for now this covers a good 80% of what I might regularly use.

The other big shortcut missing from Thunderbird is y, which is the Archive command in Gmail. For that purpose, I installed Nostalgy, which allows you to move messages with keyboard shortcuts. I don’t think I’ve set this up in 100% the right way, but here’s how I’ve approximated Gmail’s y using Nostalgy. First, at Tools > Nostalgy > Keys, I’ve set ‘Save message’ to ‘shift Y’ and ‘Save as suggested’ to ‘Y’. ‘Save as suggested’ seems to work on a session basis; it suggests the folders that you’ve used since the last time you started Thunderbird. Thus, every time I start Thunderbird, the first time I want to archive a message, I use the more verbose shift-Y. A dialog at the bottom of the window suggests places where I might put the message; I select Gmail’s All Mail folder. Then, the next time I want to archive a message, I can use y by itself to go to the suggested (i.e. the last-used) location. Since I just throw all of my email into All Mail – no complex tagging or organization – this is all I need.

Bonus! Offline Access and New Email Throttling

Moving to a non-web-based email client is not all bad. For one thing, moving to a local application means that my email – archives and all – are available offline. I know that Google Gears and some of the new HTML5 goodies mitigate the issue somewhat with respect to Gmail, but offline access for a webapp is always going to be something of a hack.

On a related note, one thing that I have always hated about using Gmail is how it weaves together the process of checking for new email and accessing old email. I like to check for new mail at specified intervals only (once every few hours); anything more than that is extremely distracting. In Gmail, this meant closing the browser tab. Yet, fairly frequently, I find that I need to access an older email from my archive in order to do a specific task. In Gmail, this meant checking my new email as well. Now that I’ve moved to Thunderbird, I can access my archive without checking for new mail – the way it ought to be!

Setting it up this way was not straightforward. There are a few things you’ve got to do:

  1. In Tools > Account Settings > Server Settings, configure the ‘Check for new messages at startup’ and ‘Check for new messages every x minutes’ however you’d like. I have them both disabled, so that email can only be checked manually.
  2. Here’s the tricky part: Gmail, in its futuristic wisdom, uses a special protocol called IDLE to push new email to remote clients – bypassing the settings from step (1). (This took me half a day to figure out.) Disable this feature at Tools > Account Settings > Server Settings > Advanced > ‘Use IDLE command if the server supports it’.

I’m planning to spend a few weeks improving and getting used to this setup before starting the migration to self-hosted email.

Project Reclaim

· 40 Replies

Update: I have begun aggregating these posts at projectreclaim.net.

Lately I have been feeling increasingly uneasy about the state of my digital affairs. I am a leader on a number of open source software projects that pride themselves on, among other things, their ability to enable users to “own their own data”. Moreover, I am trained as a philosopher, and have spent a pretty fair amount of time reading and thinking carefully about the nature of data and our relationships with it. If anyone is in a position to develop and advocate for good models of digital independence, I am.

Yet, when I look around my digital world, I see instance after instance where I am, to a greater or lesser extent, completely reliant on the good will of commercial entities and their propietary systems. To wit:

  • My Twitter account is a big part of my online identity
  • The last five years of my private correspondence, personal and professional, is in Gmail
  • I use Dropbox for syncing documents between devices (like my blog_sandbox.txt file, where I’m writing this post!)
  • I use Picasa Web Albums to back up and share photos
  • I have a Mozy account to back up the rest of my important files
  • Until recently, I had an iPhone. I still use a Mac
  • I use Remember The Milk for task management
  • I store source code for all my projects in Github

Some of these are products; some are services. Some are free; some of them I pay for. And – for sure – some of the companies behind the products and services listed above are more evil than others. So I don’t want to pretend that my reliance on each of them is equally bad. But each item on this list plays a crucial role in my digital life, and each one of them operates in a way that is beyond my control, both literally (I can’t modify the source code) and more figuratively (questions about ownership, exportability, transportability are icky).

I’m planning to extricate myself.

Project Reclaim

In order to make it sound a bit fancier, I’m giving my project a name: Project Reclaim. ‘Reclaim’ because it’s a manifestation of my desire to fight the inertia that leads us to give up control over our computing experiences, my desire to reclaim control and ownership. ‘Project’ because this will be hard, and ongoing. And why give it a name at all? I’m hoping that, by being public about it – putting my experiences in a series of blog posts and tweets under a common tag – that I’ll be able to hold myself accountable, and hopefully guide others who are hoping to reclaim their lives a bit as well.

In short, Project Reclaim is the process of weaning oneself off of digital platforms that are closed source and/or under the control of others.

Methodology

How will Project Reclaim actually work?

  1. Assess the situation I’ll first need a way of figuring out which systems and platforms are worth moving away from, what their replacements should be, and in what order I should effect the transition. I’ve got a few rules of thumb.
    • Open source is better than closed source. I write open-source software for a living. I believe that, on balance, it makes better software. And I believe that using software where one has access to the source code is a necessary component of maximizing one’s digital autonomy. Thus: if the third-party system I’m currently using is also a benefactor of open-source communities (like, say, wordpress.com), it makes it less urgent to move away. And, when selecting replacements, select open source if at all possible.
    • Paying is better than getting something for free This might seem like a contrast to the previous rule, but I don’t think it is. When you use a free service, somebody’s paying the bills. Usually that means targeted advertising – think Facebook and Gmail. Paying service fees, on the other hand, and agreeing to the contract that comes with it, generally has the effect of making the relationship more transparent. Of course, this is far from absolute, but it seems reasonable in a broad sense. Plus, I like to support developers and services that are truly valuable.
    • Go for the low-hanging fruit In cases like email, there are well-established, straightforward (though not necessarily easy…ugh) ways of fending for yourself. No need to invent the wheel. On the other hand, some of the areas where alternatives are less obvious – social networking-type data springs to mind – also happen to be areas where I have some expertise and leverage. So, in those cases, it might be worth innovating.
    • Get the important stuff first My email history is more important to me than my Twitter history; the convenience of Github is more valuable to me than the convenience of Dropbox. Plan the Reclaim accordingly.
    • Get the vulnerable stuff first Recent statements by Twitter have made me think that the way I interact with the services is more subject to change in the upcoming months than, say, the way I interact with Gmail. That’s frightening. The more profit-hungry the company is – and, thus, the more disinclined to have the customer’s freedoms in mind – the more urgent it is to pull yourself out.

    Clearly, some of these considerations are at odds with each other. But they give a rough framework for deciding whether, when, and how to carry out the mission of Project Reclaim.

  2. Make the switch Here’s where the action happens: I do what I need to do to move myself to the replacement.
  3. Write about it This weekend I spent an afternoon on the problem of Twitter, and I ran into a ton of technical problems that remain unresolved. I imagine that there will be similar hurdles for each part of the project. I’m hoping that, by writing about the problems (and, where they exist, the solutions) I can help other people to take some of the same steps themselves, or even to spur someone really smart to come up with better solutions than the ones that currently exist.

What I expect from myself

My goal, ultimately, is to move away from third-party, closed-source services and platforms altogether. It might take some time. So I’ll make some interim goals: by the end of 2011, I’m hoping to have my email moved, my microblogging federated, my own backup system on my own server space, and my computer running an open-source OS.

Even if I manage to meet this goal, there’s a very real sense in which Project Reclaim will necessarily be an exercise in futility. I’ll always have to buy server space, and who’s to say that Amazon or Slicehost won’t go berserk tomorrow? I’ll always have to connect to the internet, which leaves me perpetually at the mercy of the ISPs, who are IMO more evil than all of the other service providers put together. It’s a depressing state of affairs: the kind of autonomy I want might be impossible given the way that the economy works. I take some solace in the fact that philosophers have spilled much ink over the problem of free will without coming up with a clear formulation of exactly what kind of autonomy would be worth arguing for. At least I’m not alone in my delusion.

That said, it’s a fight that I feel I have a responsibility to fight. If I’m going to continue to argue for the use and development of open source software, I have to start putting my money where my mouth is. And so, to me, Project Reclaim is less about my being a paragon of virtue, and more about my wanting to sleep a bit better at night.

Do I think that everyone should do this? People should prefer open solutions to closed ones, all things being equal. But generally, all things are not equal. Most people don’t have the time to write their own software, to run their own servers. For those people, decisions about their digital life are (rightly, I think) made more on the basis of aesthetics and convenience than lofty concepts like Autonomy and Ownership. But there are a few considerations that are perhaps relevant for the kinds of people who read my blog:

  • Open source developers who tout the importance of data ownership and other such freedoms have a special responsibility to model best behavior in these areas.
  • Academics, more than anyone, should be sensitive to the dangers of leaving the crucial pieces of one’s online self in the hands of corporate entities. That’s true for personal artifacts like email, but perhaps doubly so for scholarly work that ought to be part of a public trust.
  • Educators, like open source developers, should model best practices, encouraging students to take control over their digital identities.

So, while I wouldn’t belabor the point for the average Joe, I do think that people who consider themselves members of one of these groups – as most people reading my blog probably do – that they should think carefully about their relationship with the tools and services that enable their digital existence.

To freedom!

New BuddyPress plugin: BuddyPress Docs

· 5 Replies
BuddyPress Docs edit screen

BuddyPress Docs edit screen

Today I am releasing the first public beta of a significant new BuddyPress plugin: BuddyPress Docs. BuddyPress Docs is a collaborative, front-end, rich-text, document editing tool for BuddyPress groups (individual user Docs are an upcoming feature). Read much more about the plugin’s features.

This plugin has been developed for the CUNY Academic Commons (though it won’t be live there for a few weeks). Today’s release is a pre-stable beta – it should run fine, but there are bound to be bugs, and you probably won’t want to run it on a production site quite yet.

You can follow the plugin’s development at http://github.com/boonebgorges/buddypress-docs.