Category Archives: wordpress

One of the suckers

At the beginning of the 4.2 dev cycle, I was made a permanent committer for WordPress. (Cool!) Here’s how Andrew Nacin summed up the promotion:

Suckers

Suckers

Any maintainer of a large free software project will recognize the aptness of the “suckers” comment. I’ve spent more than five years as a leader on the BuddyPress project, and during that time I’ve developed a devotion to that project that sometimes feels like more like servitude than volunteerism. I feel a personal responsibility to the users of BuddyPress, which manifests itself as a pang of guilt about every bug, every missing feature, every unsatisfied customer. This guilt is part (not all, but not an insignificant portion) of what keeps me committed to the project. The same guilt sometimes makes me feel like a sucker.

With BuddyPress, the “sucker” aspects have been consistently counterbalanced by (a) the belief that my work is having a broad positive impact, and (b) the positive effect that contribution has on my reputation and my marketability. Point (b) is one that I’ve blathered on about on multiple occasions. My ability to make money as a consultant is directly tied to the reputation that I’ve earned doing free software work.

Since I started investing lots of energy into WordPress itself about nine months ago, I’ve been forced to reassess the “reputation cycle” somewhat. WordPress powers tens of millions of sites, as compared to BuddyPress’s tens of thousands. By this metric, the free software work I’ve done since September has had a far wider impact than any work before then.

You’d think that this increased impact would translate directly into a corresponding increase in reputation. Yet it hasn’t, at least not if you measure reputation by job offers. This time last year, I was turning away probably three or four freelance gigs per week, and probably one or two full-time jobs per month. I don’t get a fifth of that amount today.

This is not to complain – I am doing just fine, thankyouverymuch. But it’s worth thinking about the possible reasons why increased impact and productivity don’t always translate to more work. I think there are two big ones.

First: The further down you get in the technology stack, the more it feels like plumbing. I’ve built a number of major user-facing features for BuddyPress over the years, the kind of things someone might point to and say, “Boone built that”. My work on WordPress has been a lot less visible, and thus a lot harder to brag about. This is the truth in Nacin’s “suckers” comment. No one praises the plumber when the toilet flushes properly, but boy do people complain when it does not.

Second: I joined the WP core team around the time I stopped using Twitter. The time I used to waste looking at Twitter is time I now sink into doing meaningful work on projects like WordPress, which is to say that I’m more productive now than I used to be. But I’m talking less, and talking less about myself. If the Boone brand is in decline, it’s a publicity problem, not a quality problem. This is another side of the “sucker” coin: the greater the percentage of your time you devote to being productive, the less time you have to talk about how productive you are.

The number one reason to start using git: interactive staging

Git is an excellent tool for collaborative software development in a number of ways. In my opinion, Git’s most valuable feature – which also happens to be one of its most underappreciated – is interactive staging: git add -i and git add -p. If you are a software development who does not use Git, or if you use Git but do not use this feature, you should not write another line of code until you learn it.

When working on a large project, there is no principle more important than that changesets are atomic. Combining more than one distinct change into a commit is highly detrimental to a project’s history. Muddled changesets are hell for future developers on the project, including Future You. The usefulness of log, blame, bisect, and (not to get hyperbolic or anything) version control itself, depends on commits being logically small units.

But, of course, software is not built linearly. You’re writing cool new feature X, but you notice that in order to complete X, you need to fix bug Y. The problem is that you’re already halfway finished with X. You could: generate a diff file, stash/reset your changes, fix Y, commit the fix to Y, reapply the X diff, and continue work on X. This is a pain. Or you could: commit a large changeset that contains X and Y. But this is lousy for the reasons described above. Ideally, you would be able to fix Y, continue with X, and then sort out the changesets just before commit.

This is what Git’s interactive staging lets you do. Start with a clean index. Begin hacking on X. Stumble on Y. Fix Y. Complete X. Then use Git’s stage to commit Y and X separately.

Here’s a real example. (What follows is a pretty basic situation. git add -p is a subset of git add -i, which is full of whiz-bang goodies.) Say I’m working on fixing some poor localization in BuddyPress. While doing so, I notice that some PHPDoc is missing. So I fix both issues, and git diff shows the following:

is1

Now, I know that Changesets Should Be Atomic, so I want to commit the documentation changes separately from the bug fix. So, instead of git commit -a or git add . – which blindly stage all changes – I jump into patch mode with git add -p:

is2

Git has determined what it considers to be the first “hunk” of changes. (Hunks: it takes one to know one.) I’m then asked whether I want to “Stage this hunk”. Normally I’ll just type y or n, but in this case I see that Git hasn’t made the hunks small enough, so I choose s, for “split”:

is3

The hunk is now split into two. The first sub-hunk is the documentation fix. Let’s stage it with y. The second sub-hunk is the code fix, which we’ll skip for now with n. Rinse and repeat with the second big hunk. git status now shows the following:

is4

The “changes ready to be committed” are the documentation fixes. These can be viewed with git diff --cached. The “changes not staged for commit” are the code changes. These can be viewed with git diff. I’m now ready to commit the first set of changes:

is5

(I typed that commit message in my $EDITORvim – which you can’t see in this screenshot.) Then I’ll repeat the routine for the next set of changes, this time staging them all:

is6

Git’s interactive staging is relatively simple, but will completely change the way you work, in ways that will result in meaningful improvements to your projects over time. This is, IMHO, the #1 reason to use Git, and if you’re not using it – because you’re an SVN user, because you didn’t know about it, because your GUI client doesn’t support it – it’s important enough that you ought to think about changing your toolkit.

phpunit-speedtrap and WordPress/BuddyPress automated tests

One of my personal missions over the last six months has been to shave seconds off of the WordPress and BuddyPress automated test suites. (WP’s tests run in under half the time today than in WP 4.0 – and with the addition of nearly 1000 tests. Score!) One of the tools I use to track down problematic tests is John Kary’s excellent phpunit-speedtrap, which adds a listener to each test run, and produces a report of tests whose running time exceeds a configurable threshhold. phpunit-speedtrap is designed to be used as a Composer dependency, but this is not currently convenient or necessary for the purposes of working with WP/BP (for one thing, I’m the only person doing it). Here’s how I’ve rigged it up to run locally:

  1. Grab a copy of the listener class from Github https://github.com/johnkary/phpunit-speedtrap/blob/master/src/JohnKary/PHPUnit/Listener/SpeedTrapListener.php. I chose to remove the PHP namespacing, but you can do as you wish. Save it somewhere – I put it at ~/.speed-trap-listener.php so that I can use it with all projects.
  2. WP and BP ship with a phpunit.xml.dist config file. The .dist extension means that you can run your own phpunit.xml alongside of phpunit.xml.dist – PHPUnit will prefer the non-dist version if available, while WP and BP’s version control config will ignore it. Copy phpunit.xml.dist to phpunit.xml and add the following block:
    [code language=”xml”]   


    250


    [/code]
    Change the slowThreshold and filepath to whatever you’d like.
  3. Run phpunit. You’ll see something like this:
    Screenshot_2015-03-07_11-50-30

Keep on shavin’!

WordPress 4.1 and me

WordPress 4.1 “Dinah” was just released. In September, I was asked to do a stint as a committer. It was a blast. Here’s what I worked on:

Query improvements

My primary goal for 4.1 was to rework the WP_Meta_Query, WP_Tax_Query, and WP_Date_Query classes so that they supported nested query syntax. In the process, I worked through much of the backlog of Trac tickets related to these three classes, fixing some longtime annoyances and adding a few nice features. I published an in-depth write-up on these improvements in October.

Taxonomy

I decided to use my new position to stir up trouble related to the taxonomy roadmap. My goal was to make some progress on Trac’s Worst Ticket. I wrote a couple boatloads of unit tests for functions related to term editing and creation, helped shepherd through fixes to a few edge cases related to duplicate term creation, and pulled the trigger on changing the wp_terms database schema. In #21950 we stopped creating new shared taxonomy terms, and we came very close to splitting existing shared terms in this release, but pulled it out at the last minute so we’d have a bit more time for developer education. I look forward to continuing my unofficial role as Taxonomy Gadfly in future releases.

Unit tests

I contributed hundreds of automated tests to WP during the 4.1 cycle, and so the slowness of the phpunit suite was a particular pain point for me. Based in part on some ideas of nacin’s at WCSF, I took a few steps toward reorganizing and refactoring a couple parts of the phpunit suite, with the end result that running $ phpunit on WordPress takes less than half the time today as what it took in 4.0, despite the addition of many, many tests. I’m equally proud that I was able to help a few contributors to include unit tests in their WP patches for the first time.

Comments

I spent some time in 4.1 improving comment queries. I reworked comments_template() (the function used by WP to display comments on a blog post in most themes) to use the internal comments API, resolving a ticket that has bugged me for a long time. I also helped to add some additional query params to WP_Comment_Query, and worked with a number of contributor to pave the way for a better system for custom comment types in the future.

Miscellaneous

I took advantage of what may be a once-in-a-lifetime opportunity to address a couple personal peeves, like better support for apostrophes in email addresses. It was also a pleasure to do a bit of developer outreach, in the form of helping a few folks to get their first WordPress props, as well as clearing out some old has-patch tickets.

Working on a huge project like WordPress has been a lot of fun. 4.1 FTW!

wp-cli-git-helper: Git commands for wp-cli

At this year’s WordPress Community Summit there was an interesting session about the many uses of the delightful wp-cli. One idea was that wp-cli could handle some simple git commands intelligently, so that, for instance, updating a plugin would automatically generate a changeset for the update with a descriptive message. After some back and forth with the wp-cli team, I decided to build this as a community package. It’s available here: wp-cli-git-helper.

This will be a huge time saver for me as I manage client sites. If it helps you, that’s super too. Pull requests welcome for Kewl New Features.

Unpaid labor in academic and free software communities

There are many aspects of my current free software development work that are (thankfully!) very different from my previous life as an academic. But one way in which they’re similar is the way that one’s relationship to one’s own paid and unpaid labor is connected to one’s career progress, and the personality types that this structure attracts.

I’m a known advocate ([1], [2], [3]) for fostering a symbiotic relationship between my paid client work and my unpaid work on free software projects. And while I’m emphatic that there’s value in having these two parts of my career separate from (yet supportive of) each other, the separation embodies an unavoidable tension between what I’m paid to do and what earns the respect of my peers. If people know who I am, it’s probably because of volunteer work I’ve done for WordPress, not because of my client work. As such, there’s continual internal pressure for me to focus more of my mental and emotional energies on the unpaid work. Yet it’s important not to yield completely to this pressure, since my paid client work is critical, both in terms of the financial support and the technical inspiration it gives to my work on the free software projects. Balancing these two pressures is something I’m constantly struggling with.

The relationship between labor and rewards in academic work is similarly structured. Most academics are paid primarily for teaching duties, with service and research being important but often secondary, at least as far as the official job descriptions are concerned. Yet the system of advancement in academia is structured in such a way that one’s research and publication record is of paramount importance. And in many cases, volunteer labor – things like peer review and service to professional societies – is critical to one’s reputation as a scholar. Anyone in the academic world will recognize the tensions that this arrangement can produce.

The peculiar motivations baked into free software development and academia tend to attract similar sorts of overachievers. To rise to the top of your field, you’ve got to do large amounts of unpaid labor, while still doing enough of your paid labor to keep your job. This means that the most successful people tend to be those who are spending the greatest amount of their spare time working for free. A couple of consequences fall out of this arrangement. First, people who are already in a position of privilege (financial and otherwise) are able to climb the career ladder more easily. This setup also means thatt successful people are likely to have a sense of self-worth that is closely connected to their work. And these factors mean that successful academics as well as free software contributors are more likely to suffer from burnout.

Last year, DHH of Ruby on Rails wrote an interesting piece on “the perils of mixing open source and money”. I’m very sympathetic to many of his points about motivation: the tenor of a free software project, and the quality of the software that results, is largely a consequence of the fact that the creators of the software are not primarily motivated by financial concerns. This is something that academics figured out a long time ago. As such, I think that it’s important to continue to foster “reputation cycles” and other structures that help to enable talented developers to devote energies to free software without directly paying them for it. At the same time, it’s important to be aware of the kinds of tensions I describe above, because the separation of paid and unpaid work in this area can tend to be personally destructive at the same time that it’s valuable for the (software/academic) projects as a whole.

WordCamp NYC video now available

In August, I gave the keynote presentation at WordCamp NYC. A few days later, I wrote out a condensed version of those remarks and posted it on this blog. The video for that talk was posted today to wordpress.tv and is embedded below for your viewing pleasure. Some of the meat of the argument is the same as my recent talk at WCSF, but the NY talk is more focused on freelancers, and has funnier jokes.

Disable ext4 barriers for dramatic I/O performance improvement on local machine

I was experiencing an issues that made PHPUnit unit tests run increasingly slowly on one of my development machines. The tests that ran the slowest were those that touched the database in some way, which in BuddyPress and WordPress unit tests are most of them. After a bit of poking around, I came across this thread: http://stackoverflow.com/questions/14914467/slow-phpunit-tests. I took the advice of the accepted answer and set barriers=0 in /etc/fstab and it has made a very dramatic difference in PHPUnit execution time: the BuddyPress test suite, for instance, now takes about 30 seconds to run instead of 5-6 minutes.

Making a note of it here for my own future reference. Follow this advice at your own peril – understand the ramifications of disabling barriers before turning them off.