Tag Archives: WordPress

Using a hosts file for easy management of dev, staging, and production WordPress sites

This recent article at Smashing Magazine discussed trends and challenges in deploying WordPress sites. The biggest issue cited by the article and the lengthy comments that follow is the issue of the database: because WordPress stores strings in the database that contain the site domain (including configuration options and asset paths), it’s hard to migrate content and config between dev, staging, and production environments. A bunch of possible solutions were offered up, including interconnectit/Search-Replace-DB, which I use fairly often and really like.

I was surprised, however, that no one talked about my preferred strategy, which is, in a way, the simplest: Dev, staging, and production should all have the same domain names. When you remove the need to change the domain, you make it much easier to deploy specific pieces of content, spin up new instances, etc.

Since all versions of a site have the same domain name – say, booneisthebomb.com – I use my local hosts file (/etc/hosts on *nix systems) to switch between instances. So I may have the following lines in /etc/hosts:

# Local development
# 127.0.0.1      booneisthebomb.com

# Staging site
# 123.456.789.0  booneisthebomb.com

With these two lines commented out, going to booneisthebomb.com in a browser will use DNS for the lookup, which is to say it’ll go to the production site. Uncommenting one of the lines allows me to work on the local or staging site.

The biggest pitfall of this technique is that now there is no obvious way to tell your environments apart (and you definitely don’t want to mistake your production site for a dev site). My solution for this is to drop this file into wp-content/mu-plugins/. Then, in my environment file (which contains env-specific config, such as database connection info), I put a line like this:

define( 'ENV_TYPE', 'staging' );

Now, when I load up the staging site, it shows booneisthebomb.com in the URL bar, and the following box appears at the bottom of every page:

env-type-flag

WordPress developers: Write to the filesystem the right way

Many WordPress plugins and themes need to write to the filesystem, to cache data, create a debug log, download libraries that for one reason or another aren’t distributed with the main package, etc. And many of these plugins do it wrong, by writing (or attempting to write) to their own plugin/theme directories. This is a bad idea for a couple of reasons:

  • If you use version control to deploy/manage a site, you probably have configured your repo to ignore the content of dynamic directories like wp-content/uploads. Obviously, you don’t want to ignore plugin and theme directories. When Git etc detects your newly created files, it wreaks all sorts of havoc with workflow and, depending on the content of the files and the carelessness of the deployment manager, poses the risk of losing user content or endangering sensitive data.
  • Some people have their file permissions set very conservatively, so that the webserver user doesn’t have write access to wp-content/plugins or wp-content/themes. So plugins that attempt to write to those directories often break altogether.

The good news is that every properly-configured WordPress installation will have at least one location where the webserver can write, and which is highly likely to be ignored by all version control setups: the upload directory wp-content/uploads. The situation is more complicated on Multisite, where each site has its own subdirectory of wp-content/blogs.dir. Happily, there’s an easy way to concatenate an upload path that’ll work across installations:

$uploads = wp_upload_dir();
$my_upload_dir = $uploads['basedir'] . '/yourplugindir';

WordPress has a very slick filesystem class that’ll help you if you really do need to write to a plugin or theme directory. But 99% of the time, you don’t. Please keep your stuff out of the codebase.

Anthologize automated tests now run using WP’s unit test suite

Anthologize, I haven’t forgotten about you! I have some very cool stuff in the works, but for now, a quick update on the progress of the campaign-funded work.

Back in 2011, Patrick Murray-John added some unit tests to Anthologize, covering a number of public methods in the TeiApi class. A number of the major refactoring jobs I’m currently undertaking will require additional test coverage, but they are (unlike TeiApi) dependent on WordPress being initalized. So I’ve migrated Anthologize’s tests to use the WP test suite. I’ve used the scaffold provided by the dope and phatte wp-cli (incidentally, I hope that their scaffold becomes the de facto standard for WP plugin tests).

This change means that, in addition to requiring PHPUnit, you’ll also need to have the WP test suite installed. You can install it manually, but I recommend using wp-cli to get the job done in just a command or two. In brief:


$ wp core init-tests /path/to/wp-tests --dbname=wp_test --dbuser=root --dbpass=asd
$ mysql -u'root' -p'asd' -e 'CREATE DATABASE IF NOT EXISTS wp_test'

To run the tests,


$ cd /path/to/wp/wp-content/plugins/anthologize
$ WP_TESTS_DIR=~/path/to/wp-tests phpunit

You can define WP_TESTS_DIR in your .bashrc file for quicker use in the future.

This post is brought to you by Anthologize campaign supporter Demokratie & Dialog. D&D, a Major Sponsor of Anthologize (woo hoo!), is using WordPress and BuddyPress in amazing ways both to study the way that government policy affects youth and to get youth themselves involved in the development of said policy. I had a chance to get to know the very excellent Andreas Karsten of D&D at BuddyCamp Vancouver last year, and we have big plans to start a BuddyPress jazz band. Many thanks for your support of WP, BP, and Anthologize!

Selectively deprecating WordPress plugins from Dashboard > Plugins

On large WordPress MS installations where site admins are allowed to manage their own plugins, the list of plugins tends to get crowded over time. Sometimes you introduce a plugin to the network and admins start using it, but some time down the road – a year or two later, even – you decide that you want to deprecate that plugin (maybe to replace it with another one, etc). However, migrating users of one plugin to another plugin is a logistical and technical tangle, and sometimes the best medium-term strategy is to allow existing users of the plugin to keep using it, but to prevent admins from activating it in the future.

Here’s how we’re doing it on the CUNY Academic Commons. In the gist below, $disabled_plugins are the plugins that we don’t want people to activate in the future. In most cases, however, we do want people to be able to deactivate the plugins, so by default, we don’t filter plugins if they’re active. However, we also have an array of $undeactivatable_plugins, which cannot be activated or deactivated.

Props to dev team member Dominic Giglio for writing part of this.

Safely delete spam comments across a large WP network

I’m currently working on a university WordPress network that’s been running for four or five years (an MU veteran!) and has almost 5000 blogs, most of which are defunct (because they’re from previous semesters). Akismet is activated across the network, so there’s not much of a public spam problem. However, even spam comments are stored in the database, and some of the blogs have tens of thousands of spam comments sitting in their tables. I’m going to implement a couple of tricks to keep this from happening in the future (a lightweight honeypot for non-logged-in users, tell Akismet to auto-delete spam comments on old posts). But for now, I’ve got to clean up this mess, because the very large comment and commentmeta tables are causing resource issues.

I wrote a simple script that gradually cycles through all the blogs on the network and deletes comments that have been marked as spam by Akismet. Here it is, with some comments afterward:

Notes:

  • The number of blogs is hardcoded (4980)
  • The ‘qw_delete_in_progress’ key is a throttle, ensuring that only one of these routines is running at a time. You might call this the poor man’s poor man’s cron.
  • I’ve limited it to 10 comments per pageload, but you could change that if you wanted
  • Put it in an mu-plugins file. When it’s finished running (check the ‘qw_delete_next_blog’ flag in the wp_sitemeta table – it’s done if it’s greater than the total number of blogs on the system), be sure to remove it, or at least comment out the register_shutdown_function line.

Use at your own risk – I’m posting here primarily for my own records :)

Anthologize 0.7

Anthologize 0.7 is here. Get it while the gettin’s good!

Version 0.7 includes a number of important, under-the-hood improvements. Some highlights:

  • The way Anthologize loads itself has been largely rewritten, which means that it fires up more reliably – and using fewer resources – than ever before.
  • Some validation issues with epub exportsr have been cleared up
  • In previous version of Anthologize, PDF exports sometimes failed because Anthologize could not copy inline images to the necessary temporary directory. This process has been rewritten so that our PDF library uses WordPress’s standard upload locations, avoiding permissions errors
  • A Spanish translation is now available
  • Full compatibility with PHP 5.4 and WordPress 3.5

In addition, a Credits page has been added to the Anthologize menu. This new page includes shout-outs to all those supporters of my fundraising campaign. If you donated (and opted not to remain anonymous), check out the Credits page to see your name in lights! And if I’ve made a spelling error, or linked to the wrong URL, please let me know.

This round of development was brought to you by Cyri Jones. Cyri is an educator and technologist doing amazing things with WordPress and BuddyPress in lovely British Columbia, including his ZEN Portfolios platform for student portfolios, and private social networks for a number of local school districts. I’ve had the good fortune to do some work for Cyri’s projects, and I think that the work he’s doing with these free software platforms points toward a very interesting model for putting social learning technologies in the hands of those who can use them. Cyri was also the brains (and brawn) behind the very first BuddyCamp, held last October in Vancouver. Rock on, Cyri!

Circle back to those who refer work to you

I get a lot of requests to do BuddyPress and WordPress dev work, and I can only take a fraction of that work myself. So I end up referring a lot of work to other developers. Sometimes I hear back from the client that one of the referrals worked out (or didn’t). Unfortunately, it’s very rare that I hear from the developer himself about it.

It’s unfortunate not because I need the gratification (although it’s nice to hear “thanks for the referrals” sometimes). It’s unfortunate because I want to be a good referer. I read every job inquiry carefully, and I try to make good matches between inquiries and what I know about the developers on my list. I look at how big the job will be, what kind of work it is (plugin dev, theme work, troubleshooting, etc), what field the client comes from (education, journalism, retail, etc), and other stuff like that, and match it up with the devs I think would be best for (and would most enjoy) that particular referral. When I never hear back from these friends, I don’t have the data I need. Stuff like: If you refused the job, why? Too busy? Not the kind of work you generally like to do? Did the referral make your lousy-client-sense tingle? If you took the job, how did it go? Was the client good to work with? Should I continue to send work like this?

So please: if you know that one of your client inquiries came from a friend’s referral, ping that friend at some point down the road. The more info you share, the better the referrals you’ll get from me.

As a side note, I’ve been chatting privately with David Bisset about coming up with systematic solutions to the kinds of problems I’ve described here (among others). But in the meantime, an occasional email will do the trick :)

Commons In A Box, ready to unbox

It’s been a long time coming, but it’s here: Commons In A Box. Today we’re releasing version 1.0-beta1, the first public release. For some background on Commons In A Box, here’s today’s press release, my Commons Dev Blog post explaining some of the features of Commons In A Box, and the 2011 press release announcing the project.

The primary goal of Commons In A Box, in my view, is to reduce the barrier of entry to setting up BuddyPress community sites. BuddyPress is an extremely powerful and flexible platform for developing social WordPress sites, but getting a BP site right takes knowledge (which plugins are worth installing, which ones work best together, etc) and elbow-grease (customizing your theme, keeping a complex system up to date). These practical requirements have made BuddyPress seem imposing to many users – including, and perhaps especially, the users that need free community software the most, such as educational institutions. Commons In A Box lowers these barriers in a serious way, by helping with plugin selection and installation, and by providing a beautiful and flexible default theme. My hope is that Commons In A Box will serve as a gateway for a swath of potential users into the world of BuddyPress, WordPress, and free software more generally.

The process of pitching, planning, and producing Commons In A Box has been interesting, frustrating, and rewarding. In the upcoming weeks and months, I may write more about this process, and what I’ll personally take away from it. In the meantime, I’ll say that I’m very pleased to be ending this first stage of development, and pushing it into the wild, since software – even imperfect software – is infinitely more valuable when it’s out there, being used, than when it’s mouldering on a developer’s machine. Shipping FTW!

Learn more about Commons In A Box.

Three talks in Vancouver

For those Bo(o)neheads who follow me to every event in VW vans, I’ll be giving three talks in Vancouver next month:

  1. BuddyPress: Beyond Facebook Clones, Oct 13, WordCamp Vancouver. I’ll highlight some uses of BP that are not straightforward social networks. (BTW, if you know of any really cool ones, please let me know in the comments!)
  2. Free Software and the University: The Story of the CUNY Academic Commons, Oct 14, BuddyCamp Vancouver. I’ll be using the story of the Commons as an excuse to rant about an allegory about the importance of free software in public schools.
  3. Getting Started with BuddyPress Plugins, Oct 14, BuddyCamp Vancouver. I’ll be giving an overview of what WordPress plugin developers need to know about getting their feet wet with BP plugins.

Using Git locally for a Subversion-based project (like BuddyPress)

In the past, I’ve written extensively about using Git with WordPress projects. I’ve focused primarily on Git as the primary development channel, with SVN (in this case, plugins.svn.wordpress.org) used for distribution only.

In contrast, I use Git for all my local development on the BuddyPress project. In this case, BP’s “official” history is in its SVN repo. My local Git repo is just a mirror. This setup means that you need a different kind of workflow, one that gives precedence to the Subversion repository. I’ll be using BuddyPress as an example below, but a similar workflow will work for any Subversion-based project where you want to do local development in Git.

(Side note: Mark Jaquith published a similar guide on how he uses Git to do WordPress core development. His process and mine are independently derived, but they are, of necessity, conceptually similar.)

Setup

  • Get Git.
  • Create a local directory for your BuddyPress installation. Use git-svn-clone to pull the SVN revision history into this directory.
    $ mkdir /path/to/wordpress/wp-content/plugins/buddypress
    $ git svn clone -T trunk -t tags -b branches http://buddypress.svn.wordpress.org /path/to/wordpress/wp-content/plugins/buddypress
    

    Git will crawl through the entire revision history of the BuddyPress project, which will take a while.

  • I generally have two active, ongoing branches in my local BP-Git repo, one corresponding to trunk and one corresponding to the current bugfix branch. I use master (the default Git branch) for trunk, since that’ll be the default setup after the clone. You’ll need to create the bugfix branch manually. I use the ’1.6.x’ naming convention for the 1.6 SVN branch, etc.
    $ cd /path/to/wordpress/wp-content/plugins/buddypress
    $ git checkout -b 1.6.x 1.6 # In other words, create a new branch, called 1.6.x, which tracks svn's 1.6 branch
    
  • If you’ll need to do development using BP’s bbPress 1.x implementation (“Group Forums”), you’ll need to manually download bbPress 1.1 into buddypress/bp-forums/bbpress/

Development

Day-to-day development goes something like this:

  • Make sure you’re on the right branch. Most day-to-day dev happens on trunk/master, but in some cases it’s necessary to work on the current bugfix branch. Once you’re on the right branch, make sure that you have no unstaged changes, and use git-svn-rebase to get the most recent changes from upstream:
    $ git checkout 1.6.x
    $ git svn rebase
    
  • Create a new topic branch for this bugfixing session. I generally name it after the ticket number.
    $ git checkout -b bp4453
    
  • Fix your bug or develop your feature. Commit small changesets as desired.
  • When you’re done with your development, you’ll be in one of the following situations.
    1. You’ve fixed the issue, and want to merge your commit history directly back into the public branch. This generally means that you fixed everything with a single changeset, or perhaps a small number of changesets that have good commit messages, etc. In that case, you can do a straight merge back to the public branch. Switch back, git-svn-rebase to make sure none of your collaborators have updated the SVN branch since you started working, and then merge.
      $ git checkout 1.6.x
      $ git svn rebase
      $ git merge bp4453
      
    2. You’ve fixed the issue, but you want to reduce a large number of changesets to a single commit. You have a few options here. You can use git rebase -i like Mark suggests. I generally do not do this, because rebasing on publicly shared SVN branches makes me nervous. Instead, in these cases I’ll use a squashed merge, which lays all of your changes on top of the destination branch, and leaves them uncommitted.
    3. $ git checkout 1.6.x
      $ git merge --squash bp4453
      $ git commit -m "This is the actual commit message I want to show up on BP's SVN"
      
    4. You want to share your changes with others, in the form of a patch, before committing to SVN. You’ll need to use the git diff utility, while doing a formatting trick to make sure that it’s compatible with the standard UNIX patch utility used in the SVN world.
      $ git diff --no-prefix 1.6.x...HEAD > ~/path/to/patches/4453.01.patch
      
  • Assuming you are ready to send some commits up to SVN:
    $ git svn dcommit
    
  • In some cases, you may have sent a commit to the bugfix branch that needs to be applied separately to the main dev branch (trunk). There’s a couple different ways you might handle this. I usually use git-cherry-pick:
    $ git checkout master
    $ git svn rebase
    $ git cherry-pick e1f2e3f # The hash of the Git commit on the bugfix branch
    $ git svn dcommit
    

Releasing

Releases follow a regular tag workflow:

$ git checkout 1.6.x
$ git svn tag 1.6.2

We do some weird stuff with BuddyPress (related to mirroring on wordpress.org/extend), which is outside the scope of Git – sadly I have to use svn for some of it :’(


Some of this is specific to BP, but most can be applied to any project where you want to use Git on a project that lives in SVN. Now git out there and git er done! and other ‘git’ puns.