Category Archives: edtech

Dude, Where’s My Blackboard Contract?

· 2 Replies

[UPDATE: 9-23-2011 9:54EDT] The original links to vendor searches on Open Book seem to be working again. I guess that means that the issue was a poorly-timed technical outage. In light of this, I take back my tentative speculations about Open Book actively suppressing results – I was wrong. Leaving this blog post up for historical reasons.

[UPDATE: 9-21-2011 1:46EDT] It looks like all vendor information is missing from Open Book at the moment. The contracts are still available by contract number (example). This may point toward an Open Book technical problem. Until a bit more is known, I think it’s reasonable to assume it’s an innocent accident. The general points still remain.

A few days ago I wrote a blog post about how CUNY and Blackboard have, in various ways, inspired my work in free software. In that post, I linked to a page that showed search results for CUNY and Blackboard from Open Book New York, a service provided by the NYS Comptroller’s office that lets citizens see how public institutions are spending tax money (a great idea, right?).

The blog post got many thousands of hits, and many hundreds of those users clicked on the link in question, which showed the amounts of CUNY’s current hosting contracts with Blackboard. This morning, one of my commenters, Brian, let me know that the link no longer worked. In fact, when you search Open Book for Blackboard, no contracts at all are shown for the entire state, while just a few days ago, a similar search turned up lots of results.

My decision to hotlink to the contract details in the original post, instead of spelling the dollar amounts in the text, was completely intentional. While I think that the high cost of Blackboard’s service is indeed an important symptom of a larger problem, I think that the dollar amounts have the potential to overshadow other considerations. So I linked, knowing that few readers would click through.

But now, because I don’t want that aspect of the original post to be lost, I’m going to bring to the foreground what I’d intended to leave in the background.

– The original link to the search
– Google’s cached copy
– Screenshot, 9-21-2011

If removing the results was intentional, ie if Open Book removed the results at the request of Blackboard or of CUNY (I consider the former more likely, given the evidence), it is obviously quite disappointing, and lends a certain irony to the “Open Book” moniker.

It’s here – BuddyPress 1.5!!

· 3 Replies

It’s finally here! After many, many months of bug squashing, refactoring, and general bloodsweatntears, BuddyPress 1.5 has been released!

This long development cycle has been frustrating in some ways and extremely rewarding in others. On balance, I’m quite proud of the work that’s been done, and quite pleased to have worked so closely and so well with John, Paul, and all the other contributors to BuddyPress. My sincere thanks to all the users and developers who have been supportive during this dev cycle.

Most importantly, BuddyPress 1.5 itself kicks ass. The bp-default theme has seen some serious improvements, some much-needed features have been added, and the codebase has been overhauled in terms of additional internal APIs, documentation, style, and so on. If you’ve done development with BuddyPress in the past, do yourself a favor and check out BP 1.5 – you are in for an extremely pleasant surprise.

Here’s to 1.6 and beyond!

I develop free software because of CUNY and Blackboard

· 52 Replies

For two reasons, Blackboard is the key to why I develop free software.

The first reason is historical. I first got into free software development because of my work with the CUNY Academic Commons project. As spearheaded by Matt Gold, George Otte and others, the Commons is intended to create a space, using free software like WordPress and MediaWiki for members of the huge community of the City University of New York to discover each other and work together. The project is not pitched as a Blackboard alternative, for a number of reasons (primary among which is that the Commons’s Terms of Service prohibit undergraduate courses from being held on the site). Still, the Commons was conceived, at least in part, out of frustration about the near lack of collaborative tools and spaces in CUNY. And more than anything else, Blackboard (by which I mean Blackboard Learn, the proprietary learning management software that has been CUNY’s official courseware for quite a few years) is the embodiment of what can be so frustrating about academic technology at CUNY: central management, inflexibility, clunkiness, anti-openness. In this way, Blackboard begat the CUNY Academic Commons, and the CUNY Academic Commons begat Boone the developer.

There is another reason why Blackboard is integral to my free software development. It is ideological.

Short version: I love CUNY and I love public education. Blackboard is a parasite on both. Writing free software is the best way I know to disrupt the awful relationship between companies like Blackboard and vulnerable populations like CUNY undergraduates.

Here’s the longer version. I’ve been affiliated with CUNY in a number of capacities over the last decade: PhD student, adjunct lecturer, graduate fellow, full-time instructional technologist, external contractor. I’ve seen many parts of CUNY from many different points of view. Like so many others who have philandered their way through CUNY’s incestuous HR departments, my experience has rendered a decidedly love/hate attitude toward the institution. You can get a taste of the what CUNY hate looks like by glancing at something like @CUNYfail. The love runs deeper. Those fortunate enough to have “gotten around” at CUNY can attest to the richness of its varied campus cultures. In every office and every department on every campus, you’ll meet people who are innovating and striving to get their work done, in spite of a bureaucracy that sometimes feels designed to thwart.

And the students. CUNY is the City University of New York, the City University. It belongs to New York, and its history is tied up with the ideals of free education for New York’s residents. While the last few decades have seen the institution (as a whole, as well as a collection of campuses) evolve away from these ideals in various official and unofficial ways, it’s impossible to step into a CUNY classroom without getting a sense that CUNY still serves as a steward for New York’s future. CUNY is too huge and its population too varied to make general statements about the student body, but I’ll say anecdotally that, of all the universities I’ve been associated with, none even approach the level of racial, economic, and academic diversity that you find on a single campus, to say nothing of the system as a whole. CUNY is (to use a lame but apt cliché) a cross-section of New York: her first-generation Americans, her first-generation college students, her rich and her poor, her advantaged and her vulnerable. (See also Jim Groom’s I Bleed CUNY, which makes a similar point with a lot less abandon.)

Public education is a public trust, maybe the most important equalizer a state can provide for its citizens. CUNY, with the population of New York City as its public, could demonstrate the full potential of public education in a more complete and visible way than perhaps any other public university. It’s for this reason that it breaks my heart and boils my blood to see CUNY money – which is to say, student tuition and fees – poured into a piece of software like Blackboard.

In virtue of their age, undergraduates are inherently a vulnerable population, and CUNY undergraduates – reflecting as they do the full demographic spectrum of New York City itself – are doubly vulnerable. Many CUNY undergraduates go to CUNY because if they didn’t, they wouldn’t go to college at all. This imposes certain moral strictures on those responsible for managing and spending the money paid by CUNY students in tuition and fees. Wasting CUNY money is a far worse crime than wasting, say, shareholder money in a private company. Shareholders have freedom; if they don’t like your management, they vote with their feet/wallets/brokers. CUNY students, by and large, do not have the same freedom; it’s safe to say that, for most CUNY students most students, big-ticket NYU and Ivy Columbia are not reasonable alternatives. CUNY students are, in this sense, captive, which means that their hard-earned tuition money is captive as well. Thus it is a very bad thing to spend that money on things that aren’t worth it.

And Blackboard is not worth it. Vats of digital ink have been spilled expounding Blackboard’s turdiness, and this is no place to rehash all the arguments in depth. A short list, off the top of my head:

  • The software is expensive [EDIT 9-21-2011: See this post for more details on cost]
  • It’s extremely unpleasant to use.
  • It forces, and reinforces, an entirely teacher-centric pedagogical model.
  • It attempts to do the work of dozens of applications, and as a result does all of them poorly.
  • Blackboard data is stored in proprietary formats, with no easy export features built in, which creates a sort of Hotel California of educational materials
  • The very concept of a “learning management system” may itself be wrongheaded.
  • As recently reported, the software may be insecure, a fact that the company may have willingly ignored.
  • Blackboard’s business practices are monopolistic, litigious, and borgish

In short, Blackboard sucks. Blackboard supporters might claim that some, or even most, of the criticisms leveled above are false, or that they apply equally to other web software. Maybe. And I certainly don’t mean to downplay the difficulty of creating or assembling a suite of software that does well what Blackboard does poorly. But the argument against spending student money on something like Blackboard goes beyond a simple tally of weaknesses and strengths. As Jim Groom and others have argued for years, shelling out for Blackboard means sending money to a big company with no vested interest in the purposes of the institution, which in the case of CUNY is nothing less than the stewardship of New York City’s future, while the alternative is to divert money away from software licenses and into people who will actually support an environment of learning on our campuses. Frankly, even if Blackboard were a perfect piece of software, and even if its licensing and hosting fees were half of what it costs to hire full-time instructional technologists, programmers, and the like to support local instances of free software; even if these things were true, Blackboard would still be the wrong choice, because it perverts the goals of the university by putting tools and corporations before people. The fact that Blackboard is so expensive and so shitty just makes the case against it that much stronger.

As long as our IT departments are dominated by Microsoft-trained technicians and corporate-owned CIOs, perhaps the best way to advance the cause – the cause of justice in the way that student money is spent – is to create viable alternatives to Blackboard and its ilk, alternatives that are free (as in speech) and cheap (as in beer). This, more than anything else, is why I develop free software, the idea that I might play a role in creating the viable alternatives. In the end, it’s not just about Blackboard, of course. The case of Blackboard and CUNY is a particularly problematic example of a broader phenomenon, where vulnerable populations are controlled through proprietary software. Examples abound: Facebook, Apple, Google. (See also my Project Reclaim.) The case of Blackboard and its contracts with public institutions like CUNY is just one instance of these exploitative relationships, but it’s the instance that hits home the most for me, because CUNY is such a part of me, and because the exploitation is, in this case, so severe and so terrible.

On average, I spend about half of my working week doing unpaid work for the free software community. Every once in a while, I get discouraged: by unreasonable feedback, by systematic inertia, by community dramas, by my own limitations as a developer, and so on. In those moments, I think about CUNY, and I think about Blackboard, and I feel the fire burn again. For that, I say to CUNY (which I love) and Blackboard (which I hate): Thanks for making me into a free software developer.

The GPL is for users

· 9 Replies

The General Public License (aka the GPL) is for users. This observation seems so obvious that it needn’t be stated. But for those who develop software licensed under the GPL (like WordPress and most related projects), it’s a fact that should be revisited every now and again, because it has all sorts of ramifications for the work we do.

Users versus developers

What do I mean when I say that the GPL is “about users”? Who are “users”? We might draw a parallel between software and books. Books have readers (hopefully!), and they have authors. Authors read too; proofing is a kind of reading, of course, and one might argue moreover that reading is an inextricable part of writing. Yet when we talk about a book’s “readers” we generally mean to discount its author. ‘Readers’ in this sense is a gloss for ‘just readers’, that is, those readers whose relationship to the book is limited to reading. The situation with software is more complex, but roughly the same distinction can be made between users and developers. ‘Developers’ refers broadly to those people involved in the conceptualization and implementation (and also often the use) of a piece of software, while ‘users’ refers to those who just use it.

My reading of the GPL is that it’s heavily focused on users. (References to the GPL throughout are to GPL 3.0. You can find older versions of the licence, such as version 2 that is shipped with WordPress, on GNU’s website.) Take the opening line from the second paragraph of the Preamble:

The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program–to make sure it remains free software for all its users.

Here as elsewhere in the text of the GPL, no real distinction is made between “you” as it refers to developers and “you” as it refers to users. Closer analysis makes it pretty clear, though. Take, for example, the freedoms that are purported to be taken away by proprietary licenses: the freedom to “share and change” software. Developers – or, to be more specific, license holders, who are generally either the developers themselves or, in the case of work for hire, the people who paid for the software to be developed – generally do not restrict their own rights to share and change the software that they create. Instead, restrictions are imposed on others, the (“just”) users.

Similar reasoning applies to the core freedoms that are outlined in the Free Software Definition, a sort of unofficial sister document of the GPL, also maintained by the Free Software Foundation. The four freedoms:

  • The freedom to run the program, for any purpose (freedom 0).
  • The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
  • The freedom to redistribute copies so you can help your neighbor (freedom 2).
  • The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.

On the face of it, freedoms 1 and possibly 3 are focused on developers, in the sense of “those who are able to write code”. But, with respect to a piece of software that they did not write and whose license they do not control, coders are just regular users (in the same way that Vonnegut may have been a “reader” of Twain). All four freedoms, indeed, are user-centric. The license holder, almost by definition, doesn’t need permission to use the code (0); the developer doesn’t need to study the code to know how it works (1); owners can redistribute at will (2); owners can modify and redistribute at will (3). It’s only in the context of users – those who did not write the software – that these freedoms need protection in the form of free software licenses like the GPL.

The GPL does make a few explicit provisions for the developer/license holder:

For the developers’ and authors’ protection, the GPL clearly explains that there is no warranty for this free software. For both users’ and authors’ sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions.

The second provision is a sort of legal convenience; the first intends to ease what may otherwise be a prohibitive consequence of the core freedoms guaranteed by the rest of the GPL. Both are important and valuable. But it seems fair to say that they are secondary to the user-focused parts of the document, at the very least because they are motivated by other parts of the document, while user freedom needs independent justification.

There’s no question that the people who bear the brunt of implementing and upholding the GPL are software developers. In that sense, the GPL is very much “for” them. But, in a broader sense, that’s a bit like saying that school is “for” the teachers because the teachers play a key role in education. Schools are for children; they provide the motivation and justification for the whole enterprise. Similarly, the GPL is for users; if everyone wrote their own software, and there were no “just users”, the GPL (or any free software licenses, or any licenses at all) would be unnecessary.

Sacrifice

If I buy a pizza, I trade ownership of money for ownership of pizza. Once I have the pie, I can do pretty much whatever I want with it. I can eat the whole thing myself, I can share with a friend or two, I can throw it on the sidewalk. I can save the pizza in hopes that prices rise so that I can make a quick buck in a resale, I can retail off the individual slices, or I can give the whole thing away. I can’t use the pizza to solve world hunger (not because I’m not allowed, but because it’s not possible); I can’t use the pizza as a deadly weapon (not because it’s impossible, but because I’m not allowed). In short, ownership bestows certain rights. Not all rights – I don’t have the right to murder with the pizza, or to do impossible things with it – but many, even most of them.

The situation is more complex with intangible goods; especially those, like software, which can be reproduced without cost or loss. Copyright law in the United States (so far as I understand it; IANAL etc), in accordance with the Berne Convention, grants rights over intellectual and creative works to the authors automatically, at the time of creation. Thus, if I write a piece of software (from scratch – set aside issues of derivative work for a moment), I am granted extensive rights over the use and reuse of that piece of software, automatically, in virtue of being the author. That includes copyright – literally, the rights related to the copying and distribution of the software. In short, the default situation, for better or for worse, is for the developer – and only the developer – to possess the rights and freedoms enumerated by the Free Software Definition. By default, nothing is protected for the users.

Free software licenses exist in order to counteract this default scenario. But keep in mind what that means: When a developer releases a work under a license like the GPL, certain freedoms and rights are granted to users, which necessarily restricts the freedoms of the developer. The GPL admits as much:

To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others.

“Responsibilities” is a nice way of putting what is essentially the stripping of certain rights (in the same way that, once you become a parent and thus responsible for your child’s well-being, you no longer have the right to go on a week-long bender). Once the software is released under a GPL, the original author has lost the right of exclusive distribution of the original software. Subsequent developers, those who modify and redistribute the software, are similarly restricted.

It’s a trade-off. Users get certain rights (viewing source code, copying, modifying, redistributing) because the developers have given up the default right of exclusivity. Examined in itself (without reference to subsidiary benefits for the moment), the trade-off is clearly made for the benefit of the users, and involves sacrifice on behalf of the developer, sacrifice which is usually quantified in monetary terms (Bill Gates didn’t get rich by writing open source software), but could also be associated with pride in being the sole author, etc. There are, in addition to this, secondary sacrifices involved in free software development (loss of identification with the software because of modifications or forking, less guaranteed income than in a proprietary development shop, increased support requests that come from wider use of a free-as-in-beer product [though the GPL explictly says that you can charge what you want, and that no warranty is implied]). To some extent, these secondary sacrifices can be mitigated by the realities of the market, and are anyway subject to the particulars of the scenario in which you find yourself. But the core sacrifice – giving up exclusivity over distribution – cannot be separated from free software licenses.

Software licenses are political documents

Developers have all sorts of reasons for releasing software under free software licenses like the GPL. A few, off the top of my head:

  • You want to modify and redistribute existing software that is GPLed
  • You want to distribute somewhere that requires GPL-compatibility, like the wordpress.org plugin repository
  • You believe that forkability and other GPLy goodness makes for a better product
  • You want to develop for a platform, or contribute to a project, that requires GPL compatibility

I classify these reasons as prudential, in the sense that they are focused on the material benefits (money, fame, better software) that you believe will come from developing under the GPL. All of these reasons are great and important, and many of them have motivated my own work with GPL-licensed software. Taken together or even individually, it’s easy to imagine that these (and other) benefits would outweigh the sacrifice involved in giving up exclusive distribution rights over your work.

There’s another kind of justification for releasing under the GPL: you endorse, and want to advance, the political and moral ends that motived the creation of the GPL. The GPL assumes that it’s a good thing for users to have maximal freedom over their software:

If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.

The assumption here is that “greatest possible use to the public”, and by the extension the good of the public, is something to be actively pursued – a moral claim par excellence.

And, among free software licenses, the GPL is perhaps the most explicit about the ways in which user freedoms (and thus the greatest good of the public) should be guaranteed and propagated. The “viral” nature of the GPL constitutes a kind of normative statement about the value of user rights over developer rights, which goes beyond other free software licenses that do not share its viral nature. The difference might be summed up like this. Alice and Bob are coders, and Carol is a potential user of the software. If Alice writes a piece of software and licenses it under a free software license like those in the BSD tradition, Bob can fork the software, make a few changes, and sell it to Carol under any terms he’d like – he can compile a binary executable for distribution, without making the source code available, converting his fork into closed-source, proprietary software. If Alice licenses the software under the GPL, on the other hand, Bob can still modify and sell to Carol, but he may not change the terms of the original license – in particular, the source code must be made available for further modification and distribution.

The normative aspect of the difference is in the value that each license scheme ascribes to the rights and freedoms of various individuals involved. BSD is more permissive with respect to Bob; GPL limits his ability to license the derivitive work as he pleases. GPL is more focused on Carol, and protecting her – and other “just users” like her – at the cost of some of Bob’s freedoms. (The GPL is for users.) One might express the difference in political terms thus: the GPL is more liberal, and less libertarian, than the BSD. Users, who are on the weak end of the power spectrum when it comes to software, are protected under the GPL, in the same way that society’s underprivileged and weak are often the focus of political liberalism. On this picture, licenses, like laws more generally, are designed in part to create the restrictions necessary to protect the positive freedoms of a vulnerable population.

For developers who agree independently with the normative principles underlying the GPL, its moral benefits can outweigh the sacrifices it entails. Such a justification is the starting point for Stallman and the Free Software Foundation (see, for example, the FSF’s about page). You may, of course, foreground other aspects of free/open-source software when justifying your licensing. I’ve listed some justifications above, and entire movements have sprouted to focus on prudential, rather than moral, justifications for open source development.

But – and here’s the rub – licensing your work under the GPL constitutes an endorsement of its moral justifications, even if it’s not (from a cognitive point of view) what motivated you personally to apply the license. If you choose a free software license for prudential reasons, you are not justified in complaining when your project is forked. If you choose the GPL for prudential reasons, you can’t altogether disavow the inherently altruistic underpinnings reflected in the license’s preamble. Put another way: Among other things, software licenses are political documents, and it’s incumbent upon developers to understand them before adopting them.

It’s important for developers to think carefully about this before diving into a license. My own take is that the original motivation for free software – that user control over the software they use is fundamental to their autonomy – becomes truer every day, as more and more of our agency is mediated through software. For that reason, licenses like the GPL are ethically important, at least if your worldview depends (as mine does) on respecting the agency of other human beings.

This post was prompted by a recent post by Ipstenu. Much of my thinking on the matter is clarified and inspired by the first few chapters of Decoding Liberation: The Promise of Free and Open Source Software, a book about free software written by philosophers/computer scientists Samir Chopra and Scott Dexter. You can (and should) buy the book here.

Anthologize 0.6

· 1 Reply

I just tagged version 0.6-alpha of Anthologize in the wordpress.org plugin repo. This new version has a bunch of bug fixes, improvements to stability and consistency of output, and a few new feature goodies. Read more about the release here.

On a related note, Anthologize development has recently moved 100% over to our Github repo. We’d previously used Trac for ticketing and Github for code management, but now we’re doing everything on Github. If you’re a user, please feel free to open new issues. If you’re a developer, send those pull requests!

Building a baby photo site with WordPress

· 17 Replies

My wife and I just had our first baby, which is the occassion for much nachas and, by extension, picture sharing. Facebook is, for better or for worse (emphasis on the latter), the de facto place for such sharing to happen. For a number of reasons – a desire to be somewhat selective about who sees my family pictures, my Project Reclaim sensibilities, the fact that I don’t have a Facebook account and generally think that Facebook is an evil company – I don’t want to use FB for this purpose. As in the case of my Twitpic-like photoblog, I figured I could use WordPress to set something up that was nearly as seamless as Facebook, or Google+, or Flickr, or whatever.

The criteria

There were a few things I wanted out of the baby site.

  1. Easy (or zero) login for users
  2. Control over who has access
  3. Optional email notification for new content
  4. Easy, javascripty gallery browsing

When I started, I was pretty sure that I’d be able to get all of these things pretty easily, using existing WordPress plugins. I was both right and wrong about this: plugins exist for all of these purposes, but none of them were very easy to implement. As a result, I ended up building several pieces from scratch. I’ll go through each of the criteria, talk a bit more about what I was looking for, and then say something about how it was achieved. By doing this, and sharing the code (spoiler alert: https://github.com/boonebgorges/Hard-G/tree/master/wp-content), I’m hoping that I can help others with similar sensibilities to get started on their own sites.

Non-sucky registration and login

I love WordPress, and I understand the important reasoning behind the decisions that led to the design, but WP’s user registration system sucks. I didn’t want just anyone to be able to create and activate an account. I didn’t want users to have to click an activation link. I didn’t want users to have randomly generated passwords that would need changing later on. And I wanted users to have the option of logging in a non-WP way.

Several of these problems could be solved by using Facebook logins. I’m not willing to give my photos over to the horrific FB leviathan, but I’m happy to piggyback on their login APIs if it will save my family and friends a few headaches. I wanted my users to have the option of clicking a single button that would give my site the ability to provision them based on their persistent Facebook login.

I started by looking at some popular Facebook Connect plugins from the wordpress.org plugin repository. I didn’t really like them. Most were linked to the Comments section of blog posts, while I wanted to use the logins for overall site access. Most were dependent on Javascript for logins, while I wanted to handle logins on the server side. Most used an outdated version of FB’s API (or at least of the PHP API classes that FB offers). And, to be blunt, most were too much of a mess, having been retrofitted many times over, and as a result next to impossible to extend. I tried modifying one or two of the more popular FB-WP plugins to do what I wanted, but I ended up writing so much garbage spaghetti code that I decided to cut my losses and start from scratch.

So I boned up a bit on the FB API, and wrote a small plugin that I call Wally Login. Together with the registration page template from my custom theme (a child of TwentyEleven), it does a couple of key things.

Your choice

Your choice

  • Rudimentary access control · If a non-logged-in user tries to visit any page on the site (other than the registration page), he is redirected to the Register page.
  • FB login integration · If a user clicks the “Log me in using Facebook” link, they’re directed to the FB authorization page for my website (which is registered as a Facebook app). There, they’re asked to approve the app – a one-time process – and are then returned to my site. Based on the display name, email address, etc that I get from FB, I create a WP user corresponding to the FB account. On future visits, approved users who are logged into Facebook will be automatically logged into my WP site whenever they visit it (an important point, because FB cookies are persistent over browser sessions, while WP logins, by default, are not). As a result, in the best-case scenario, a user will authorize their FB account with my site one time, and will never again have to think about authorization on Wally’s page.
  • A customized WP registration process · If users opt not to go the FB route, they can create a WP account directly on the site. I wanted to avoid sending users to an unthemed wp-login.php or wp-signup.php page, so I cribbed a few lines of code from BuddyPress and made my own registration and login dialogs. Wally’s site is part of a larger WP network, but I wanted to bypass WPMS’s built-in registration stuff (which requires users to activate their accounts, and is thus generally too hard for newbies to grok). My custom registration therefore creates the user directly (with wp_insert_user()), using a password that he provides, and skips the activation email. (By bypassing account activation, I’m removing an important spam prevention tool. More on that in the next section.)
  • Customized email notifications · Because I’m not using the built-in registration process, I needed to write my own email notifications for account applications and approvals.

If you decide to use my code, keep in mind that it’s not particulary beautiful. I wrote it for my own use, which means that it will take a bit of elbow grease to get it to work on your own site. In particular, if I were writing something for more general distribution, I would not be so reliant on a theme template as I am here. But if you’re looking to create a site like mine, this code is a great place to start – especially the FB integration stuff, which has made the registration and login process about as smooth as it can get.

Access control

The final important thing that the Wally Login plugin does is to provide me (the site admin) with control over who has access to the site. There are a couple ways I could have approached this issue. One is to whitelist users ahead of time. The problem with this is that I’m bound to forget some names, get email addresses wrong, and run into other problems that stem from my unfortunate lack of omniscience. Another strategy is the invitation code. When unique to the individual, this method suffers from the same drawbacks of the whitelist; when non-unique (ie when everyone uses the same invitation code) it takes away much of the security, as the code can be passed around quite easily; either way, invitation codes are clunky, easily misplaced, and all too often mistyped.

Thanks for applying

Thanks for applying

As a result, I ended up going with a third option: an application and approval process. Here’s the idea, conceptually. Anyone who wants can create an account on my site (either through Facebook or natively; see the previous section). However, the account does not actually allow access to the site unless the account is also approved by the administrators. Thus, after the initial application, two emails are sent: one to the applicant saying “thanks for applying, please be patient”, and one to me saying “there’s a new applicant, please approve them”. Then I go to my approval interface and click the Approve button (if I want), which marks the user as approved in my database and sends them an email saying “You’re in!”

Here’s a brief description of how it works technically. All applicants have a WP account created for them. Every new account is marked, at the time of creation, with user_status = ‘2’, and I make sure that no page other than Register can be viewed by an account with user_status = ‘2’. In this way, I am turning the idea of activation around a bit – natively, WP makes the user do the activation, but in my case I do it. The admin tool I use to activate users is my Unconfirmed plugin, designed for a slightly different purpose but quite at home here. (For technical reasons, Unconfirmed needs users to have activation keys; thus Wally Login also generates some dummy keys during the user creation process so that Unconfirmed will work right.) Unconfirmed, in turn, does the work of flipping user_status to 0 upon approval.

Taken together, Wally Login and Unconfirmed (with custom WP registration, FB integration, user approval by admin, etc) has given me a comfortable level of access control, without making the process unduly difficult for my users.

Email notification for new content

One of the biggest drawbacks of creating a standalone picture site instead of using an all-purpose social network (in practice, this means Facebook) is that the standalone site is likely to be forgotten. FB collects all of your network’s activity into a single stream; it’s highly unlikely, on the other hand, that Wally’s site will become part of anyone’s daily routine, so that they stop by to check for new content. For that reason, good email notification of new content is essential to making the site work.

Dead simple email subscription

Dead simple email subscription

I first tried using the popular Subscribe2 to handle these notifications. But I ran into a bunch of problems. For one thing, I didn’t like that S2’s subscription management happened in the Dashboard – I want to keep my users on the front end. S2’s category-based subscription is too complicated for my site, where people are either going to want to subscribe to all posts or to none at all. And the widget that comes with S2, for display on the front-end of the site, is pretty much atrocious. (Sorry. The rest of the plugin is nice. But that widget sucks.) At first I tried solving these problems just by building my own widget for S2, one that would tell the user whether he was currently subscribed, and show an Unsubscribe/Subscribe button, as appropriate. But, given the structure of S2’s data (which is somewhat arcane, and in any case far too complicated for my purposes), it ended up being a lot harder than it should have been.

So – wait for it – I wrote something from scratch. It is dead simple. Two parts: (1) a widget, which does exactly what I describe in the foregoing paragraph; and (2) hooks into publish_post to send an email to all subscribed users (along with some gentle checks to make sure dupes are not sent). This plugin has no admin UI and no options, because I don’t need any of those things.

Pretty galleries

Since the main point of the site would be to look at lots of pictures, it was quite important to have an easy, pretty way to do so. By “easy” I mean, primarily, navigable by keyboard; by “pretty”, I mean, primarily, bigger than the content area of a typical blog post. Less important, but still desirable, was the admin interface: I wanted it to be easy to upload lots of pictures at once, to add captions and other metadata if necessary, and to turn it all into a gallery that would look good on the front end.

Pretty, easy

Pretty, easy

You know the drill: I tried a couple of the more popular free plugins, but all of them were annoying in one way or another, and each one was way overengineered for my meager needs. I was especially disappointed by the back-end admin for the popular gallery plugins, which I found lugubrious, unintuitive, and impossible to extend. After some consideration, I decided that I actually preferred WP’s native Add Media interface for uploading photos and adding metadata, and that I was perfectly happy with the way that WP’s gallery shortcode displayed content on the front end, at least when viewing thumbnails.

So the only thing I really needed was to implement the javascript that would allow for keyboard navigation and lightboxing of gallery photos. Thanks in part to his extremely uncreative and literal plugin naming schema, I found Viper007Bond’s jQuery Lightbox For Native Galleries plugin. It does almost exactly what I want, right out of the box.

I did make a few minor mods, though. First, the plugin is a bit greedy in the way that it filters the output of get_attachment_link(), which was either breaking things (as in the case of comment_post_redirect on attachment posts) or making it hard to display links to the attachment page instead of the raw attachment file. The former problem I solved with a filter; for the latter problem, I was a bit lazy, so I modded the plugin itself in addition to adding an explicit ‘lightbox’ class to attachment links. This combination of hacks makes it work perfectly for my purposes.

Odds and ends

A little bonus

A little bonus

With my absolute requirements met, I was able to add a few other goodies to the site. My theme is a child of Twenty Eleven, which I’m pretty much using as-is. But I’ve added a few fun bits. First, on each attachment page, I added Download links, so that users could download images of various resolutions for printing or editing. I messed with the WP Admin Bar so that users coming from Facebook wouldn’t see Log Out and some other inappropriate links. And under each thumbnail in Gallery view, I’ve added Download/Comments links, so that users could bypass the jQuery lightbox and go straight to the attachment permalink if they wanted.

It took some work, but I think I’ve ended up with a site that is nice to use and easy to maintain, without resorting to the extreme discomfort associated with Facebook. Hooray!

BuddyPress and the YOURLS: WordPress to Twitter plugin

· 4 Replies

A few weeks ago, I wrote about reclaiming short URLs using YOURLS. That post raised some interest among the CUNY Academic Commons team in having a URL shortener just for the Commons, with full integration into BuddyPress. So I emailed Ozh Richard, author of YOURLS, about the possibility of adding BuddyPress support to his official YOURLS WordPress plugin, YOURLS: WordPress to Twitter. He graciously accepted my offer to do the leg work.

Today I’m releasing the fruits of this collaboration: version 1.5 of YOURLS: WordPress to Twitter. YWTT 1.5 automatically detects when you’re running BuddyPress, and adds the following BP-specific features:

  • Member and Group URLs – Generate short URLs for member profiles and for group home pages.
  • A “pretty URL” setting – Instead of generating random URLs (like http://blo.so/54), you can make member and/or group members ‘pretty’ (like http://blo.so/username or http://blo.so/groupname).
  • User customizability – Optionally, you can add new options under groups’ Admin > Group Settings and members’ Settings > Short URL allowing users to request a custom short URL of their choice. (This feature requires that you set YOURLS_UNIQUE_URLS to false in your YOURLS configuration file.)

Down the road, I plan to flesh out BP-YOURLS functionality, with optional short URLs for forum topics, activity items, and so on.

I’ve also slipped full localization support into version 1.5. Send me your mo/po translation files if you’d like them to be distributed with the plugin.

Download YOURLS: WordPress to Twitter 1.5, with BP support.

Shorten your own dang URLs

In my last Project Reclaim post, I talked about using WordPress as a Twitpic-like personal mobile photo service. When the ultimate goal of the photoblog is to send a tweet, it’s almost always necessary to use a URL shortener. But trusting your URL shortening to a free service is a dangerous move. If that service goes out of business, or if they decide to take down the database for some reason or other, the links in those tweets will break. (This problem is delightfully called “linkrot”.)

So, while URL shorteners are sometimes necessary, they’re also an obvious instance for reclaiming your data. Moving to your own URL shortener means that you control the domain, you control the content, you can back up the database however you’d like, etc.

I went with a piece of software called YOURLS. It’s written by Ozh Richard, a WordPress developer, and there’s a slick WP plugin that makes it a great choice for use with my WP photoblog. Here’s a short walkthrough of how I set it up.

Setup

  • Get a domain. Something short is nice, obviously. I just started typing two- and three-letter domains into my domain registrar’s search box (I use Dynadot), which showed me the top-level domains available, until I found one that was easy to look at and remember (http://blo.so). Make sure you do whatever setup your registrar requires to get the domain working – probably as simple as setting the nameservers to your host’s NS addresses.
  • Install YOURLS. The instructions provided at the YOURLS site are pretty concise, but here’s the gist: upload the software to the server, create a new database, copy the sample configuration file to user/config.php, and fill in the configuration file with the proper database info, etc. You can get more YOURLS config info here.
  • Configure an Apache virtual host, if necessary. If your hosting provider doesn’t have cPanel or some other tool that easily lets you point your new short domain to a subdirectory, you’ll need to do it manually by creating a new Apache virtual host file and activating that site. This website has a pretty good explanation. But essentially, just copy the default configuration in sites-available (likely at /etc/apache2/sites-available) and change the info in the VirtualHost section.
  • Install the WordPress plugin. The YOURLS: WordPress to Twitter plugin is easy to install and set up. Once the plugin’s installed, go to Dashboard > Settings > YOURLS and fill in the necessary information. Setting up the Twitter bit is a pain, thanks to Twitter’s requirement that you get a developer’s key, but it’s easy to do. Just follow the on-screen instructions.

At this point, everything should be set up. Send a test post or two to try it out.

Bonus! Use me with Tweetdeck

YOURLS has a REST API that can be used with a bunch of applications. For instance, I’ve configured my TweetDeck installation to do its URL auto-shortening with blo.so. Go to Settings > Services and choose Other from the URL shortener dropdown. Your endpoint will look something like http://blo.so/yourls-api.php?signature=XXXXXXX&action=shorturl&url=%@&format=simple. You’ll have to replace blo.so with your own URL, of course, and the XXXXXXX signature with a custom YOURLS signature password. You can get it from the YOURLS admin screen (http://example.com/admin/tools.php > Secure passwordless API call)

Here’s the great thing. There’s no reason why a couple people can’t share a single YOURLS installation. In fact, I’ll put my money where my mouth is, and start my own URL shortening co-op. I’ll give usernames/passwords to blo.so to the first couple friends who want in. Send an email to boonebgorges at gmail if (1) you are my friend, (2) you want in on blo.so, and (3) you promise to actually use it and break the bit.ly/tinyurl habit.

BuddyPress Docs 1.1: Doc History

· Leave a reply
BuddyPress Docs History

BuddyPress Docs History

I’ve just released version 1.1 of BuddyPress Docs, my collaborative editing software for BuddyPress.

The big new feature in version 1.1 is the History tab. After upgrading, you’ll notice that what used to be a single Edit button has been reorganized into three tabs: Read, Edit, and History. History allows you to brows the entire revision history of a document, to compare the differences between two revisions side by side, to view a single revision, or to restore to any point in the document’s history. Access to the History tab can be limited in the same way that access to the Edit tab can be, on a doc-by-doc basis.

This new feature will, I hope, bring some of the best qualities of wikis to BuddyPress Docs, and make Docs an even better way to collaborate.

Download BuddyPress Docs from the wordpress.org plugin repo or follow development at Github.

Revisiting Git, Github, and the wordpress.org plugin repository

· 39 Replies

Some months ago, I wrote about using Git and Github with the wordpress.org plugin repository. Since that time, I’ve been refining my plugin development workflow. I now do all of my development with Git, using git-svn tools to do all svn management.

Preamble: Git as primary vs secondary

Before talking more about my workflow, I should make a sort of conceptual distinction. When I develop most of my plugins, I am working alone. In these cases, Git is primary. It’s where the meaningful version control happens, while svn.wp-plugins.org is just a gateway for wordpress.org/extend, the distribution channel. I am not using the wordpress.org svn repository for version control or code sharing in any interesting way. Likewise with a few team projects that I’m involved in, notably Anthologize. All of our code-sharing and true version control happens via Git and Github (here’s our repo), with the wordpress.org svn repo used only as a distribution mechanism.

In contrast, I also use Git to develop BuddyPress, but the strategy is quite different. In the case of BuddyPress (as in the case of WordPress), the svn repo is the officially sanctioned version control system for the project. Git, in this case, is a secondary, local versioning system – essentially, my development sandbox. This setup places additional restrictions on how the git-svn link is managed, raising issues such as juggling svn branches, tagging version off of an svn branch, exporting patches in an svn-compatible format, and so on.

In this post, I’ll be focusing on the first kind of development setup, in which the wordpress.org svn repo is serving merely as a distribution channel. It’s a bit simpler to start there. Moreover, I’ve been chatting with Mark Jaquith about Git for WP development, and I know from those chats that he’s planning on writing up a description of the second kind of workflow (which characterizes the WordPress core work that he’s more concerned with). So I’ll leave the sophisticated stuff to him.

Part One: Getting your repos set up

You’ll need to have Git installed on your computer, with git-svn compiled. Here’s a guide to compiling it yourself on OS X; it’s also available through a number of repos on various OSes.

Create a directory for the plugin in the plugins directory of your dev install. I’ll use my recently released plugin Unconfirmed as an example; you can follow along using the example (until it comes time to push and commit!), or use your own plugin.

[code]

cd wp-content/plugins

mkdir unconfirmed

[/code]

The next step will require you to clone your wordpress.org svn repo. This assumes that you have requested and been granted space in the repository already. If you’re just starting plugin development and don’t want to request space yet, that’s OK – just begin your development in a normal Git repository. As long as you are pushing your Git changes somewhere (like Github), you’ll be able to wipe out your local copy when it comes time to send it to wordpress.org and start from this point as if you were starting from scratch.

Get a revision number for the plugin. You don’t want to force git-svn to crawl through the 300,000+ revisions on the wordpress.org repository. (For those keeping score, this is the only time you will have to run an svn-native command!)

[code]

svn log http://plugins.svn.wordpress.org/unconfirmed

[/code]

Look for the first commit number, where the plugin was added to the repository. In my case, it’s r387893.

[code]

r387893 | plugin-master | 2011-05-23 04:10:19 -0400 (Mon, 23 May 2011) | 1 line

adding unconfirmed by boonebgorges

[/code]

Clone the svn repository into the local plugin directory. Here’s the syntax, followed by some explanation.

[code]

git svn clone -t tags -b branches -T trunk -r387893 http://plugins.svn.wordpress.org/unconfirmed unconfirmed

[/code]

  • git svn clone is the command itself
  • -t tags -b branches -T trunk tells Git about the directory structure of the svn repository that you’re cloning. The flag --stdlayout or -s is shorthand for the same thing, though I’ve had mixed luck getting it to work – so I just enter the whole thing explicitly.
  • -r387893 is the svn revision number where I want Git to clone. In the next step, we’re going to tell Git to fetch the svn revision history starting with this commit; that’s why we chose the first, rather than the last, commit from the unconfirmed repo.
  • http://plugins.svn.wordpress.org/unconfirmed is the address of the plugin on the wordpress.org repo. It’s important to use this address rather than the alias http://svn.wp-plugins.org. As I discovered (after much frustration), Git is not able to trace the branch/tag history from the wp-plugins.org address – you have to use plugins.svn.wordpress.org.
  • unconfirmed is the relative path to the directory where I want the repo cloned.

Next, fetch the svn commit history like so:

[code]
cd unconfirmed
git svn fetch
[/code]

When you hit enter, git-svn queries the wordpress.org repo, starting with the initial revision number you pegged in the previous step, and walks all the way forward to the (entire) repo’s most recent revision, mirroring your plugin’s svn revision history in your local Git clone. If you’ve just received the space in the svn repo, this will only take a few seconds. If you’re doing this with a plugin that was put into the repo some time ago, or one that has a lot of revisions (and branches and tags) in the svn repo, you’ll have to wait a long time. Drink a beer or ten while you wait.

You’ll know the process is done when you are returned to your command prompt. Test to make sure that you’ve pulled in all of the remote tree by entering the following:

[code]
git branch -r
[/code]

This will show you a list of all the remote branches being tracked by this Git repository. If it’s a fresh repo, you’ll only see the trunk. If it’s a repo with existing branch and tag history, you’ll see one Git branch corresponding to the trunk, one corresponding to each svn tag, and one corresponding to each svn branch.

If you’re not planning to use Github or to share your code with anyone else, you’re done. If you are planning to use Github (as I do), you’ll need to add your Github endpoint as a remote repository. Assuming you’ve already created a repo on Github,

[code]
git remote add origin git@github.com:boonebgorges/unconfirmed.git
[/code]

origin is the name I’ve chosed for the Github endpoint, but you can call it whatever you want.

Now it’s time to reconcile your commit histories. Hopefully, you’ve only got one history to deal with – either in the Git or in the svn repo – so you won’t have too much trouble. In such a situation, something like the following should work:

[code]
git pull –rebase origin master
[/code]

So, here’s the thing about --rebase. It rolls back, albeit temporarily, all of the changes on your local copy of the repository back to the last common commit, applies all of the remote revisions, and then attempts to apply your revisions on top of it. If you are working with a fresh Github repository, there will be no common commit in the history, and there won’t be any revisions from Git to rebase back onto the tree. Thus, the svn history will be rolled back, zero commits will be put onto the local repo, and the svn history will be laid back down. In other words, nothing will happen, except that the Github repository will establish a common ancestor revision, allowing you to push. Mutatis mutandis, if you have an active Github repo but an empty svn repo, zero commits will be rolled back, and your Github history laid on top of the local repo, with the zero commits laid back on top. In other words, it’ll sync with the Github history seamlessly, and allow you to commit back to the svn repo by establishing a common revision history. If you have active, separate commit histories in both svn and Github, may God have mercy on your soul.

In any case, use --rebase with great caution. Read the git-rebase docs and try to wrap your head around it before doing anything that will mess up the revision history.

Once you’ve successfully pulled from Github, your three repositories – the local Git repo, the Github Git repo, and the svn repo on wordpress.org – will all be aware of each other and in sync. You are ready to develop.

Part Two: Day to day development

Let’s say you’ve found a bug (OH NOES) and you need to fix it. Before digging into the code, get yourself to the command line to make sure you’re in good working condition.

The first thing I do before I touch any code is to see what’s happening in my local repo:

[code]
git status
[/code]

There’s a lot of good information that git status can give you, for which I refer you to the docs or to Google. The important concept for our purposes is that we start a new local branch every time we want to fix a bug or develop a new feature. The notion of hyper-specific, temporary branches is one of Git’s biggest selling points, as well as one of the places where it differs the most from the way that many WP developers work with svn. Since branches are strictly local unless explicity pushed, and since Git’s merge and rebase tools are so nice, you can create, merge, and destroy branches at will, and keep your work separate. To add (and switch to) a branch called stupidbugfix, use this syntax:

[code]
git checkout -b stupidbugfix
[/code]

This does two things: it creates the new branch stupidbugfix, and it checks out that branch (similar to svn switch). git status will show you that you are now On branch stupidbugfix.

Do your bugfixing as you would normally do. When you’re ready to commit the changes, do another git status. You might see a result that looks like this:

[code]
# On branch stupidbugfix
# Changed but not updated:
# (use “git add …” to update what will be committed)
# (use “git checkout — …” to discard changes in working directory)
#
# modified: readme.txt
#
no changes added to commit (use “git add” and/or “git commit -a”)
[/code]

In this case, I’ve made changes to one file: readme.txt. In order to commit these changes, you’ll first need to stage the file with git add:

[code]
git add readme.txt
[/code]

(git add is automatically recursive, so you can add whole directories this way too.) Now you can commit the changes:

[code]
git commit -m “Fixing that stupid bug.”
[/code]

If you’re confident that you want to stage all changed files to a commit, you can skip the explicit git add and use the -a flag:

[code]
git commit -am “Fixing that stupid bug”
[/code]

Now, let’s get your changes from the stupidbugfix branch into the master branch, which we’ll use to push to our central repository. Then we’ll delete the temporary stupidbugfix branch.

[code]
git checkout master
git merge stupidbugfix
git branch -d stupidbugfix
[/code]

If you are sharing your code on Github, you can push your commits at this point:

[code]
git push origin master
[/code]

Part Three: Releasing a new version to the wordpress.org repo

If you’re following the steps I’ve outlined above, all of your development is happening in Git and Github. The only time you’ll need to touch the wordpress.org repository is when you want to release a new version. Here is the procedure I use. [EDIT 2012-09-14: See the Addendum below for an improved workflow for this step.] First, rebase the svn trunk to your current branch.

[code]
git rebase trunk
[/code]

This small bit is crucial, and it took me many months of frustration before I finally stumbled upon this (cache only!) blog post, which put me on the right track. The concept here – so far as I understand it, at least – is that git-svn actually rewrites the MD5 hash that Git uses to identify the changeset, and when you rebase the trunk into your current local branch, you forcing git-svn to match Git-changeset MD5s with svn-changeset MD5s. If you don’t do this, you’ll get infinite merge conflicts.

Now, let’s send those commits to wordpress.org svn.

[code]
git svn dcommit
[/code]

This sends all changesets (since the last time you dcommitted) up to the WordPress svn repository. (In other words, you’re mirroring the revision history.) Once this is done, the revision history on your svn trunk will match that of your current git branch.

Now we’re ready to tag the release in svn. (I’m assuming that you’ve already changed current version and stable tag numbers in your plugin files before dcommitting. If not, make those changes, git commit them, git rebase trunk, and git svn dcommit.)

[code]
git svn tag 1.1
[/code]

This will do the same thing as when you svn cp your trunk into tags/1.1.

I like to keep tag history in Git as well. Git tags are metadata (while svn tags are really just branches), so they have to be created separately, and pushed up to Github.

[code]
git tag -a 1.1 -m “Tagging 1.1”
git push –tags
[/code]

Finally, before making any more changes that can be pushed back to Github, you’ll have to rebase from Github – again, to make sure that Git understands that your Git MD5s have been overwritten by git-svn. Assuming you’re on your master branch:

[code]
git pull –rebase origin master
git push origin master
[/code]

Again, this rebase business is scary and dangerous, so try to understand the possible ramifications if you are working on projects that involve other people.

Conclusion

This might seem like a lot to learn. But developing in Git is hugely beneficial, well worth the learning curve. Git’s agile branching allows a kind of focused, compartmentalized development strategy that can’t easily be replicated with svn. And having Github as a bugtracker and changeset-viewer is also really great. Plus, working with Git in my own development means that I don’t have to code-switch (pardon the pun) when working with clients who use Git for collaborative work, which, I’m finding, is increasingly common. Learning about Git with your own WP plugins is a great way to learn some of the finer points of Git.

Addendum (14 Sep 2012): Be a better wordpress.org citizen and squash

The biggest headache in the git/wordpress.org workflow has always been lining up the revision histories. Git revision histories are non-linear, and SVN does not understand them. So the process of rebasing the development branch into the trunk-tracking branch is always precarious, as trunk is a flattened, svn-friendly branch. Moreover, even when the rebase does work, you end up reduplicating revision histories in both Git and SVN, when the real purpose of wordpress.org SVN here is really just distribution. (This makes Otto sad.)

So, for the last six months or so, I’ve used a modified version of the release workflow described in Part Three. First, I have a local branch, which I call svn, which tracks trunk:

[code]
git checkout -b svn trunk
[/code]

At release time, I check out the svn branch, and do a squash merge. This means that all changes since the last commit to the svn branch are laid on top of svn as if they were a single set of changes. You can then commit these changes as a single changeset (thus keepin’ it linear, and keepin’ it short for Otto’s sake):

[code]
git checkout svn # if you’re not already on it
git merge –squash master
git commit -m “Merging changes from Git for 1.1 release”

[/code]

Note that you might get a message from Git that there were merge conflicts, in which case you’ll need to use mergetool to clean up. But this is a subject for another post.

Then, do your git svn dcommit and git svn tag from the svn branch. It’s important that you never rebase to or from the svn branch, or do any development there; all changes on that branch should be squash-merged from a git-only dev branch.