Category: WordPress

Categories
WordPress

The Case for WordPress Governance

What role does WordPress play in the larger discussions about the web platform and the internet? What responsibilities does WordPress have to speak for the tens of millions of content creators, business owners, designers, developers, organizations, institutions, and governments who rely on this free open source software to share their thoughts, ideas, information, products, and services with the world? And how does WordPress responsibly take part in the external decision making processes which impact not only the application itself but every person who in any way interacts with WordPress?

These questions, and others in the same vein, were the starting point of what would become the WordCamp US 2018 talk Moving the Web Forward with WordPress, and the WordPress Governance Project.

Today, some 4 months later, I want to shed some light on why I think WordPress Governance is so important, and why to me it is less about uprooting existing leadership models and starting a revolution than taking stock of the position WordPress and its community finds itself in and accepting its new role as a driving force of the web.

In this article I speak not as a representative of the WordPress Governance Project. These are my personal reflections on where we are and where I think we need to go next. Consider them part of a larger discourse, and an invitation for you to join that discourse.

33% and growing

A few months ago WordPress crossed a significant mile marker: According to W3Techs, WordPress now powers more than 33% of the 10,000,000 most popular sites of the web as ranked by Alexa. That’s ? of the web gathered under the banner of democratizing publishing through free open source software. Even if we question the methodology or validity of this particular number, the reality is clear to see: WordPress has a larger footprint on the web than any other content management system.

33% is a testament to the millions of hours of free labor the WordPress community has invested in the application. It proves, beyond any doubt, that free open source software published under the GPL is not only a viable but a preferred option in the eyes of the end-user, the people who want to publish content on the web. In short, WordPress has been and remains an unprecedented success story. And that success is the result of a vision of the web defined by project co-founder Matt Mullenweg and supported by every contributor to the WordPress open source project: that web publishing should be available to all; that open source software published under the GPL is the best vehicle to make this happen; that the Four Freedoms must have precedence above all else.

These ideals, combined with a strong belief in meritocracy and an ideological aversion to hierarchical management, have paved the path leading to today. Which begs an obvious question:

If it works, if it got us this far, why change it now?

My answer, “what got us here won’t get us there,” needs both context and more substance. In this article I’ll lay out my two main reasons for wanting governance in the WordPress open source project: Representation for the open web, and deliberately moving the web platform forward.

Who speaks for the open web?

Decisions are being made by politicians all over the world about the future of the internet, the web, and what types of services and contents can be published and used on them. These decisions are already having a direct impact on every person who interacts with WordPress, and there are more to come.

As a person using the internet, and probably using WordPress in some way, two questions should be front of mind in all this:

Who speaks for WordPress when these decisions are made? And what are they saying?

Right now, the answer is nobody, and nothing. This is concerning, and may have significant consequences for the future of the web and the internet, even for those who never interface with WordPress.

If you think I’m being overly dramatic look no further than the UK Home Secretary linking WordPress.com to the spread of terrorism and you realize politicians and lawmakers are in their seats of power, policy proposals in hand, and we have so far chosen to not help them craft those proposals into laws that protect rather than hurt the open web.

To put a fine point on it, right now the future of the internet and the web is being decided by lawmakers with a poor understanding of how these technologies and the communities they foster work. These politicians make their decisions based on input from lobbyists, special interest groups, and industry bodies. While there are some organizations working to promote the interests of the open web, WordPress has been absent in these conversations.

This is a deliberate choice of political non-intervention, and it’s an irresponsible one in my opinion.

By not taking part in these conversations, by not claiming a seat at the tables of power, the WordPress open source project says “we power 33% of the web, but we do not promote the rights of the people who use our software.”

But what policies would a representative from WordPress promote? Who would appoint that representative? And who would they represent? Every WordPress user? The contributors?

All these questions have no clear answers today because the WordPress open source project has no governance structure to allow policies to be created, appointments to be made, or discussions of representation to even take place. Today, the sole representative and only person who can speak on behalf of WordPress with any authority is project lead Matt Mullenweg. When I asked him how we go about representing WordPress in these decision making bodies at WordCamp US in 2017, his answer was quite vaguely that someone should step up and lead. Which is, in part, how the WordPress Governance Project came about.

Necessary conditions as a starting point

I think one of the major sticking points in this whole conversation is how to define policies that can be legitimately said to represent every WordPress user. My proposal is to start by identifying the necessary conditions which must be in place for WordPress to be able to achieve its core philosophy.

The core philosophy of the WordPress open source project is the “democratization of web publishing.” What exactly that means is not clearly defined, but the words themselves have meaning. “Democratize” in this context most likely means “make something accessible to everyone,” so “democratize publishing” means making publishing accessible to everyone.

From this, we can stipulate some necessary conditions including:

  • Unrestricted access to the information on internet
  • Net neutrality
  • Accessibility
  • Privacy
  • Security

These conditions, and others like them, can be turned into concrete policies representatives can bring to politicians and lawmakers around the world. That way, when a proposal for stricter regulation on link attribution, or the removal of net neutrality, or anti-encryption, or pro-privacy laws are proposed, the WordPress project can claim its seat at the table and present comprehensive and consistent policy opinions as the representative of every entity who chooses to use WordPress to publish content on the web.

“These are the necessary conditions for 33% of the open web to flourish” is a strong statement, and one politicians will listen to.

Silence is consent

The current absence of WordPress in these conversations is seen as consent. When 33% of the web stays silent as laws are enacted, the lawmakers read that as 33% of the web approving of those laws being enacted. This silence is not a decision made by the WordPress community. It’s an ideological decision made by WordPress leadership on behalf of the community, and I question whether it is one the community agrees with.

This is a conversation the WordPress community needs to have. To continue thriving on the web, the WordPress community needs to play a role in the decision making about the web. To do that, we need systems in place to propose and ratify principles and policies, elect or appoint representatives, and fund their work to promote the interests of WordPress site owners.

Other communities including Drupal are hard at work trying to put together exactly this type of representation, but WordPress’ absence is a proverbial elephant in the room. If Drupal goes to the EU and says “we speak for our users,” cunning lawmakers and the corporate interest lobbyists who influence them will say “You represent maybe 3% of the web. 33% of the web is silent. Get back to us when they have something to say.”

WordPress should be banding together with Drupal and other open web communities to create a body of influence representing the open web. To do so, WordPress needs the tools and mechanisms to create policy and enact it. That in turn requires governance.

Interlude: Paving the Cowpaths

Behind my childhood home on Nesodden right outside Oslo, Norway, there was a hidden path leading to a bigger path leading deep into the forest. As children my brothers, our friends, and I would take these path as soon as our homework was done and disappear for hours into the wilderness. Over time, what started as tracks of broken twigs and crushed vegetation became well-trodden dirt paths, and eventually the municipality came in to clear away boulders and lay down gravel turning what was a natural path between the trees into a maintained pedestrian road through the forest.

We are a society of road builders, and the roads we build typically start off as paths laid down by animals and people exploring new ways of getting where they want to go. There’s a term for this: paving the cowpaths. We use it to describe the formalizing a de-facto practice, whether that be a path through the forest or a coding pattern for the web.

A significant portion of the modern web standards we work with today were established by paving the cowpaths of the web. Standards bodies look at what established practices exist and what solutions people use consistently, and build new standards around them. Prime examples are how modern JavaScript (aka ES2015, ES6, ESNext, etc) has adopted many of the de-facto practices established by jQuery and similar libraries, and how modern CSS has adopted many of the de-facto practices established by preprocessors like Sass. Seeing cowpaths being formed on the web, the standards bodies get together to pave them, formalizing the practices and making them part of the web platform proper.

What does this have to do with WordPress? To put it bluntly, at 33% and growing, WordPress is the cowpath. Or rather, wherever WordPress decides to go, it puts down a cowpath wider and more established than anything else on the web. That means wherever WordPress goes, others will follow, and standards bodies, browser manufacturers, and every other player in the ecosystem surrounding WordPress has to start paving the path laid down behind it.

I propose this is a one reason why large multinational corporations, major hosting companies, and other commercial parties are investing heavily in developer resources and contribution to the WordPress open source project: Pushing WordPress in a particular direction means pushing the web in a particular direction. WordPress is becoming a vehicle driving the web forward. But where exactly that vehicle is going is not clear. And that’s a problem, not just for WordPress but for the web.

Moving the Web Forward with WordPress

A few months ago a friend reached out to me asking what my goal was for the WordPress Governance Project. I told him I want to move the web platform forward with WordPress. “WordPress could be a vehicle to push the web forward,” I said. “Imagine if WordPress collaborated with the various standards bodies governing the web to implement new features. We could quite literally deploy a proposed standard to +33% of the web over night thereby establishing it as a de-facto practice ready to be implemented across all browsers. It could mean an end to waiting for years for tools like flexbox, CSS grid, service workers, and modern JavaScript to become well established enough to get widespread support.”

He shook his head and said “From where I stand, it’s the other way around. WordPress is conservative, regressive even. It’s not moving the web forward, it’s slowing it down.”

I wish I could say he was wrong.

From my perspective WordPress is cautious to the point of being conservative. And in some cases it is indeed regressive. There are many causes at play here, maybe most importantly WordPress’ policies of backwards compatibility, continued support for older browsers and server infrastructure, and reluctance to adopt progressive enhancement principles.

There are good reasons for these policies and practices: WordPress powers 33% of the web, and its users are distributed across every continent and country on our planet. WordPress needs to work for all the people who use it, and that means providing solutions that work on old and outdated infrastructure, in poor connectivity environment, on pay-per-byte connections, on legacy devices, running outdated browsers.

But these challenges are not unique to WordPress, and they do not need to be a grand piano dragging behind the vehicle of progress. What WordPress needs is a long-term strategy for upgrading the web while protecting every user. That requires a stated vision and goals. That requires the involvement of subject matter experts from within and outside the community. That requires the establishing of plans, assignments, roles, and delegation of tasks. In other words, we need to rethink how the WordPress open source project is structured, from the ground up.

Leadership matters

What got us here won’t get us there, because what got us here was a flat-structure management system based on meritocracy where each contributor works on what they find valuable, and the application evolves based on their contributions. This model is designed for small open source projects developed by the people who use it. WordPress is a mission-critical application built by a dedicated team of contributors for tens of millions of people, businesses, and organizations who never interface with the WordPress community. On this scale, leadership matters.

What’s needed to move forward is a system where the community agrees on a direction, tasks are defined and delegated, and decision makers are assigned to ensure things are done and done correctly. This is how well-functioning organizations work, and it’s for a good reason: No single person can have complete oversight of an entire complex project. Without proper hierarchical leadership structures, management becomes non-existent and project success relies entirely on individual contributors. In a leadership vacuum, ad-hoc leadership structures naturally emerge introducing confusion and often conflict. The end result might be functioning software, but it is guaranteed to be wasted time for everyone involved. Leadership, and leadership structures matter.

Many who read this will immediately object saying this is not how we’ve done it so far, it goes against the very essence of open source, it is not tenable, it means some people get to decide what other people (volunteers) work on, etc. These are fair arguments. In response I’ll say WordPress powers 33% of the web, and every decision made in WordPress affects tens of millions of people. Continuing to develop WordPress in an ad-hoc fashion is no longer responsible. Furthermore, it prevents us from being a true vehicle of progress for the web. And as Gutenberg has proven, it is also not how things are done today:

The evolution of WordPress is not as ad-hoc as it appears. It is very much guided by the grand vision of Matt Mullenweg, and his vision is the reason for WordPress’ continued success. The ad-hoc nature of the evolution happens further down the decision tree, in the minutia of application development. That’s where chaos takes root, conflicts arise, and significant volunteer time and resources are wasted, all because there is no system in place for proper project management. There is no governance.

When I told my friend about my vision for WordPress being used by standards bodies as a vehicle to move the web forward, his immediate response was it wouldn’t work because WordPress can’t be trusted: “Right now” he said, “anyone from WordPress can come to a meeting of a standards body and say they want to implement a new standard. But there’s no guarantee that will actually happen. We can’t trust WordPress to actually do what it says, because nobody speaks for WordPress. Even if a solution is built, there is no guarantee it gets implemented. The process of actually accepting something into core seems arbitrary at best. It’s like there’s a cabal of secret leaders in the project giving each feature a yay or nay. That’s not something we can work with.”

The reality is WordPress does have a hierarchical management structure, and there are people in the project with powers to stake out a vision and delegate tasks. One of those people is Mullenweg. Other people include sub-project leads appointed by Mullenweg. Yet others are people with core contributor access. All of them are promoted to leadership positions through meritocracy. Yet, if you ask them if they are leaders in the project, many will say “no” and explain WordPress doesn’t have leadership; it just has contributors and the playing field is even. This is, and pardon my harsh words here, nonsense. And the refusal to admit there is hierarchical leadership in the project is a problem.

Let me give you one general example: Over the years I’ve observed subject matter experts attempting to contribute to the project only to be overruled by contributors with higher meritocratic status. In several of these cases, the refusal to accept the contributions of the subject matter expert was due to a lack of understanding of the issue by the person turning the proposal down. Because of the lack of any type of transparent leadership structure within the project, there is no clear way for a contributor to raise this issue and get a fair resolution, so they shrug their shoulders and walk away.

What WordPress needs to move forward is to have a conversation first about how to manage a large open source project responsibly, then how to introduce a long term vision and concrete plans for the future of the project and the web, and finally how to introduce more hierarchical management structures with checks, balances, and decision makers to ensure the contributions made by thousands of volunteers result in meaningful progress for the application and the web.

These are conversations and decisions we need to make together as a community.

Only with these structures in place can we start taking part in the larger conversation about how to move the web platform forward, and how to use WordPress in a meaningful way to do so.

We can move the web forward with WordPress, but to do so we have to act like the leaders we want to be. If we don’t, WordPress could become a battleground for competing corporate interests who want to use WordPress to carve paths into the futures they envision.

Build the future you want to live in

People ask me why I started the WordPress Governance Project. My answer, as you’ve seen in this article, is it’s complicated.

I believe WordPress can be a force of good on the web, but the way things are right now we can’t take on that role because we don’t have a clear way of governing ourselves.

I’m not here to start a revolution, or a coup, or even stir things up. I’m here with a plea to the community to take a breath and think about how we got here, where we want to go, and what we need to do to get there.

WordPress is important. For many it is mission-critical. We owe it not just to the people who use WordPress or to the web but to ourselves to take stock and create the necessary structures for WordPress to continue to thrive on an open, accessible, progressive web for years to come.

WordPress needs governance. What that looks like, I am not sure. That’s why the WordPress Governance Project exists: to have the conversation; to explore possibilities; to work together and find solutions.

Those are my thoughts. I’d love to hear yours.

Categories
gutenberg WordPress

A simpler way to add SVGs to custom WordPress (Gutenberg) blocks using SVGR

For the past several months I’ve been developing a new course about custom WordPress block development for LinkedIn Learning. Early on in the process I started looking for a simple way to add custom SVGs to custom blocks. The solution (or more specifically a solution) was found deep inside Create React App: SVGR.

From the SVGR documentation:

SVGR transforms SVG into ready to use components. It is part of create-react-app and makes SVG integration into your React projects easy.

SVGR documentation

Unifying custom block development with @wordpress/scripts

To take full advantage of the features of both the WordPress block editor and modern JavaScript, you need to use some combination of Webpack and Babel to introduce full support for ECMAScript 2015. Configuring Webpack and Babel and all the necessary WordPress packs and everything else can be a challenge. To make things simpler, the Gutenberg team has released @wordpress/scripts aka wp-scripts, a “collection of reusable scripts for WordPress development.” This package has now been updated to include configurations for Webpack and Babel to provide a unified build process for the WordPress community.

Using SVGR in WordPress (Gutenberg) block development

In my own block development I’d been using a custom Webpack + Babel + a bunch of other stuff setup. This setup included SVGR for simplified SVG inclusion. Last week I refactored my build process to using @wordpress/scripts and in the process had to figure out how to extend the pre-packaged webpack configuration to include my custom script.

Since there’s a good chance you’ll want to add some SVG magic to your WordPress blocks, I figured I’d share my findings with you here. I’ve also added a simplified code example to the official @wordpress/scripts documentation.

By the end of this process, you’ll be able to

  • turn any SVG into a React component
  • have that SVG output inline by calling the React component
  • add the SVG as a base64 encoded image element if you want to call it directly from the src attribute inside an <img> element

For this we need two packages: @svgr/webpack and url-loader.

Here’s the full breakdown (this assumes you are already using @wordpress/scripts):

From within your project folder where package.json resides, run the following commands in terminal:

npm install @svgr/webpack --save-dev 
npm install url-loader --save-dev

Create a new webpack.config.js file.

Require the original webpack.config.js from @wordpress/scripts:

const defaultConfig = require("./node_modules/@wordpress/scripts/config/webpack.config");

Extend the existing config with custom settings for @svgr/webpack and url-loader:

module.exports = {
  ...defaultConfig,
  module: {
    ...defaultConfig.module,
    rules: [
      ...defaultConfig.module.rules,
      {
        test: /\.svg$/,
        use: ["@svgr/webpack", "url-loader"]
      }
    ]
  }
};

Place your SVG(s) in the /src folder of your project.

In your index.js file (where you’re using registerBlockType()), import the SVG:

import customLogoURL, { ReactComponent as customLogo } from "./logo.svg";

Now you can call customLogoURL (the SVG transformed into a base64 encoded URI) and customLogo (the React component) from anywhere within your block definition. Example:

registerBlockType("block/myblock", {
  title: __("A Block", "myblocs"),
  icon: {
    src: customLogo // The React component
  },
  category: "regular",
  attributes: {
    blockImage: {
      type: "string",
      default: customLogoURL // The base64 encoded URI
    }
  },
  (...)
}

Cool, right? And clean and easy to read.

Stay tuned for my upcoming LinkedIn Learning course on WordPress Block Development where you’ll learn more about how to build your own custom blocks!

Cross-posted to LinkedIn Pulse.

Categories
governance WordPress

WordPress: Users, Stakeholders, and Known Unknowns

Who is the WordPress user? It’s the question everyone involved in WordPress in some way asks themselves at one point or another. And it’s a question without any clear answers beyond the Open Source Dogma: The User is anyone using the software. Which, to be frank, is not very useful.

Yesterday (February 26th, 2019), the WordPress Governance Project published a research document titled “Identifying the Stakeholders of WordPress” which serves as a starting point for a rigorous exploration of the question: “Who is the WordPress user?” This is a question we, as a community, need to answer, and to find the answer we need to work together. Consider this your invitation to take part: Read the document, contribute your comments, and if you find something missing or inaccurate, help correct it.

Personas: a Primer

One of the tools in the design process toolkit is the Persona, a “fictional character created to represent a user type that might use a site, brand, or product in a similar way.” Personas serve as prototypical users with properties and attributes representing user groups. Over the years, the idea of the Persona has evolved from a strictly fictional character to a more nuanced representative entity. Personas are often paired with Empathy Maps (XPlane, NN Group), Journey Maps (NN Group), Affinity Diagrams, and Microsoft recently introduced the idea of Persona Spectrums (Microsoft Inclusive Design).

Personas and associated tools are there to help designers and developers get a better understanding of their users. They allow us to ask questions about the user’s Goals and Needs, Hopes and Fears, Challenges and Opportunities. They help us define success and failure criteria for projects. And they give us direct references to point to when asking questions about decisions within the design process.

That said, a Persona is only as good as the data it’s based on, and a good persona is typically built using data from both qualitative and quantitative research.

For a project like WordPress, much of that research is lacking. Which begs a question whether creating personas for WordPress is possible, or even feasible. I think it is, and here’s why:

Personas for WordPress?

I teach design principles both as an instructor with LinkedIn Learning and at Emily Carr University of Art + Design. A central part of the process I teach is clearly defining and investigating both the Stakeholders and target Users of the design project. In simple terms, the distinction between “Stakeholder” and “User” in this context is the level of ownership and vested interest. For an ecommerce site, the Stakeholder is anyone with ownership of and/or a vested interest in the success of the store, while a User is someone literally using the ecommerce site to find, research, and possibly buy a product. For WordPress, the distinction between Stakeholder and User is more obscure, if it exists at all: Every person who uses WordPress in some way, even if it’s just as a visitor to a site powered by WordPress, is a Stakeholder in some small way because of the ideology WordPress is rooted in: Democratizing publishing through open source software. By using WordPress in any way, a person takes an active part in the furthering of this ideology, which makes them a Stakeholder.

Early on in the WordPress Governance Project meetings, the question of a clear definition of WordPress Users and Stakeholders came up. With no official definitions beyond “WordPress is for the people who use it” and “WordPress is for everybody”, the project started work on a research project to group the different stakeholders of WordPress into Personas. The document presented yesterday is the beginning of this research project.

I should preface this by saying I was not part of the team working on this project. My role was solely as a reviewer of the final text. That said, I stand behind the result 100%.

To start, there are two statements:

  1. The WordPress User is anyone using WordPress (past, present, or future)
  2. Every WordPress User is by definition also a Stakeholder in the project.

If these two statements can be agreed upon as a hypothesis, it is possible to work from them to create personas identifying the different types of Stakeholders within the project. This is important because it turns the poorly defined amorphous blob of the “WordPress User” into structured and segmented groups of Stakeholders, each with their own distinct Goals and Needs, Hopes and Fears, Challenges and Opportunities. This grouping also allows for a weighting of influence for each group: When the goals or needs of one group conflict with those of another, what group takes precedence? How is this weighting decided? And what mechanisms are in place to challenge this weighting? Defining Stakeholder Personas is an essential part of understanding WordPress: Its impact, its cultures, its diversity, and its governance.

This is the beginning

“Identifying the Stakeholders of WordPress” is not a complete and final decree of how the WordPress Stakeholders should be sorted into Personas. It is the beginning of a longer process of identifying Personas within the WordPress user base so we can better serve those we build the application and its community for.

In the wake of the publication of the document, some relevant queries have been raised including questions about its purpose (which I believe I’ve answered above), and what research it was based on. This last question brings us back to something I mentioned earlier: if we don’t have qualitative or quantitative research to build on, how can we define reliable and accurate Personas? Here’s my answer:

The liberal definition of the WordPress user as “everyone” using WordPress is, from my perspective, dangerous because it stands in the way of having serious conversations about the real impacts WordPress has on the people who use the application because with such a diverse user group, it appears as if no clear Personas can be defined. However, that definition is unlikely to change in the foreseeable future so as they say, it is what it is.

If we take this definition literally, it can be further interpreted as “anyone in the world with the capability to use WordPress” meaning anyone with access to a computer and a reliable internet connection. If we agree to this broad definition, we have made the user base so broad and diverse any research about the internet-using public becomes relevant data for the creation of Personas for WordPress. This means until we do extensive and rigorous qualitative and quantitative research on the WordPress user base, we can rely on other research to inform our Personas.

The Stakeholder Personas proposed in the Identifying the Stakeholders of WordPress are a starting point for this process. The document proposes rational Personas based on levels of involvement and types of uses of the application. Moving forward, we need contributions from people like you to improve these Personas through research (new or related) and exploration.

Contributions welcome!

Categories
WordPress

Newspack: Automattic, Google, and the SaaSification of WordPress

Earlier today Automattic (the company behind WordPress.com headed by WordPress co-founder and project lead Matt Mullenweg) announced Newspack by WordPress.com, a “next-generation publishing platform” “aimed at small- and medium-sized news organizations”. The new platform is funded in part by Google, through the Google News Initiative, and other companies. Here are my preliminary thoughts.

A missed opportunity for established players

While I’m zero percent surprised this service has been announced, I’m surprised it didn’t come sooner, and from someone other than Automattic. Automattic is not uniquely placed to launch this project, it is just the only player in the WordPress ecosystem with a proven record of offering WordPress as SaaS (Software as a Service). I’m surprised nobody else beat them to it: There are agencies in the WordPress ecosystem with extensive experience working with the exact target audience of Newspack (small- and medium-sized news organizations). In my mind these companies were uniquely placed to either individually or together launch exactly the type of service Automattic has announced.

These agencies have long offered a mix of custom self-hosted WordPress setups and integration with WordPress VIP (also from Automattic). Now the question becomes whether the Newspack platform will be open to agencies in the same way WordPress.com and WordPress VIP are (history points to “yes” on that front) and whether the above mentioned agencies will offer services to its users.

The SaaSification of WordPress

Back in 2013 I wrote a post on my blog predicting a splitting up of the WordPress ecosystem into specialized silos targeting industry segments with unique needs. My argument back then was the user base of WordPress is too heterogenous for one solution to fit all needs. In short, a hobby blogger does not have the same needs as a newspaper. What we’ve seen in the subsequent years is WordPress (the application) continuing in its attempt at being everything to everyone, and Automattic (and in small ways some other agencies and service providers) booting up customized SaaS solutions to fit the unique needs of different user groups. Automattic now has WordPress.com for blogging and small site building, WordPress VIP / Professional for large enterprise solutions, and the just announced Newspack for news. My expectation is they’ll soon offer a dedicated solution for e-commerce as well, through a custom SaaS offering powered by WooCommerce.

My prediction is this latest announcement marks the beginning of the all-out SaaSification of WordPress. The reality is because of the open source status of WordPress (the software), what Automattic has done with WordPress.com and their other services can be done by anyone, and I’d be shocked if Automattic stays the only player in the market for long. To put it bluntly, the days of WordPress self-hosting as the go-to solution for small-, medium-, and enterprise businesses are numbered.

What competing services like Wix and Squarespace have shown us is people want simple solutions to their web hosting that do not involve having a horde of web maintainers on staff or hiring an expensive agency to run everything for them. The obvious answer to this need is to spin up WordPress as a service and extend it to provide customized services for each use case.

We already see this in small ways. Managed hosting companies like WPEngine and Kinsta specialize in hosting and keeping up to date WordPress sites for users with above-average needs. Plugin providers like Yoast offer subscription-based services that reach well beyond what a plugin can do on its own.

The near future will likely see some of these services spin up their own WordPress-powered SaaS offerings targeting specific users. And they may not even say it’s powered by WordPress! As Automattic has demonstrated, you can slap a whole new skin on top of WordPress core and create a custom experience which looks and feels nothing like the main application. The Wix / Squarespace killer may well be a new service running WordPress in the background and a completely unique experience on top. Think managed hosting + a page builder like Elementor or Beaver Builder and a host of other customized services and you see where I’m going with this.

The Fracture

If my predictions here come to pass, the fracture I spoke of in my 2013 article might well be upon us. The divide between self-hosted DIY WordPress users, those on a SaaS solution, and enterprise customers is already big enough the Venn diagram is starting to come apart. The further SaaSification of the ecosystem will make this fracture permanent and put enormous pressure on the WordPress Open Source project.

If/when commercial entities like Automattic start fighting for control over the open source project to shoehorn in the services they need for their new SaaS project, the priorities of the project as a whole will suffer. How WordPress, as a community, decide to deal with this will determine the future viability of the open source project.

At this moment, my crystal ball is pretty foggy, but I’m starting to see a path emerging where SaaS becomes the bread and butter of WordPress agencies and they eventually evolve their products into discrete forks, at which point their participation in the open source project becomes irrelevant and they back out.

Cross-posted on LinkedIn.

Categories
WordPress

Gutenberg, Forks, and the need for an LTS version of WordPress

Over the past week, developments which I predicted back in December last year have come to fore, and I am deeply concerned about the effects they will have on WordPress (the application) and the community unless we take decisive action.

Short version: For various reasons, many WordPress users will be faced with a complex dilemma when 5.0 and Gutenberg comes out:

a) Get the latest version of WordPress and risk compatibility issues / costly retraining, redesign, or entire rebuilds, and/or other problems, or b) choose not to upgrade and end up running an old and eventually insecure version of the content management system.

So far, the response from WordPress leadership has been to install the “Classic Editor” plugin which as the name suggests reintroduces the classic WYSIWYG editor once the Gutenberg Block Editor becomes the default. This is, in my opinion, a dangerous road to go down both for the end-user and WordPress itself. 

Classic Editor as a permanent solution won’t work

Classic Editor is a bit like using a band-aid to plug a hole in a ballon as you are inflating it. It may work right now, but as the balloon continues to grow, the band-aid not only won’t do its job, it will actively harm the balloon itself. Here’s why:

Gutenberg as an editor replacement is just the first step for the WordPress Next project. The next step for Gutenberg is to migrate to the Customizer, at which point blocks move out of the editor and into other interfaces and displayed spaces in WordPress. Which means the Classic Editor can’t do its job and another patch plugin needs to be introduced.

It does not require a lot of imagination to see how this solution is not scaleable.

On top of this comes the issue of plugin and theme developers having to serve up solutions that work both for Gutenberg and non-Gutenberg installs which dramatically increases complication, cost, and management. And the “this plugin only works with Gutenberg” path flies in the face of the deeply held WordPress ideology of everything working for everyone and breaks the reasonable expectations of end-users. 

From my perspective (you are free to disagree – please leave your own opinion in the comments below) the Classic Editor plugin was a misstep and should be abandoned for a more robust solution, A Long-Term Support version of WordPress 4.9.x (link to Trac ticket):

WordPress LTS

On the release of 5.0, give site owners the option of “freezing” their install at 4.9.x by choosing to stay with a terminated LTS (Long-Term Support) branch which only receives security and maintenance updates, and only for a specified number of years.

Practical implementation

When 5.0 ships, site owners are presented with a panel similar to the mockup added below where they can choose to upgrade to 5.0 or stay with the LTS version. Information about each choice and what that entails is provided in links in the panel.

Proposed modal window offering upgrade to WordPress 5.0 or off-ramp to LTS version.

Users who choose the LTS version are given two options in the Updates panel: “Update WordPress LTS” (security updates only) or “Upgrade to 5.0“.

Why this is necessary

While the majority of WordPress users will weather the 5.0 upgrade without issues, a not insignificant group will run into issues which may end up requiring a significant investment in time or money or both which they are not able to take on at this time. Providing a clean off-ramp allowing them the time and space to prepare for the upgrade or otherwise resolve their unique challenges is essential to avoid harm to the end-user.

Additionally, offering a LTS version provides a relatively uncomplicated solution to the problem of backwards compatibility faced by many plugin developers.  As I mentioned earlier, the popularity of the Classic Editor plugin means plugin developers have to either ship two versions of their plugins (Gutenberg and Classic) or ship a larger plugin with both options built in. An LTS version of WordPress would mean these developers could offer a similar LTS version of their plugin as it was when 4.9.x was released, and continue development on their 5.x support plugin as normal.

This will split the community

A public LTS version of WordPress will split the community between those who upgrade to 5.0 and those who stay on LTS. This is neither new nor is it unusual. There are countless WordPress sites out there intentionally or unintentionally running older versions already, and within the community there is talk of launching an official fork of WordPress which would see a true split.

An LTS version is the lesser of all evils in this situation in my opinion. It is an unavoidable fact that some users will choose to not upgrade to 5.0. How this issue is addressed will impact the development of the application, its community, and its users long term. Offering a Long-Term Support version, and making each user who considers it aware of what this means, falls squarely under the open source philosophy and the WordPress philosophy of democratizing publishing. Granting users agency and the capability to choose solutions that work for them is a key ingredient to keeping the open web alive. That’s what we all strive for. 

LTS is what WordPress already does

WordPress has a long shipped security updates for older versions for an extended period of time, so technically this is no different from what is already happening. My proposal is to make this an explicit feature of the WordPress 5.0 release by giving each site owner the ability to opt out of the 5.0 upgrade until they are ready by activating a “WordPress LTS” release.

On a personal note…

This proposal might seem controversial, or inflammatory, or playing into the #WPDrama or #GutenDrama or #GutenSpiracy or whatever. It really is not. All I’m proposing here is to take something we already do – offering LTS versions of every version of WordPress – and make it a feature rather than something which happens behind the scenes. It changes nothing about how Gutenberg works or how it is implemented, and changes nothing about how we work. Only two minor things are different:

  1. Every WordPress site owner is offered an explicit choice to upgrade to 5.0 or stay with LTS for however long they want (within the clearly defined support timeline)
  2. The Classic Editor plugin is deprecated as a solution to Gutenberg upgrade woes.

That’s my take. Now I want to know what you think. Leave a comment here, or share your thoughts with me on LinkedIn or Twitter. 

Oh, and this proposal is also a Trac ticket: #44851

Categories
WordPress

WordPress: The 15 Year Revolution

Salt Crystals. Photo by Ales Kladnik

Tuesday May 27th, 2003 is a significant marker in the history of the web. On that day 15 years ago, in a tiny corner of the internet, a new software package titled “WordPress” made its debut. It’s goal: Make this thing called “blogging” possible for anyone and everyone.

Just three years earlier I sat in a lecture hall at the University of Oslo where a professor explained how open source software was a dead-end; the fever dream of anarchist software developers with no footing in the real world. “Free software,” he proclaimed (I’m paraphrasing here), “is a waste of time. Ten years from now we will look back on this whole movement in shame.” The same professor also wrote off JavaScript as an irrelevant experiment in useless application design soon to be extinguished.

Looking back at these statements from today where open source software powers much of the internet and JavaScript is eating the web it is hard not to scoff at his seeming ignorance. But back then Open Source really was a fringe movement in an industry still in its infancy, and it was hard to imagine how giving software away for free and allowing anyone to copy it and turn it into whatever they wanted would be a good idea.

All things to all people

Today WordPress powers more than 30% of the top 10,000,000 sites on the web. It is the engine driving your friend’s dog blog, the website of the flower shop down the street, the membership site for your gym, the fundraising site for your favorite charity, the newspaper you read every morning, and the sites of your local, federal, and international governments. If you spend any time on the web today, you are almost guaranteed to encounter a WordPress site. Though you probably would never know unless you inspected its code base. From its roots as a “blogging platform”, WordPress has evolved into a full-featured Content Management System and web publishing platform.

GPL or bust

The key to WordPress’ success, what turned a tiny blogging application into the go-to publishing application for the web, was captured in the last sentence of the announcement post published all those years ago:

“WordPress is available completely free of charge under the GPL license. Enjoy!”

The GPL, or GNU General Public License, is one of the most radical “copyleft” licenses software can be published under. It embeds in the software itself the “four freedoms that every user should have:

  • the freedom to use the software for any purpose,
  • the freedom to change the software to suit your needs,
  • the freedom to share the software with your friends and neighbors, and
  • the freedom to share the changes you make.

The GPL is also passed on to any derivative work meaning anything built on or with WordPress (which is GPL) automatically inherits the same GPL license.

The use of GPL in WordPress is not unique. What makes WordPress special is how fiercely and literally the GPL is enforced by project leader Matt Mullenweg. Over the 15 years of WordPress’ life, Mullenweg has entered the fray numerous times to protect and uphold what he sees as the absolute freedoms of every user of WordPress. And though this has caused no small amount of controversy, it is in my opinion one of the main reasons for WordPress’ success and its transformative impact on the web and the web and internet industries.

Democratize publishing

With Mullenweg’s vision as its foundation, a thriving world-wide community of content creators, software developers, and community organizers have made reality of the project mission, to mission of the WordPress project is to “democratize publishing through Open Source, GPL software.” Thanks to WordPress, anyone with an internet connection and access to basic web hosting can create their own website to publish their ideas, creations, products, and communities. And thanks to the GPL embedded in WordPress, anyone can contribute back to the project in any way they desire. Around the world, more than 1300 Meetups and hundreds of WordCamps bring users, administrators, designers, and developers together to learn about and share their knowledge of and build the application and its community. Around the world thousands if not millions of people make a living thanks to WordPress. And around the world, millions of people leave their mark on the web through their WordPress sites every single day. No small feat for a bunch of files you can download for free.

WordPress Next

As we enter the 16th year of WordPress, much of what made the application unique has been adopted by its competitors and companies including Mullenweg’s own SAAS platform WordPress.com are offering WordPress-like features with extended management options for a premium. Where WordPress goes next will determine not only its future success or whether it reaches the lofty (and in my opinion irresponsible) goal of powering 50% of the web, but whether it will continue to push the web forward and keep web publishing democratic and accessible to all.

With a 30% market share WordPress has the potential to drive the web forward through participation in web standards bodies and by simply implementing its own opinions of how the web and the internet should work. So far, the application and its community has been agnostic to the point of riding the back seat in these conversations; a consequence of its roots as a tool built by the people who show up with no clear management structure or long-term strategy. This inaction has not gone unnoticed, and large influencers like Google have realized WordPress can be used as a vehicle to bring us to the future of the web faster. One of our major challenges as a community will be to navigate this new reality and find ways to make decisions about how WordPress should wield its power to democratize publishing beyond its own borders.

The challenges facing the application are also internal. As I write this, the WordPress project is in the midst of its largest and most revolutionary evolution to date. Project Gutenberg, announced some 18 months ago, reimagines the very nature of content creation treating each piece of content placed in a post, or page, or view as an individual block with its own unique properties. The WordPress block concept is a fundamental transformative departure from the core feature of the application itself – the content blob – that may carve a new path into the future for web publishing as a whole. The question yet to be answered is whether this new vision for the web will bring another generation of users to the WordPress table. Only time will tell.

WordPress and me

I published the first post on my own blog a late summer evening 11 years ago. At the time I’d transitioned from table-based layouts through Flash and DWTs and ASP.NET to Mambo and Joomla! and Drupal and eventually WordPress. What I found in the little blogging engine that could was an end-user experience for my clients that allowed me to build what they needed and them to manage it without running the risk of destroying everything in the process. Over the years I’ve transitioned from WordPress user and implementer to theme developer, educator, and project contributor. I’ve met some of my closest friends and collaborators through WordPress. I attribute much of my career to the success of WordPress. And I have every hope that 15 years from now my son will be building his own web experiences using software that traces its roots back to the application I write this article in today.

We have proven the democratization of publishing is possible. Our next challenge will be using that power to bring democracy to every corner of our shared world. I invite you to join us.

Categories
gutenberg WordPress

Gutenberg and the Future of WordPress: Conditions for Success

NB: This article is part of a series on the new WordPress editing experience called "Gutenberg". If you are not familiar with Gutenberg, check out the first article in the series.

As the Gutenberg project moves ever closer to merging with WordPress, users, administrators, designers, developers, and core contributors are raising concerns about timelines and what will happen to the community once the merge is complete. I've been thinking about this for the past year and over the last few weeks I've had several conversations that have helped bring into focus the key conditions I think need to be in place for Gutenberg to be a success. I present them to the community to start the conversation. Please add your own voice in the comments below, on social media using the #Gutenberg hashtag, on your own blog or website, and in the GitHub repository. Every viewpoint is relevant, every voice matters.

Before we continue, be advised these are my personal opinions. As of this writing I am not a functioning member of the Gutenberg team nor am I speaking on behalf of anyone other than myself. What you'll read below does not indicate nor dictate the future course of this project, they are the conditions I personally think need to be met for the success of the project. In my opinion. Clear? Ok, you can continue:

Condition 1: Accessibility

The web was built to make content and information accessible to anyone and everyone. WordPress' core principle is to "Democratize Publishing" and a key part in achieving this is to make sure WordPress itself and the code generated by WordPress is accessible.

The first condition for the success of Gutenberg is accessibility, both on the back end and on the front end. WordPress core is ostensibly accessible, and over the past several years significant efforts have been made put accessibility at the center of every conversation about WordPress. As Gutenberg merges with core, it is imperative that accessibility is not just maintained but improved.

Web accessibility is not just good practice (an inaccessible site is a broken site) and a moral duty, it is also becoming a legal requirement in many regions. Meeting this condition is a necessity for the future success of WordPress and the millions of sites it runs.

For this condition to be met, Gutenberg and the merged version need to be able to pass independent comprehensive accessibility review.

Condition 2: Change moratorium and ample ramp-up time

During his State of the Word address at WordCamp US 2017, project lead Matt Mullenweg suggested Gutenberg would be "ready for the widest audience" in April 2018. The comment is ambiguous (view it for yourself here) and it is not clear whether he means the beta plugin is ready to be pushed to all WordPress users for testing or Gutenberg is ready for core merge or if he thinks Gutenberg will be merged in core and WordPress 5.0 will release in April. I think (hope) he means Gutenberg will be ready for full-scale testing in April and will be pushed out so people can test it before it is merged to core at that time. If I'm informed otherwise I'll update this section.

Be that as it may, the second condition for the success of Gutenberg is that the community is given enough time to prepare before WordPress is updated. To achieve this, once Gutenberg is considered ready for core merge, the Gutenberg plugin should be advertised to all WordPress users along with a training regimen, and I propose a 4-6 month moratorium on Gutenberg and core updates from the time Gutenberg is considered ready and WordPress 5.0 ships. This sounds extreme, but it is necessary for several reasons, most notably:

Gutenberg changes how WordPress works

WordPress users need time to prepare for Gutenberg. They need comprehensive documentation and training, and need time to figure out how to run their blog/website/e-commerce store/community using this new version of WordPress. This is especially important for large-scale operations like educational institutions and publishers as they typically have complex publishing processes in place that need to be updated.

Without a significant marketing push and proper documentation and training, WordPress users will feel as if their platform has been changed without their knowledge and it will cause them unnecessary hardship. Bridget Willard has written about this in more detail. A moratorium would provide ample lead-up time to ensure WordPress users and the businesses that rely on the platform have time to prepare.

Plugins and themes need to be updated in advance

Because Gutenberg changes the way WordPress works, many themes and plugins will either work differently or not work properly with the rollout. Unless they are updated in advance. A change moratorium provides enough time to allow plugin and theme authors to a) learn how to build solutions for Gutenberg, and b) roll out necessary updates in advance of the final release. The goal here is to ensure when the switch happens, WordPress seamlessly transitions from old to new and everything continues to work.

For that to happen, plugins and themes must already be updated to account for Gutenberg before WordPress updates. The alternative, to update WordPress and then be told to update a long list of plugins, or worse be told some plugins or themes will no longer work, is not tenable as it will appear to the people who use WordPress as WordPress being broken.

If/when plugins and themes are updated, they will have to invest time and money in providing documentation and training for their users to help them prepare. Which leads us back to the first point: Gutenberg changes how WordPress works.

Condition 3: A line in the sand

This is easily the most controversial point, so I'll reiterate this is my personal opinion.

The third condition for the success of Gutenberg and the WordPress of the future is for WordPress to cleanly fork itself at version 5.0. Let me explain:

Even if the two other conditions are met: Gutenberg sets a new gold standard for accessibility on the back- and front-end, and the community is granted ample time to prepare, train, reengineer, and build their businesses around Gutenberg, there will still be a large number of WordPress sites and solutions that will be negatively affected by this update. There are many reasons for this:

  • Plugins and themes that won't update for various reasons.
  • Custom setups that would need to be completely reengineered to work under Gutenberg.
  • Custom setups that are too expensive to update.
  • Processes and procedures that require expensive retraining of tens or hundreds or thousands of users.
  • Legacy systems that can't be updated to Gutenberg for various reasons.
  • your reason goes here.

Bottom line, there are valid reasons why many WordPress users and WordPress sites can't or will be unduly negatively affected by the upgrade to Gutenberg. The only responsible course of action from WordPress is to provide a clean solution for these users to opt out of the upgrade and stay on the last stable release, which will be 4.9.x. In other words, WordPress forks itself at 5.0 creating "WordPress Classic" for users and sites that can't or won't upgrade.

For this to work, WordPress Classic would need to accept a feature freeze receiving only security updates. If it evolves independently, we'll invariably end up with two incompatible solutions.

Here's the thing: If this is not done, the community will fracture and those who rely on pre-Gutenberg WordPress will either refuse to upgrade causing major security and maintenance issues, force a fork which will split the community, or abandon WordPress for other options. Neither of these outcomes are beneficial to anyone.

Benefits of a clean fork

Cleanly forking WordPress at 5.0 brings benefits for the Gutenberg project and for WordPress as a whole:

  • The requirement of backwards compatibility can be removed, or reset to 5.0 meaning WordPress' code base can finally be updated and modernized.
  • Data structures inside WordPress can be properly modularized and Gutenberg can move beyond the content blob.
  • Plugin and theme authors can ship solutions that only work for WordPress Classic or WordPress New removing the need to support both the new and the old editor.
  • PHP and other infrastructure version requirements can be updated without leaving people behind.
  • People who can't or don't want to upgrade can cleanly choose WordPress Classic as their solution.
  • If people try to install WordPress in an environment that does not support the new version, they can automatically be sent to WordPress Classic for the top-of-the-line experience we have today.

Drawbacks of a clean fork

There are significant drawbacks to forking WordPress as well:

  • WordPress would be forked, which splits the community.
  • Maintaining two versions requires twice as much work as maintaining one.
  • Eventually a fork will evolve on its own, feature freeze or not.
  • A fork provides "bad optics" as they say.

Dangers of not forking WordPress

There are significant risks to not forking WordPress, and at present we are headed straight toward all of them. Here's a small sample:

  • Not forking WordPress means the project needs to retain backwards compatibility. That means continued support for both the current (classic) editor and the ever evolving Gutenberg experience. This will hold back and eventually block future innovation and WordPress will become a monster even Dr. Frankenstein would shun.
  • The current approach to handling plugins and themes lacking Gutenberg support is to allow them to declare lack of support and force a fallback to the old editor. This will be used by plugin and theme developers who do not like Gutenberg to force WordPress to revert. Which in turn holds back future development. It also means plugin and theme developers have to ship code that supports both the classic editor and Gutenberg because any other theme or plugin may force WordPress to roll back. This issue was brought up by Kevin Hoffman brought this up during State of the Word, and the answer he got was, in my opinion, less than satisfactory (judge for yourself).
  • One of the major issues with WordPress is how it handles content. Presently all post content sits in the post table as a single blob which makes it impossible to effectively query data without querying the whole post content. To get around this problem theme and plugin developers use custom fields, but these just stash all the info in another table so we end up with two blobs. To truly meet the goals of Gutenberg, the blob needs to be broken up and the data structures of WordPress need to be rearchitected. If WordPress is not forked, this will not be possible and the current structures will eventually render the application obsolete.
  • Finally, if a clean fork is not created, parts of the community will shard off and force a fork. This will be damaging to the community as a whole because it will create an antagonistic relationship between those who support the "classic" fork and those who stay with the Gutenberg branch. With a clean fork, much of this animosity can be avoided or remedied, and we won't end up with a fractured community.

Your voice matters, and I want to hear it

Now that you know what I think needs to happen for this project to succeed, I want to know what you think. Like I said before, add your thoughts in the comments on this article, share them on social media using the #Gutenberg hashtag, publish your own articles, file issues and contribute to the project on GitHub, and keep the conversation going. I'm looking forward to having this conversation with you.

This article was cross-posted on LinkedIn Pulse.

Categories
gutenberg WordPress

WordPress is Changing. Here are 3 Things You Need to Know About Gutenberg

Everything you know about WordPress is about to change. The project codenamed "Gutenberg" is reimagining what content creation and editing on the web should look like and how it should work. Some time in 2018 the way WordPress works and the way you work with WordPress will change in a fundamental way, whether you are a visitor, a content creator, an administrator, a designer, or a developer.

Last weekend I had the privilege of presenting a talk titled "Gutenberg and the WordPress of Tomorrow" at WordCamp US in Nashville. You can view the video below and slides below. Once you're done, keep scrolling to see the three things you need to know as WordPress moves into the future.

1. Gutenberg is coming, fast. Be prepared.

If this is the first time you've heard about Gutenberg, this is the time to familiarize yourself with the project. The second half of Matt Mullenweg's State of the Word talk from WordCamp 2017 is a good place to start:

The project will probably reach maturity around April 2018 and I expect WordPress 5.0 will ship with this new interface to every WordPress site around June/July 2018. That means right now you have roughly 6 months to prepare.

Here are my recommendations:

  • Familiarize yourself with Gutenberg by installing the plugin on an experimental site.
  • Train your contributors / staff on content creation and management using Gutenberg.
  • Check with the developers of your themes and plugins that they are preparing for Gutenberg.
  • As the release gets closer, start testing your sites with Gutenberg to ensure everything works properly.

2. Gutenberg is not just an editor replacement, it changes everything!

The core concept of Gutenberg is every item you add to WordPress is a "block". Every heading, paragraph, image, blockquote, list, and other content you add is a block, and every block has unique properties and settings. That means when you create content, you can work with and customize each individual block, move those blocks around, and even make individual blocks reusable so you can build them once and use them in different locations and different views.

Notice how I said "views", not "posts and pages"? This is key: The current version of Gutenberg replaces the content editor. That's step one. Once it merges into WordPress proper, the project will expand its scope to encompass the Customizer, and eventually the way we create and mange content in WordPress will undergo a complete overhaul. Today we think of creating posts and pages. In the WordPress of Tomorrow, we will be making blocksand placing them in views. As weird as it sounds, the concept of post and pages is going away, and along with it the WordPress theme as we know it will likely fall to the wayside replaced by something all together different.

To wrap your head around this, imagine every piece of content you create as a literal building block and the environment (or view) the visitor accesses a space where you can place and organize those blocks in any combination and configuration you want. Then realize the current limitation of the browser viewport (the rectangular area within the screen you probably use to access the web) is going away thanks to everything from text-to-speech browsers to VR/AR/MR/XR.

Bottom line: The way we interact with content on the web is changing in a fundamental way and Gutenberg is what can bring WordPress into that future.

3. Gutenberg needs you!

Which brings me to the final point: Gutenberg is not ready, and the Gutenberg team needs your input and help to get the project and WordPress where it needs to go. Whether you are an absolute beginner and just started using WordPress yesterday or you have been contributing to WordPress core since the first line of code was written, your input is vital for the success of the project.

Here are some ways you can help:

These are exciting times and we are privileged to be part of a community that values every member and hears every voice. Your contributions matter. Make your voice heard!

This article was cross-posted on LinkedIn Pulse.

Categories
WordPress

The Case for Telemetry in WordPress

Update August 10, 2017: After publishing this article I reopened the Trac ticket arguing the reason for closing it was no longer valid in lieu of Gutenberg’s collection of telemetry. A few hours later it was announced that usage tracking will be removed from Gutenberg and the ticket was closed as the original argument for closing it is once again accurate.

This is an expanded version of a Twitter thread from earlier today.

#WordPress needs a core method for collecting quantitative user data through telemetry (aka “metrics”). I wrote a post about this on my blog back in December 2016 and filed a ticket on the WordPress development system Trac at the same time. That ticket was promptly closed by project lead Matt Mullenweg arguing “it is off the table for 2017 as it is not within the three focus areas.”

Since then, Gutenberg – the proposed new editor feature for WordPress – has introduced an “opt-in usage tracking system” (telemetry) meaning the argument that telemetry is not within the three focus areas (of which Gutenberg is one) is no longer valid.

One of the biggest challenges WordPress faces is the lack of reliable data about global day-to-day use. Like most Open Source projects, WordPress has relied on community feedback as it’s primary data source. Which is fine for a small project. Problem is WordPress is a Very Big Project with global reach and the majority of it’s users never interface with the community.

I like to say we, the people who talk about, provide feedback for, and design/develop WordPress are the 1%. I wrote about this on my blog in December 2015.

I now think the number is more like 0.1%.

Making decisions based on the traditional community feedback model is making decisions without knowing anything about the majority of users. Some will argue this is fine, that WordPress is developed by those who show up. That’s not a workable or responsible model for a project. It also goes against one of the core principles of WordPress itself, that “the core should provide features that 80% or more of end users will actually appreciate and use,” although the validity of the 80/20 rule was put into question by project lead Matt Mullenweg earlier this year (I wrote about this on my blog as well).

We, the people who build WordPress, have a duty of care to the people we build it for. And those people are not us.

“We can just do user testing,” you say? Sure. Let’s do proper qualitative user testing. That requires staffing, funding, and infrastructure. User testing for a project like WordPress is non-trivial. It requires professional analysis.

Testing one UX feature (like a view) would take 3 dedicated testers, at least 10 subjects, 3 weeks, and result in a >10 page report. User testing like this would be great, but it’s not something we are doing right now, and it’s not on the horizon either. Which brings me to telemetry and quantitative user metrics in WordPress.

Done responsibly and with care, telemetry can be a treasure trove of information for the evolution of WordPress. The key is collecting the RIGHT metrics using the right methodology. Telemetry data often ends up being a numbers soup because too much irrelevant data is collected. What I’m proposing is a lean and targeted approach to telemetry in WordPress:

  1. Telemetry should be an opt-in option that when activated installs a plugin. Admins should be informed about this option by the WordPress dashboard when WordPress is installed and reminded at regular intervals that the plugin is available and whether it’s activated.
  2. Anonymize all collected data at client level before submission.
  3. Collect only basic data at the core level of the plugin (WP/PHP/MySQL version, locale, language, etc.)
  4. Provide up-front info to end-users about what data is currently collected and what it’s being used for, with opt-out options for the plugin as a whole and for granular data collection.
  5. Allow for targeted data collection based on research needs.
  6. Store data on servers owned by the community (not corporate interests). Share data openly to ensure transparency.

Telemetry implemented in this way would give WordPress the ability to inform decisions about current and future features. Some, notably project lead Matt Mullenweg, have said this is not necessary, that it won’t be useful. I disagree.

In my view, making decisions that impact millions of users without metrics to back them up is irresponsible and quite frankly foolish. We run the risk of doing a bear’s favor: something we think will help that actually hurts, all because we don’t have enough information.

There are plenty of arguments against telemetry: anonymity, security, oversight, Big Brother, competitive advantage, etc. If we do this right I am certain we can build a system that alleviates the concerns over anonymity, surveillance, etc. Couple that with up-front disclosure, transparency, and explanation of what data is collected and why and people will sign on.

As for the competitive advantage aspect; we don’t want to share data with our competition; that runs counter to the Open Source idea in my opinion. We can and should share this data with everyone, because it’ll make the web a better place for everyone. It has purpose beyond WordPress. Not collecting this data because we don’t want competitors to have it is like leaving a broken window unmended lest it be broken again.

In short, WordPress needs telemetry. There’s a ticket on Trac proposing this, and Gutenberg has a PR for telemetry.

Inexplicably the Trac ticket is closed because “it is not within the three focus areas” which thanks to Gutenberg is not the case.

What WordPress needs is an open debate on this topic. What are the arguments for and against? What can be gained and what is lost? Should we do this? And if so, how do we do it in an open, transparent, and responsible way that helps inform and elevate the conversation while looking after the interests of all WordPress users?

This discussion belongs in Trac in an open ticket. Closing it down before a proper discussion has been allowed is not the Open Source way.

As of this writing the WordPress telemetry ticket remains closed: https://core.trac.wordpress.org/ticket/38418

Cross-posted from the original at LinkedIn.

Categories
My Opinion WordPress

The End of 80/20 and the Future of WordPress

History is filled with events that at the time seemed like footnotes but in hindsight revealed themselves as pivotal turning points. Such an event may have occurred Friday March 3rd, 2017. Buried in the comments section of a post on WPTavern a comment from WordPress co-creator and project lead Matt Mullenweg reads:

It might be time to retire 80/20 from the philosophy page, as it is seldom used as intended.

Below the surface of this short sentence lies a highly stressed fault line, and what Mullenweg seems to suggests is to deliberately trigger its release, causing a tectonic shift that will permanently change how WordPress is developed and as a consequence how content is published online.

For the record I do not know what Mullenweg thinks about the 80/20 rule beyond this single sentence. The following is my personal thoughts and reflections around the 80/20 rule. I am merely using his comment as a starting point for a discussion.

tl;dr: The 80/20 principle as applied in the context of current WordPress development is an ideal without a tether to reality. However much we say we develop the application for 80% of users, the reality is we know almost nothing about 99% of WordPress users. That means at best the rule is without consequence, at worst it is doing harm. The big question is how do we change our philosophy to solve this issue?

I have some thoughts, and I’d also like to hear yours. But first, a short primer:

What is the 80/20 rule, and why does it matter?

The 80/20 rule is a term you’ll hear a lot in business circles. It is called the Pareto Principle and goes as follows: “for many events, roughly 80% of the effects come from 20% of the causes” so for example “80% of your sales come from 20% of your clients.” This principle shows up everywhere. As an example Microsoft found that “fixing the top 20% of the most-reported bugs, 80% of the related errors and crashes in a given system would be eliminated.”

The Pareto Principle is also sometimes used in reverse, as is the case in the WordPress Philosophy (my highlighting):

“The rule of thumb is that the core should provide features that 80% or more of end users will actually appreciate and use. If the next version of WordPress comes with a feature that the majority of users immediately want to turn off, or think they’ll never use, then we’ve blown it. If we stick to the 80% principle then this should never happen.”

This version of the 80/20 principle is a good one, both from a development and a business standpoint: Build solutions that work for 80% of the user base, and let the remaining 20% find or build extensions to meet their needs.

The Challenge

As WordPress continues to grow in popularity and user base, changes made to the application and related services like the WordPress.org website are met with tighter scrutiny and critique. Many WordPress users are expressing a feeling of disconnect between themselves and the people who build the application, and changes are often criticized for being imposed in a “top-down” fashion without taking end-users into consideration. This is what the post on which Mullenweg left his comment was about. There are many reasons for this (most of which I will not get into here), including how the 80/20 principle is used when decisions are made.  

For the 80/20 principle as described in the WordPress Philosophy to work, the design and development team must have a firm fact-based understanding of the entire user base so they can identify the needs and abilities of the 80% and build solutions for them. That in turn requires data. The problem for WordPress is that data does not exist. As a result, any decision made under the banner of the 80/20 principle is actually a decision made based on the educated guesses of the development team and their immediate circle of contributors. This is even stipulated in the WordPress Philosophy (my highlighting):

“while we consider it really important to listen and respond to those who post feedback and voice their opinions on forums, they only represent a tiny fraction of our end users. When making decisions on how to move forward with future versions of WordPress, we look to engage more of those users who are not so vocal online.”

When Mullenweg says the 80/20 principle “is seldom used as intended” I am fairly certain this is what he’s referring to. Put bluntly, we can’t say we’re building solutions for the 80% because we don’t have the data to back that claim.

The Fork in the Road

When Mullenweg suggests removing the 80/20 principle from the WordPress Philosophy page, I think he is really suggesting replacing the 80/20 principle with something more practically feasible that moves the project forward. Lacking any further details, I’ll extrapolate the two possible paths I think this move could lead us down:  a Contributor-centric approach and a User-centric approach.

Contributor-centric

If we simply remove the 80/20 principle from the philosophy page, we will effectively end up where we are right now, with a Contributor-centric approach: WordPress is designed and developed by the people who contribute to the project, and user testing is done primarily on contributors and active community participants. This is already stipulated in the WordPress Philosophy:

“We do this (engage more of those users who are not so vocal online) by meeting and talking to users at WordCamps across the globe, this gives us a better balance of understanding and ultimately allows us to make better decisions for everyone moving forward.”

This Contributor-centric approach is common in open source communities, and has its origins in how open source projects come about: A small group of contributors build a solution for the contributors and anyone else interested. Even when the actual users outnumber the contributors, this dynamic remains because it’s what made the project successful.

It is important to understand the Contributor-centric approach is not based on a “we know what’s best for the user” attitude, rather “we know the application better than anyone, and we do what’s best for the application because we want it to grow and succeed.”

The overarching principle here is one of meritocracy: decisions are made by those who show up, and those who contribute the most have the most sway when decisions are made. This stems from the reality that open source contributors typically volunteer their time and are more likely to work on things they care deeply about. Allowing them to choose their own focus, and keeping a flat/no management structure is thought to ensure they have a continued vested interest in the project. In the background of all this there is also a real concern that limiting the autonomy of contributors may lead them to move on to other less restricted projects.

The Contributor-centric approach leaves itself open to critique that it is rooted in privilege: to have influence and “show up” requires availability of free time and a fairly high level of technical expertise, neither of which we can reasonably expect or demand from the average user. When the barrier to entry, or even constructive feedback, is high (as is the case with WordPress), the only voices heard are from the experts. The Contributor-centric approach solidifies this development model, and if that’s the path we choose to take, it needs to be explicitly stated.

User-centric

If we either commit to the spirit of the 80/20 principle or replace it with a similar principle, we are adopting a User-centric approach. User-centered design is a well-established method of iterative design based on extensive user testing and data gathering. From Wikipedia:

“user-centered design tries to optimize the product around how users can, want, or need to use the product, rather than forcing the users to change their behavior to accommodate the product”

The User-centric approach is what most major software and service vendors use to develop their products. It typically involves large-scale quantitative data gathering through telemetry, surveys, and other statistical methods as well as qualitative user testing on individuals and groups. A common argument against the user-centric approach is that it’s costly and slow, but in my experience the major challenge with it is that it tends to cause cognitive dissonance in stakeholders: User research often concludes that what designers and developers think is the best solution is not what the end-user prefers. This sometimes leads back to the “we know what’s best for the user” argument which is why some development teams cycle back and forth between the two approaches.

Another common critique of the User-centric approach is that the user does not always know what they want or need, and maybe more importantly, they don’t know what’s possible. The extreme version of this argument is Steve Jobs’ famous quote “A lot of times, people don’t know what they want until you show it to them.”

The User-centric approach requires rigorous discipline and careful management to work: Research and data gathering has to be unbiased and statistically sound and significant. At the same time, too much research and data, or research and data that has little value, can cause stagnation and confusion. And to top it all off, the User-centric approach will result in contributors having to work on projects they are not passionate about or even disagree with simply because they are not building solutions for themselves but the end-users.

In many ways the User-centric approach is a walk across a tightrope: Success is only achieved with training, planning, focus, and constant corrections. For the WordPress community, this would be a whole new way of doing things requiring a radical shift in attitudes.

The Duty of Care

Before we can make a decision about where to go next, we need to consider our Duty of Care: When we add, augment, or remove a feature in a tool used by someone else, we have a duty of care to that person to ensure your actions do not negatively impact them. The Decisions, Not Options principle comes into play here. From the WordPress Philosophy:

“Every time you give a user an option, you are asking them to make a decision. When a user doesn’t care or understand the option this ultimately leads to frustration.”

On the surface, the duty of care outlined here is to protect the user from confusion and frustration. Deeper down, we find the duty of care when making a decision on behalf of the user. The reasonable expectation from a user is that the application (and by extension its designers and developers), makes decisions that are in the best interest of the user.  

Which begs maybe the most important question:

How do we, the people who build WordPress, best meet our duty of care to the people who rely on WordPress in their personal lives, their communities, and their businesses?

Where do we go from here?

Whatever decision is made, it will cause a tectonic shift in how WordPress moves forward. That’s why every voice needs to be heard.

It is no secret that I land firmly on the side of the User-centric approach. From my experience user research is an integral part of any successful project, and the larger the user-base, the more important this research gets. I agree with Mullenweg that the 80/20 principle is not working, and I believe the solution is to realign the WordPress project to a more user-centric approach. This is not a quick and easy pivot, and it will require fundamental changes to how WordPress development is done on all levels. For one, we will need to gather large volumes of data on the existing user base and implement statistically sound methods for large-scale user testing of redesigns and new features. The first step in that process is to implement telemetry for the core application, a proposal that has already been shelved by Mullenweg.

Now you know what my answer is, but I’m just one of thousands of contributors, one of tens of thousands of active community participants, one of millions of WordPress users. I want to know what you think. Everyone who uses, interacts with, or contributes to WordPress has a stake in the project, and this is a decision that needs your voice. So, speak up and share your thoughts. I am listening, and I guarantee I’m not the only one.

—

Cross-posted on LinkedIn Pulse.

Categories
WordPress

The Case for WordPress Telemetry

WordPress prides itself on being an application built by the user for the user. The problem is with the popularity and reach of WordPress today, the distance between the WordPress 1% (or even .1%) and the average user is becoming so vast we (the people who contribute to WordPress core) know almost nothing about the actual people who use WordPress or how they use the application. This will become more of an issue as the application evolves, and it is high time we do something about it.

Lack of data means we’re flying blind

During the development of WordPress 4.7, I was involved in several conversations centered around assumed use of features. The general argument was that based on the 80/20 rule, certain features should be added while others should be removed. I kept brining up the well known fact we don’t have a clue what features 80%, or even 20%, of WordPress users actually use so any claim of validity in the 80/20 rule is guesswork at best, and in response one developer told me, point blank, “we know what the user wants.” I don’t know about you, but in my book that is not the way to build an application for real people.

What we need is raw data based on actual use, and lots of it. What we need, is telemetry. And there is ample industry precedence for collecting such telemetry, in online and offline applications and even in the WordPress ecosystem.

Here’s what I propose:

WordPress core should ship with an opt-in Telemetry feature that collects anonymized data on feature and functionality use.

This is in line with what major software providers do, and it is a feature most users will be familiar with.

The purpose of the Telemetry feature is to collect relevant data about how WordPress is used in the wild. This begs the questions “what is relevant data?”, “who decides what data is collected?”, and “who has access to the collected data?”

Here’s how I imagine it would work:

Implementation and activation

WordPress Telemetry is shipped as a core feature in new installs and an update to existing installs. When Telemetry is first added to the site, the admin gets a prompt asking if they want to contribute anonymized use data to the WordPress project. The default setting is “No” and the admin can change this to “Yes”. Once activated, the Telemetry setting can be changed at any time by the admin.

In more detail:

  • The opt-in selector for the feature should be surfaced on first install or when the site is updated to the first version of WordPress containing the feature is installed.
  • For new installs the opt-in question should appear on the 5-minute install page along with “Allow search engines to index the site” or similar.
  • For upgrades, the opt-in question should be revealed in a dedicated modal.
  • The feature should be disabled by default and the admin can make an active choice to participate.
  • The feature should be controllable at any time through a dedicated section under Settings->General
  • It is possible the best way to make users feel this feature is not a Trojan horse is to ship it as a plugin that auto-installs on opt-in and auto-uninstalls on opt-out.

Data collection

As a benchmark, some core data should always be collected, including but not limited to:

  • Number of themes and plugins installed
  • Frequency of use of specific views (Settings, Customizer, etc)
  • Current version
  • Update status
  • Locale (generalized to country)
  • Language
  • etc

In addition it should be possible to push custom queries to activated users to test for specific interactions, as an example how many users click the Underline button in TinyMCE. I’m not sure exactly what the best approach here is, but this is one idea: The feature queries a centralized service on a weekly / monthly basis to get instructions on what type of data is currently being collected.

The decision on what data to be collected should be done by committee based on current active tickets and features in development that require user data as well as longitudinal studies of user behavior.

Anonymity and transparency

A core requirement for the success of this feature is that data collection must be 100% anonymized. No data collected can be traced back to an individual user. Ideally the feature itself will be built in such a way that even accidental collection of personal data is impossible.

At any time, information about what data is being collected should be available to end-users both on a dedicated page on WordPress.org and through the setting in admin.

All data collected should be made public for scrutiny and use to ensure transparency and enable actual use.

Practical way forward

To prove the viability of this feature I propose a slow incremental deployment: Start with collection of certain uncontroversial datapoints like current language setting, number of themes and plugins, and one UI interaction that needs testing. Once this MVP has proven itself effective, a larger scale testing program can be shipped.

I’ve already created a ticket for this proposal on Trac and I’d love to hear your thoughts and ideas on the topic. To keep the conversation in one place I request that all comments are left on the Trac ticket. For this reason I’ve disabled comments here.

Update February 3, 2017

Matt closed the ticket shortly after an article published in WP Tavern brought it to the surface. I personally think that closure was premature and will continue pushing for the feature. In the meantime I’ve reopened comments here so you can voice your opinion on the matter.

 

Categories
Accessibility WordPress

Community Challenge: Let’s Caption All WordCamp Videos

WordPress.tv + Amara.org = captioned WordCamp videos

2015 was the year the WordPress community started taking web accessibility seriously, and both in WordPress Core and in themes, plugins, WordCamp talks, even WordCamps and WordPress Meetups, accessibility is becoming a first-level citizen.

As we start writing “2016”, let’s use this momentum to make information about WordPress more accessible to all.I challenge all WordCamp speakers to caption their WordCamp videos on WordPress.tv.

The information contained in the thousands of videos on WordPress.tv, from WordCamps, Meetups, and elsewhere, is invaluable to anyone wanting to learn about WordPress or wanting to expand their existing knowledge. Unfortunately, only a few of these videos have captioning because captioning takes time and effort.

If every community member who has had the privilege of speaking at an event and had their talk recorded and uploaded to WordPress.tv invested the time to caption their videos, either by doing it themselves or getting (paying?) someone else to do it, we would dramatically increase the accessibility of WordPress training materials.

In the process, we would also reap other benefits:

First of all, captioning is not just for the hard of hearing: Studies show a vast majority of TV and video viewers use closed captioning for increased comprehension. If you’ve watched a lot of WordPress.tv talks you’ll also know that the audio quality isn’t always the best, so captioning will help everyone.

Secondly, captioning your videos means others can translate them into other languages making them accessible to people who don’t speak the original language the video was recorded in.

Thirdly, I will put forward a proposal to publish the full transcripts of all WordPress.tv videos with the videos. This will allow visitors to make their own decisions about how to consume the content, and will allow search engines and other tools to index the contents of the videos properly.

Practicing What I Preach

To make sure I’m not making an impossible request, I have already started captioning my own WordPress.tv videos, and my goal is to have all of them captioned by the end of February (I’m realistic about my time).

The actual process of captioning a WordPress.tv video is relatively straight forward thanks to Amara.org. The full process is explained in the video below (a full rundown is also provided when you click the “Subtitle this video” link on each video page):

Captioning a 10 minute video took me about 60 minutes (mostly due to getting used to the interface and my severe dyslexia) and I expect once I get used to the tool it should take me about 2 hours to caption a 45 minute talk. In the grand scheme of things this is a minute investment to ensure more people can access (and possibly translate?) my talks, and it’s one I think all speakers should commit to.

Build Better Accessibility Together

If every one of us commits to captioning our own videos, the burden of what would otherwise be an insurmountable task becomes one that is shared in a fair and achievable way. If other community members pitch in, that task becomes even simpler.

By working together, we can make WordPress accessible, and part of this job is captioning each and every WordPress.tv video. We can do this, and we can do it well, so let’s get crackin’!