When we started working on

Automattic’s new experimentation platform (ExPlat)

, it became clear that we needed to agree on the development practices for the project. Achieving consensus was critical, given

Automattic’s fractal nature

and

its strong emphasis on autonomy

. In practice, these attributes lead to a high variability of project development cultures across the company, depending on specific project needs and individual backgrounds. As the tech lead for the project, I proposed some principles and practices on

our P2 workspace

, which led to discussion and iteration over time. We’re sharing the most recent version of the document here with minimal edits, in the hope that it will be useful for others who are working on similar projects.

Top principle: we’re building for maintenance

We’re building a robust new system, not another quick-and-dirty library. A maintenance mindset will help ensure that ExPlat is built to last, and that it doesn’t become

yet another legacy experimentation solution at Automattic

. The following principles and practices derive from this mindset.

Production-ready code must have full test coverage

We mostly work with dynamic languages. The only way to be confident that code actually runs is to run it. In addition, full test coverage makes refactoring easier and serves as documentation. When it comes to the full coverage rule, we should only make exceptions around experimental code that is likely to change. However, such code should be clearly marked as experimental.

New code must adhere to the coding conventions of its environment

This means no phpcs warnings in PHP, full linting in Python, small maintainable queries in

SQLT

, a build that fully passes in

Abacus

, etc. Ideally, we will add new conventions where there are gaps.

All public-facing functionality must be fully documented

Writing documentation can be a pain, but trying to guess what undocumented code does can be worse. If code is meant to be used outside the module in which it’s defined, it should be fully documented. This includes typed input arguments, typed return values, English descriptions, usage examples for non-trivial functionality, and P2 posts or wiki pages for additional detail when needed.

All production code must be reviewed before it’s committed

We’ve adopted

Google’s code review standards

and recommend reviewing the resources on

how to do a code review

and

how to author changes

. All changes should get their initial review within 24 hours from the time the author says that they’re ready for review. Emergency patches and experimental (i.e., non-production) code may be reviewed after they’re committed.

Strive for small, simple, and descriptive changes

It’s better to err on the side of implementing changes that are too small — such simplicity makes changes easy to approve and merge (see also

Google’s guide to small changelists

). The code author should provide context, links, and inline comments to help communicate the background and intent of a change, especially since the reviewer may not be online at the same time as the author to ask questions synchronously. Changes will be considered a work-in-progress until the build passes, unless the author has clearly communicated the reasons why a build fails (e.g., temporary issues with external dependencies).

Address all review comments

Address all review comments by making a code change, replying, marking as done, or responding with an emoji reaction. Communicate that comments have been received or the intent to address with code changes. This may include prompts for clarity or discussions. Effective and timely communication is key, in line with

the general expectations from all Automatticians

.

Fix bugs before writing new code

Joel Spolsky said it many years ago

, and we believe it’s still applicable today. In addition to making extensions to our code easier, keeping the bug count at zero should also increase user confidence in our systems.

Don’t wait for hack weeks to refactor messy code

Requirements change and evolve, so it’s inevitable that once-clean code might need to be rewritten to accommodate new use cases and a better understanding of the problem space. To avoid ending up with a pile of unmaintainable code, we should refactor as we go between work on functional changes.

Save time by spending time on design and architecture

It’s common for developers to have a bias toward writing code rather than thoroughly discussing requirements. However, for non-trivial changes, it is often best to get some consensus within the team before jumping in to write code. In fact, some pre-coding discussions may save time on code reviews.

Sound interesting? Join us!

We’re currently hiring

a Senior Experimentation Backend Engineer

. Other positions on the team may open in the future. If you’re reading this at a later time, it’s likely that

Automattic’s Work With Us page

still contains dozens of positions with the company: work from anywhere and help make the web a better place!

Posted in

Experimentation Platform



Tagged

a/b testing

,

Automattic

,

coding conventions

,

software engineering

,

software testing