Introducing the Developers blog

September 15th, 2014 by  | 6 Comments

Hello all,

This is the first of what will hopefully become a series of posts about the technical aspects of PKP software, including tips and tricks for developers and system administrators. We hope it will be useful to anyone maintaining or customizing an OJS installation or interested in contributing. You will find all the developer blog posts gathered together on our new page for developers: https://pkp.sfu.ca/developers/

This entry will mostly be about some changes to how we use branches in github, starting with OJS 2.4.5.

The Problem

Best practices for hosting PKP software while keeping up to date with interim fixes have not been clear.

Historically, we have always maintained a page of “recommended patches” for each release of our software (e.g. the list for OJS 2.4.4-1). This can be used by a system administrator to keep on top of the common or criticial issues that we had fixed but not yet shipped in a subsequent release of the software. This list has been maintained by us manually and used sporadically in our own hosting operations.

This is useful but fairly time-consuming and error-prone. Patches needed to be applied manually, in the correct order, and it isn’t easy to keep an installation (never mind several hundred) up to date. When upgrading to a new release, it means manual review of the recommended patches that have been applied, which might be mixed in with other modifications.

Another option that we experimented with was to install OJS directly from the “ojs-stable-2_4” branch, which is actually the development branch from which 2.4.x releases are built. (Calling a development branch “stable” is a misnomer stemming from the fact that we call our 2.4.x line our “stable” line.) This works well when the ojs-stable-2_4 branch is fairly fresh from a release but in the middle of a development cycle, after new features have been added but before the testing phase works out the kinks, it can become unstable. Additionally, it’s not clear how to manage database changes since the upgrade scripts only handle upgrades between final releases gracefully.

The Solution

Starting with OJS 2.4.5, we will move all of the tracking of recommended patches into git, using some new branches.

The long-standing ojs-stable-2_4 branch will be renamed ojs-dev-2_4 to reflect its nature as a development branch, not a stable branch.

For each new release, we will create a specific “stable” branch — for OJS 2.4.5, this will be called ojs-stable-2_4_5. This branch should always be suitable for production purposes; it will receive stability fixes only, and nothing that will require database schema changes.

Our own hosting operations continue to make greater use of git, as it permits us to manage customizations. These proposed changes will further facilitate hosting with git — ensuring that an installation has all the most recent stability fixes will simply mean rebasing the modifications onto the end of the stable branch.

For users who are not hosting with git, we will continue to link from our wiki to a “recommended patches” page for each release — but in the future these will be generated directly from github.com, making sure they’re always current and applicable.

Future Steps

In our spring development sprint event, we spent a lot of time talking about automated testing. This work has continued over the summer; we have scripted a set of standardized test data and pushed more testing into the automated testing framework using Travis. Though work still continues, this has already resulted in better coverage than we’ve ever had before — including in particular PostgreSQL compatibility — and more immediate feedback when part of the test suite is broken by a buggy change.

With the OJS 2.4.5 release, a comprehensive structure for testing has been introduced and a test data set and the beginnings of a test suite have been written.

What this means is better, more consistent quality control. In addition to improving the software directly, it means we’ll have more options for our software release cycle — completing and testing a release will no longer be a 2-month process with considerable developer time spent on manual testing.

Feedback

We depend on a number of groups, particularly our Technical Committee, for guidance and feedback. We’re always happy to hear from the broader community. If there’s something we could do to further refine this process and make your life easier, let us know!

Regards,

Alec Smecher
PKP Technical Architect

Tags:

6 Comments on "Introducing the Developers blog"

  1. I also recommend for future work manage OJS plugin with composer, so updating will be easier or other mechanism for dependencies management.
    Greetings!

  2. Antonio Jesús Sánchez Padial says:

    BTW,

    What is the strategy to merge into stable from dev?

    Cherry-picking? packaging-releases?

    Best.

    • Alec Smecher says:

      Glad this makes sense! As for merging between “stable” branches (e.g. ojs-stable-2_4_5) and “dev” branches (e.g. ojs-dev-2_4), we’re still coming to a landing on exactly how much we’re going to merge into stable.

      So far I’d argue that we’ve merged too much — the “recommended patches” list as maintained by github is hard to work with. Plus the more we commit, the more likely we are to introduce additional bugs into what’s supposed to be a production-stability branch. (We’ll be using it ourselves for production work.)

      As we’ve done in the past, we’ll only build follow-up releases (e.g. OJS 2.4.5-1) from the stable branches when there’s a serious bug e.g. a security flaw.

      So for production use, I’d suggest depending on the stable branch (e.g. ojs-stable-2_4_5) and maybe expecting to cherry-pick some minor fixes in from development (e.g. ojs-dev-2_4) if you can’t wait until the next release.

      One long-term goal we’re working on is automated testing to decrease the human costs and time between stable releases. We’ve made good progress on this already and will continue to push it further.

      Feedback on all of this is very welcome!

  3. Antonio Jesús Sánchez Padial says:

    Best move!

    It took me a month to discover the changes. I couldn’t get it without reading this document (that @ctgraham gently pointed from PKP forum).

    I think it will help a lot to people who manage their OJS with git, and not only for fresh deployment but to keep it updated, and synced with our local modifications.

    Thanks,

  4. Adam Doan says:

    Sounds like a great solution. Not only will it make maintenance easier, it also brings OJS more inline with other projects. Hopefully this will help new developers feel at home when they want to contribute.

    Have you considered using something like git flow (http://nvie.com/posts/a-successful-git-branching-model/)? I’ve had great success using this model in my projects.

    • Alec Smecher says:

      Thanks, Adam! I’ve just flipped the switch on the branch changes — see the updated links in the post. This should make a lot of things easier and more predictable.

      The git flow model is very close to what we’re using; we use feature branches for both active development and hotfixes (resulting in PRs, both for the team and external contributors), and stable and development branches for them to merge into (not to mention master, which currently gets heavy hacking for OJS 3.0/OMP 1.2).

      Cheers — Alec

Comments are now closed.