Difference between revisions of "Github Documentation for PKP Contributors"

From PKP Wiki
Jump to: navigation, search
(Add section on installing node dependencies)
(Added plugin-based dependencies)
Line 135: Line 135:
  composer update
  composer update
  cd ../..
  cd ../..
cd plugins/paymethod/paypal
composer update
cd ../../..
cd plugins/generic/citationStyleLanguage
composer update
cd ../../..
All done!
All done!

Revision as of 11:49, 16 November 2017

Github Documentation for PKP Contributors

Cheat Sheet

To clone one of your git repos locally:

git clone git@github.com:your-username/ojs.git my-ojs-clone

Or (to avoid SSH key issues):

 git clone https://github.com/your-username/ojs.git my-ojs-clone

To check out a new branch based on the repo's 2.4 development branch (this branch would be named "contribution-name"):

git checkout -b contribution-name origin/ojs-dev-2_4

To configure submodules in your repo (may require git version > 1.8):

git submodule update --init --recursive

Install Composer dependencies:

cd lib/pkp
composer update
cd ../..

To pull in code from another remote repo (eg. a developer's remote branch):

git remote add jmacgreg git://github.com/jmacgreg/ojs.git // adds jmacgreg's remote git repo
git fetch jmacgreg // retrieves all of jmacgreg's remote branches, commits, etc. 
git cherry-pick 8db6deb // retrieves the commit with that particular hash tag and merges it


The Public Knowledge Project uses Github to make several repositories available to interested parties who wish to contribute. The repositories are publicly accessible, but if you would like to contribute modifications or enhancements, you will need to configure your own copies of the official repositories. The typical life cycle of a contribution is this:

  1. Something is identified, which needs to be fixed, or contributed (in the case of a new feature or enhancement).
  2. The modifications are made to your own copy of our repositories, and tested to make sure that things work as they should.
  3. A request is issued to us, so that we then become aware of what you’d like to contribute.
  4. Someone from the PKP development team will review your work, and either merge it into our own repositories, or comment on your contribution and possibly suggest enhancements or minor modifications before it is accepted.
  5. Once accepted, the request is closed. The cycle can then begin again with a new code enhancement.

This document will attempt to outline the necessary steps in order to achieve the scenario described above.

All of the PKP repositories can be found from the organization page on Github, available at https://github.com/pkp

As you will see, each project (OJS, OMP, OCS, Harvester) is listed, with links to those individual repositories. In addition, there is a repository called “PKP-lib” which contains the shared library code used by all of the PKP software products. To obtain a working installation of any of the PKP products, you will need both the repository for the product and also the PKP-lib repository. The PKP-lib repository will be configured as a sub module. This document will explain how to do that.

Create your Github Account and fork the repositories

We shall use Open Journal Systems, as an example, in this document but the procedure is the same for any of the products. In order to create your own ‘fork’ of the official OJS repository, you will first need to register for a free Github account. Once you have registered and signed into your account, navigate to the repository you wish to fork. For the OJS repository, this would be https://github.com/pkp/ojs

In the top right of the screen, click the button labelled ‘fork’. The process will take just a moment to run, and when it is completed, you will be redirected to the repository page for your own fork.


Repeat the same process for the PKP-lib repository, at https://github.com/pkp/pkp-lib. You will need both repositories in order to run a working installation of any of the PKP software products.

Once you have done this, you should see two repositories listed on your Github home page.


Configure your local development environment

At this point, we will need to configure your local development environment to use these two forked repositories. This document assumes that you have access to a command line shell environment (e.g. a terminal window on Linux, or an application like iTerm under Mac OS X), which we shall be using for the rest of this exercise. You will need to be be familiar with a command line text editor and basic shell commands in order to get the most out of this document. For the remainder of this document, commands that are meant to be typed will be shown in a monospaced font, and preceded by a dollar sign ($). This dollar sign represents your command prompt and is not meant to be typed when you are reproducing the commands.

First, you will need the Git application for your development environment. Git is available for many different platforms. Please download and install the most suitable one: http://git-scm.com/downloads

Once Git is installed, you will need to configure it. There are two variables that should be set, in order to let Github correctly tag commits that you make to your own repositories. Open your terminal program and run the following two commands, replacing the necessary information with your own.

$ git config --global user.name "Your Name Here"
$ git config --global user.email "your_email@example.com"

Clone the PKP Repositories

Once this is done, it is time to clone the main repository representing the project you wish to contribute to. You will need the URL to your repository. To get this URL, visit the home page for your cloned repository on the Github website and look on the right.

Click the icon next to the text field to copy the URL to your clipboard.


Return to your terminal window in your local development environment. Change to the directory that you wish to clone your repository in, and use the ‘git clone’ command to clone the repository, replacing ‘your-username’ with your Github username.

$ git clone git@github.com:your-username/ojs.git my-ojs-clone

The example above uses ‘git@’ URLs to clone the repositories, but these URLs do depend on having your command line environment configured to use SSH keys. Github supports several different styles of URLs, and if you encounter errors when running the above command, you should consult the informative help section on the Github website for further guidance, at https://help.github.com/articles/which-remote-url-should-i-use.

If you wish to use ‘git@’ URLs and you’d like more information in configuring SSH keys, please read the SSH help section on the Github website.

If you get "permission issues" try with the "https" protocol as descrived here: https://help.github.com/articles/fork-a-repo

The cloning process may take a few minutes to run, depending on the speed of your network connection. You will see something similar to the following output:

Cloning into 'my-ojs-clone'...
remote: Counting objects: 192852, done.
remote: Compressing objects: 100% (47019/47019), done.
remote: Total 192852 (delta 135066), reused 179930 (delta 122572)
Receiving objects: 100% (192852/192852), 49.85 MiB | 104 KiB/s, done.
Resolving deltas: 100% (135066/135066), done.
Checking out files: 100% (3152/3152), done.

Once the process completes, you will have a new directory called ‘my-ojs-clone’. If you navigate into this directory, you will see that it contains all of the files contained in the Github repository.

$ cd my-ojs-clone/
$ ls
cache        controllers    favicon.ico    lib    plugins
robots.txt        templates    classes    dbscripts    index.php
locale    public    rt    tests        config.TEMPLATE.inc.php
docs        js

Checking out the branch you want to work on

If all you want to work on is the "master" branch of the application you just checked out, you are ready to go at this point. If you are working on a stable branch of the software (for example OJS 2.4.x, or OCS 2.3.x) you will want to checkout a new dev based on the dev branch of that software. For OJS the command would be:

git checkout -b analytics origin/ojs-dev-2_4

This would create a new "analytics" branch in your local repo (named as such because you are presumably working on some analytics code - the name of the branch can be whatever you want). When you push this up to your remote repository, this branch and its committed code will be pushed up and available to submit a pull request from.

Configure the PKP-lib Sub-modules

We are almost ready to begin work. At this point, we must now configure the sub-modules for the PKP-lib repository.

All you have to do is the following one-liner to recursively initialize and update all submodules:

git submodule update --init --recursive

Install Composer dependencies

Ensure that your server has Composer installed. Composer can be used to manage and install dependencies, which are listed in lib/pkp/composer.json.

To install the dependencies, run...

cd lib/pkp
composer update
cd ../..
cd plugins/paymethod/paypal
composer update
cd ../../..
cd plugins/generic/citationStyleLanguage
composer update
cd ../../..

All done!

Install Node dependencies

Ensure that your server has Node.js installed. Node.js is used to manage and install JavaScript dependencies, which are listed in package.json.

To install the dependencies, run...

npm install

To build the JavaScript package, run...

npm run build

If you plan to work on the JavaScript, and want the files to be rebuilt automatically, run...

npm run dev

Contributing your work back to PKP

At this point, you are free to make changes to the code present in either of the two repositories that you have cloned into your local development environment. Since both of these repositories point to your own forks on Github, you can safely work knowing that your code will not affect the official repositories.

As you work, git provides several useful commands to track your progress. Since you are working off of your own forked repository, you can be relatively certain that there will be no upstream changes when you are ready to commit (unless you are sharing your repository with another collaborator). The typical lifecycle for a change may look like this:

Use the ‘git status’ command to see what files have been modified (or added) to a repository. Since you have two repositories (OJS and PKP-lib), this command will have to be run in both the OJS directory, and in the lib/pkp directory. Running this command may produce the following, if a single file has been changed.

$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#    modified:   .gitmodules
#    modified:   templates/common/header.tpl
no changes added to commit (use "git add" and/or "git commit -a")

As you can see, there are two modified files. The first one, .gitmodules, is marked as modified because we had to make a change to it in order to point it to our correct sub-module. This can be ignored. The second file, “templates/common/header.tpl”, contains a trivial modification. We can see this modification using the ‘git diff’ command.

$ git diff templates/common/header.tpl
diff --git a/templates/common/header.tpl b/templates/common/header.tpl
index 81c4330..3858836 100644
--- a/templates/common/header.tpl
+++ b/templates/common/header.tpl
@@ -5,6 +5,7 @@
 * Distributed under the GNU GPL v2. For full terms see the file docs/COPYING.
- * Common site header.
+ * This is a Modification by Me!
{capture assign="deprecatedThemeStyles"}

Git will mark changed lines with either ‘+’ or ‘-’, to indicate whether they have been added or removed. It will also include some other lines around the changed ones to provide you with a bit of context. By default, git will not show changed lines in different colors, but if you would like this to be the case you can run the following command:

$ git config --global --add color.ui true

Committing your changes to your repository

If you are finished with your modifications at this point, you may commit them to your forked repository. Once they are committed, you may use Github’s ‘pull request’ feature to let the PKP development team that you have modifications that you’d like them to review.

To commit your changes, use the ‘git add’ command.

$ git add templates/common/header.tpl

You may use wildcards or directory paths if you wish to add many files at once. Once this is done, confirm that things have been added by re-running the ‘git status’ command.

$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#    modified:   templates/common/header.tpl

To commit your changes, use the ‘git commit’ command. This will commit your changes locally at first, and let you attach a message to this commit.

$ git commit -m "my initial commit"
[master 4d4c132] my initial commit
 1 file changed, 1 insertion(+), 1 deletion(-)

At this point, your change is committed to your local (on your computer) repository, but does not exist on the Github website. You must now push your change to your remote repository.

$ git push
Counting objects: 9, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 448 bytes, done.
Total 5 (delta 4), reused 0 (delta 0)
To git@github.com:your-username/ojs.git
   f917f19..4d4c132  master -> master

The important part of the status message is in bold. It indicates that you have pushed your changes to your own fork of the OJS repository, to the master branch. At this point, if you view your forked repository on github, you will see the new commit.


PKP library submodule changes

If you changed something in library, you will also need to commit the library submodule changes in the application repository. This is necessary especially for our continuous integration structure. Basically, the submodule changes commit will tell anyone that pulls the application code where the submodule repository HEAD is, including the CI script.

This is needed because the CI script is prepared to get the library from PKP's official repository. In case of a pull request from a developer, we need to tell the CI script to also update the library with commits from the developer's library repository. That way the CI tool can correctly build and test your pull request code before merging. To make that happen, do the following:

  • create your commits normally, both for application and library;
  • if you have library commits, commit the submodule changes, at the application repository, in an UNIQUE separate commit, making sure that it is the HEAD:
git add lib/pkp
git commit -m 'pkp/pkp-lib#1234 Any message here ##githubUser/myBranch##'

1234 is the issue id from https://github.com/pkp/pkp-lib/issues; githubUser is your github user; myBranch is the branch in your github repository where the library commits are;

  • make sure your local application and library repositories are updated with the official, as it is expected if you want to do a pull request;
  • push both library and application commits to your repository in github;

After doing this, you can create a pull request using the GitHub interface, and check the CI build result. If it doesn't pass, you can read the logs, see the errors, fix the code and update your pull request by pushing your code to your own repository again.

Sending a Pull Request to PKP

At this point, the work is essentially finished. Had there also been changes made to the pkp-lib repository, the process would have been the same, except that the commands would have been run inside of the lib/pkp directory.

If you wish to submit your modifications, to PKP for review, you must now create a pull request. To do this, click on “pull requests” to the right of your forked repository page in Github.


This will load a new screen containing all of the pull requests for your project. Click the green “New Pull Request” button on the right to begin the process.


The next screen will contain detailed information about your current commit, and will include a link that can be clicked to initiate the request. Click the link.

At this point, you are finished. The PKP team will receive a notification that you have submitted a pull request. They will review your modifications, and possibly comment on your work, request small revisions, or merge it into the main PKP repositories outright.

Keeping your fork in sync with PKP

Github has become very good at automatically merging pull requests if it is possible. However, since you are not working in a vacuum and the PKP development team is continuing to contribute to the original fork, you may periodically want to pull any recent changes made to the official repositories into your own forked repository. This is done by creating a new ‘remote’ on your local development environment which is set up to point to the original PKP repository. You can then periodically fetch new commits from this remote and merge them into your local fork. Please be careful when doing this, since there may be occasions when changes made to the official repository will conflict with changes you have made to your own fork. An example of this would be when both repositories have changes made to the same file. Conflicts like this will need to be manually merged.

To set up a remote which tracks the official PKP repository, again using OJS as an example, please type the following command.

$ git remote add upstream https://github.com/pkp/ojs.git

You can then verify that this remote is now set up by running:

$ git remote -v
origin    https://github.com/your_username/ojs.git (fetch)
origin    https://github.com/your_username/ojs.git (push)
upstream  https://github.com/pkp/ojs.git (fetch)

This indicates that there are two remotes, one called origin and the other called upstream. The former is your own fork since it points to your own account on Github. The latter, which only has fetch access, points to PKP’s OJS repository.

To sync changes made to the official repository into your own fork, you must first fetch the upstream repository with:

$ git fetch upstream

Which will generate output similar to the following:

remote: Counting objects: 75, done.
# remote: Compressing objects: 100% (53/53), done.
# remote: Total 62 (delta 27), reused 44 (delta 9)
# Unpacking objects: 100% (62/62), done.
# From https://github.com/pkp/ojs
# * [new branch]      master     -> upstream/master

Please note that the command has fetched content from the official PKP repository, not your own, and has stored the content in a new branch called ‘upstream/master’.

Merging any new changes into your own fork is a two step process. First, we must change to the ‘master’ branch of your own fork:

$ git checkout master

And then we must merge in changes from the ‘upstream/master’ branch:

$ git merge upstream/master

Which should generate output similar to the following:

Updating a422352..5fdff0f
# Fast-forward
# README                    |    9 -------
# README.md                 |    7 ++++++
# 2 files changed, 7 insertions(+), 9 deletions(-)

It is possible that much more content will be displayed if your own fork is significantly out of date. If you have local changes that may affect the merge process, you can ‘stash’ them before doing the merge, by running the following command right after you change to your own master branch:

$ git stash

You may re-apply your changes to the updated fork by running the following command *after* the merge is complete:

$ git stash apply

Again, be aware that if the the merge has significantly changed content in the repository, your stashed changes may not re-apply cleanly. In such instances, git will provide you with suggestions for resolving these conflicts.

For a tutorial on resolving Github conflicts, please visit https://help.github.com/articles/resolving-a-merge-conflict-from-the-command-line

Tips and tricks

Dealing with permissions

If you plan to change your file permissions and/or ownership, and you want git to ignore those changes, you will like to:

$ git config core.fileMode false

Untracking files or folders

If you just don't want to track a file or a folder, but you want to keep the file in the remote repository (non-bare), you can use the following command:

$ git update-index --assume-unchanged  <filename or folder>

After this, git stops checking the file for possible modifications.

At any time, you can track again by setting "--no-assume-unchaged" flag

$ git update-index --no-assume-unchanged  <filename>

These command do not affect the remote repository.

More info: https://help.github.com/articles/ignoring-files

Dealing with submodules when changing branches

You may run into a situation where you try to change branches from eg. ojs-master to ojs-stable-2_4_5, and run into the following error (or something similar):

error: Untracked working tree file 'plugins/generic/customBlockManager/CustomBlockManagerPlugin.inc.php' would be overwritten by merge.

The problem is that when you originally checked out the master branch and ran a 'git submodule update', you checked out Custom Block Manager Plugin submodule code to plugins/generic; but the ojs-stable-2_4_5 branch includes code there, not as a submodule but as actual code in the repo, and this results in a write conflict when trying to switch.

To resolve this issue, you can delete the plugins/generic/customBlockManager folder and try to checkout the stable branch again. If you need to change back to master, delete the same folder and do a git checkout master and a git submodule update --init --recursive, so all submodules will be updated and in place again.

Of course, only delete the folder if you didn't change anything there. If you made any changes, you first need to fork the custom block manager repository, commit the change, send to your repo, and then you can delete.

Useful Resources