Difference between revisions of "Installation: Multiple OJS & mOJO"

From PKP Wiki
Jump to: navigation, search
(Will Site Administration operations affect the rest of magazines?)
m (What is mOJO?)
(2 intermediate revisions by one user not shown)
Line 8: Line 8:
  
 
* Video: [http://www.youtube.com/watch?v=DSCQWYhhf9w Introduction & tutorial (15 minutes)]
 
* Video: [http://www.youtube.com/watch?v=DSCQWYhhf9w Introduction & tutorial (15 minutes)]
* Video: [https://www.youtube.com/watch?v=mL9m5EphDFs Seminar with Tech Committee (17 minutes)]
+
* Video: [http://www.youtube.com/watch?v=-u3ZM5Bb74g Seminar with Tech Committee (17 minutes)]
  
 
== The question ==
 
== The question ==
Line 108: Line 108:
 
* '''setdomain:'''          Recreate config files to let the magazine respond under a domain.
 
* '''setdomain:'''          Recreate config files to let the magazine respond under a domain.
 
* '''cleancache (cc):'''    Clean OJS Cache.
 
* '''cleancache (cc):'''    Clean OJS Cache.
 +
* '''sethome:'''            Replace index.php with an alternative page.
 +
**    |_ open:        Opens the magazine (revcvers OJS original index.php).
 +
**    |_ lock:        The magazine is Locked.
 +
**    |_ work:        The magazine is in Mantainance.
 +
* '''tools (t):'''          Call pkp-tools (see your OJS /tools folder)
 
* '''upgrade:'''            (ToDo) Replaces "current" links to "new" version links and upgrades ojs-DB.
 
* '''upgrade:'''            (ToDo) Replaces "current" links to "new" version links and upgrades ojs-DB.
 
* '''setup:'''              (ToDo) mOJO auto-installation.
 
* '''setup:'''              (ToDo) mOJO auto-installation.
  
The code is used daily and it's (and will be) Beta.
 
  
Code is published under GPL3 license in github that it's sometimes a little outdated (we will work to fix this).
+
The code is used daily and it's (and will be) Beta. Code is published under GPL3 license in github.
  
 
== Requirements ==
 
== Requirements ==

Revision as of 09:08, 25 November 2013

Introduction

This document aims to be useful for middle-big institutions that plan to install Open Journal Systems (OJS).

Here we describe, very briefly, the main concepts around "mOJO", a bash script to automatize common OJS tasks.

This document focuses in OJS but mOJO approach could be extended to any OxS application.

The question

When middle or big organizations plan to install an institutional OJS, the first question they need to face is: Are there any considerations in managing a large number of journals from a single OJS installation?

As far as PKP developments are able to do the job "out of the box", some organization decided that a single OJS installation is enough for all their magazines.

mOJO is designed for the opposite: To have each magazine in a separate OJS installation and facilitate the usual tasks over those publications.

Why multiple OJS installations?

A separate installation for each magazine (aka. multiOJS), have some benefits and some lacks. Notice that in some contexts a benefit could be considered a lack.

Advantages

"More flexibility"

  • Management:
    • Independent list of users for each magazine.
    • Plugins' system:
      • Install different plugins for each magazine.
      • Enable/disable different plugins for each magazine.
  • Theming:
    • Different templates for each magazine.
    • Different themes for each magazine.
  • Performance improvement:
    • Database level: Smaller DB (less tables).
    • Cache level: Independent cache folders (less files).
  • Escaliability:
    • Magazine granularity: You can "operate" at magazine level.
    • Script/DB/Files segmentation: You can balance your charge (even with different servers)
    • Different Apache settings for each magazine: multi-domains, multi-webusers, multi-caching...
  • Maintenance:
    • Keep different OJS versions for each magazine.
    • Independent Backup & Restore for each magazine.

Disadvantages

"More complexity"

  • Management:
    • More complexity: multiple OJS instead of a single OJS (mOJO helps)
    • Independent list of users for each magazine.
    • You can not "operate" over multiple magazines at same time (mOJO helps)
  • Presentation:
    • Catalog of magazines it's lost.
    • Global searcher (between magazines) it's useless.
  • Maintenance:
    • You need to configure/upgrade each magazine independently (mOJO helps)

What is mOJO?

mOJO is the acronym for "Multiple OJs Operations (aka. mojo)" and as described in the GitHub's repository [1] "it's a bash script to create multiple Open Journal Systems (OJS) installations and "rule them all".

Was developed to fill the "gap" between both installation approaches so helps administrators to deal with some of the listed disadvantages as well as offers new features as:

  • Create pre-configured magazines from a pre-formated base in seconds (including crontab and htaccess)
  • Multidomain management (with RESTful URLs)
  • Fast command-line operations against OJS: clean cache, upgrade... (To be extended)
  • Backup and Restore (all, code, data or db)
  • Staging: Easy migration between testing & production environments (ToDo)


This is the list of operations mOJO (version 0.25) is able to perform:

  • help (h): Script syntax.
  • list (l): List all the magazines of the service.
  • create: Create the folder structure of an ojs-magazine.
    • |_ files (cf): Create the folder structure of an ojs-magazine.
    • |_ db (cdb): Create the DB of an ojs-magazine from BASE template.
    • |_ all (call): Create a full ojs-magazine from the BASE template.
  • delete: Delete the folder structure and/or the DB of an ojs-magazine.
    • |_ files: Delete the code & of an ojs-magazine (first backups)
    • |_ db: Delete the DB of an specific ojs-magazine.
    • |_ all: COMPLETE removes of a ojs-magazine (scripts, files and DB)
  • backup (bck): Backup the files (code or-and data) and DB of an specific ojs-magazine.
    • |_ files: Backup code & webdata of an specific ojs-magazine.
    • |_ db: Backup the DB of an specific ojs-magazine.
    • |_ all: COMPLETE backup specific ojs-magazine (scripts, files and DB)
    • |_ code: Backup code of an specific ojs-magazine.
    • |_ data: Backup webdata of an specific ojs-magazine.
  • restore: Recover the files (code or-and data) of a formerly backup for ojs-magazine.
    • |_ files: Recover code & webdata of an specific ojs-magazine.
    • |_ db: Recover the DB of an specific ojs-magazine.
    • |_ all: COMPLETE recovery of an specific ojs-magazine (scripts, files and DB)
    • |_ code: Recover code of an specific ojs-magazine.
    • |_ data: Recover webdata of an specific ojs-magazine.
  • htaccess: Recreate the global htaccess file.
  • crontab: Recreate the global crontab file.
  • r-links: Recover symlinks for an specific site.
  • link2fold: Replace a symlink with the folder's content.
  • setdomain: Recreate config files to let the magazine respond under a domain.
  • cleancache (cc): Clean OJS Cache.
  • sethome: Replace index.php with an alternative page.
    • |_ open: Opens the magazine (revcvers OJS original index.php).
    • |_ lock: The magazine is Locked.
    • |_ work: The magazine is in Mantainance.
  • tools (t): Call pkp-tools (see your OJS /tools folder)
  • upgrade: (ToDo) Replaces "current" links to "new" version links and upgrades ojs-DB.
  • setup: (ToDo) mOJO auto-installation.


The code is used daily and it's (and will be) Beta. Code is published under GPL3 license in github.

Requirements

mOJO has some strong requirements at three levels:

  • Software:
    • GNU/Linux operating system
    • Apache
    • mySQL
    • Bash
  • Installation:
    • You need a very specific configuration (Folder structure and webuser) [2]
  • Usage:
    • It's not "foolproof": Some operations are harmful (pe: delete, r-links, restore) so you need to think what your are doing.

Instructions to setup your server: https://github.com/marcbria/mojo#installation

mOJO was developed and is tested against OJS 2.3.6 but it's expected to work with any 2.x version and will work with 3.x with minor changes.

The script could be (will be) extended to do better error checking, to work with other DB engines, Web Servers or migrated to other languages (PHP?) but as far as the whole system is based in symlinks mOJO will only work on UNIX operating systems.

Example of mOJO callings

  • mOJO usage:
   $ mojo
   $ mojo help


  • List all your magazines:
   $ mojo list


  • Create a "pre-configured" magazine:
   $ mojo create all magazineAlias
   $ mojo create all magazineAlias "editor@example.com"
   $ mojo create all magazineAlias "editor@example.com" "Real Name of the Editor"
   $ mojo create all magazineAlias "editor@example.com" "Real Name of the Editor" "Title of the Magazine"
   $ mojo create files magazineAlias           # files = code + data
   $ mojo create db magazineAlias "editor@example.com" "Real Name of the Editor" "Title of the Magazine"


  • Delete a magazine (preBackup will be done)
   $ mojo delete all magazineAlias
   $ mojo delete files magazineAlias
   $ mojo delete data magazineAlias


  • Backup a magazine (with checkPoint)
   $ mojo backup all magazineAlias true
   $ mojo backup code magazineAlias true
   $ mojo backup db magazineAlias
   $ mojo backup data magazineAlias true
   $ mojo backup files magazineAlias


  • Restore a magazine (to the "last" checkpoint)
   $ mojo restore all magazineAlias last
   $ mojo restore code magazineAlias last
   $ mojo restore db magazineAlias 20131114-4629
   $ mojo restore data magazineAlias last
   $ mojo restore files magazineAlias 20131114-1514


  • htaccess operations (aka. RESTFUL setup)
   $ mojo htaccess magazineAlias               # Creates an htaccess-chunk for the specified magazine.
   $ mojo htaccess                             # Regenerates the global htaccess from each htaccess-chunk.


  • crontab operations
   $ mojo crontab magazineAlias                # Creates an crontab-chunk for the specified magazine.
   $ mojo crontab                              # Regenerates the global cronMagazines.sh from each crontab-chunk.


  • Linking operations
   $ mojo r-links magazineAlias current        # Replace symlinks to a different version-folder (from ./source/versions)
   $ mojo link2fold magazineAlias templates    # Replaces a symlinked folder for it's physical content.


  • Homepage operations
   $ mojo sethome magazineAlias lock           # The magazine is Locked.
   $ mojo sethome magazineAlias open           # The magazine is in Mantainance.
   $ mojo sethome magazineAlias open           # Opens the magazine (recover OJS original index.php)


  • Domain operations
   $ mojo setdomain magazineName www.yourdomain.org        # Setups config.inc to work with a full domain name.
   $ mojo setdomain reset magazineName                     # Resets config.inc to subdomain settings.


  • Other operations
   $ mojo tools magazineName upgrade.php check             # Call pkp-tools (see your OJS /tools folder)
   $ mojo clearcache magazineName                          # Clears OJS cache.

How mOJO works

The key word here is "symbolic links". It could be summarized as "sharing code, but keeping different databases and data"

The following post there is a detailed explanation (as well as lots of posts about how to make it RESTful):

http://pkp.sfu.ca/support/forum/viewtopic.php?f=8&t=7578#p29284

Imagine that's what you need (a quite common structure in institutional contexts):


      http://example.com
                                   http://example.com/ojs1
       ----------------                                           http://example.com/magazine1
       |              |               -----------------
       |              |               |               |                ---------------
       |              |-------------> |    OJS 1      | -------------> |  Magazine 1 |
       |              |               |               |                ---------------
       |     CMS      |               -----------------
       |              |               -----------------            http://magazineN.foo.net
       |              |               |               |                ---------------
       |              |-------------> |    OJS n      | -------------> |  Magazine n |
       |              |               |               |                ---------------
       |              |               -----------------
       ----------------

In the root directory you will like to place a CMS to publish news and a list of all the magazines included in the system, whatever... but this is not our concern now (See it running at: http://revistes.uab.cat)

The interesting part is in each OJS sub-folder that will be a physical directory with symbolic links to a "base" ojs. What is the "base" ojs? It's the OJS code that will be shared between ALL journals.

Notice that not everything could be "symlinked": We need different index.php, config.inc.php and caches for each OJS installation. A few extra folders (ie: webdata) will be also specific for each OJS: files, public and registry. An .htaccess (in root folder) and/or vhost will do the redirection work.


   /home/ojs
   |__ backup
   |   |__ all
   |   |__ code
   |   |__ data
   |   |__ db
   |__ logs
   |__ scripts
   |   |__ [mojo.sh]
   |__ htdocs
   |   |__ (a few CMS files and folders)
   |   |__ ojs1
   |   |   |__ [index.php]
   |   |   |__ [config.inc.php]
   |   |   |__ [htaccess.chunk]
   |   |   |__ [cron.chunk]
   |   |   |__ public
   |   |   |__ cache
   |   |   |__ (the rest of ojs folders are linked to: /home/webuser/ojs/versions/current/*)
   |   |__ ojs2
   |   |   |__ [index.php]
   |   |   |__ [config.inc.php]
   |   |   |__ [htaccess.chunk]
   |   |   |__ [cron.chunk]
   |   |   |__ public
   |   |   |__ cache
   |   |   |__ (the rest of ojs folders are linked to: /home/webuser/ojs/versions/current/*)
   |   |__ ojs3
   |   |   |__ [index.php]
   |   |   |__ [config.inc.php]
   |   |   |__ [htaccess.chunk]
   |   |   |__ [cron.chunk]
   |   |   |__ public
   |   |   |__ cache
   |       |__ (the rest of ojs folders are linked to: /home/webuser/ojs/versions/new/*)
   |__ webdata
   |   |__ ojs1
   |   |   |__ files
   |   |   |__ public
   |   |   |__ registry
   |   |__ ojs2
   |   |   |__ files
   |   |   |__ public
   |   |   |__ registry
   |   |__ ojs3
   |       |__ files
   |       |__ public
   |       |__ registry
   |__ source
       |__ versions
       |   |__ current (-> ojs2.3.4)
       |   |__ ojs-2.3.4
       |   |__ ojs-2.3.3
       |   |__ ojs-2.4
       |   |__ new (-> ojs2.4)
       |__ templates
           |__ [baseMagazineDump.sql]
           |__ [config.inc.php.base]
           |__ [config.inc.php.Domain.base]
           |__ [crontab.base]
           |__ [crontabMagazine.base]
           |__ [htaccess.base]
           |__ [htaccessMagazine.base]
           |__ [htaccessMagazineDomain.base]
           |__ [createDB.sql]
           |__ [deleteDB.sql]


Keeping this structure up to date could be very boring and manual operations inevitably drive to errors, so we use mOJO to do the hard job.

The script generates the structure of a new OJS installation (web code & web data), and fills the database with a "preconfigurated OJS model" (the "base" one). The script also plays with string-replacement (sed) to substitute some tags with the appropriate information in config.inc.php, database dumps and so on... so when you do:

   $ mojo create all athenea juan.munoz@uab.es "Juan Muñoz Justicia" "Athenea Digital"

A new magazine is created and filled with the specified title, editor's name-mail, etc. in just seconds.

Comment: After we create the magazines, we meet the staff and we fill together OJS's configuration 5 steps to fit the magazine to their needs.

So mOJO helps in create/delete/backup/restore operations but also include a few snippets to work with symlinks, domains, crontab.

Notice here that as far as templates, plugins... folders are symlinked, you can replace any of those folders with physical folders and keep something a little bit different. This let us keep same code (except templates) for magazines as different as:

As said, mOJO was developed and tested against 2.3.6 but works with any 2.x (or any OJS version if the PHP code plays nicely with relative path instead of using absolute ones).

ToDo

This is what we have in mind, but we are open to suggestions and/or prioritizations:

  • [DEV] Request PKP a few minor changes in SessionManager class, bootstrap and config to suport multiOJS "out of the box" (without patching).
  • [DEV] Ask for confirmation in every potentially harmful operations.
  • [DEV] Autoinstall mOJO: Let the bash script build the whole system.
  • Info command: With status, versions, paths, plugins, users and other OJS basic info.
  • Update command: To update OJS DB&code (based on /tools)
  • CreateBase command: To generate a DB model for any future OJS.
  • ReplaceVar command: To gloably change one OJS variable.
  • ExecuteSQL command: To run a single query against all magazines.
  • PluginStatus command: To list/enable/disable plugin status.
  • Password command: To periodiaclly change admin password globally (all the magazines) or set different DB usr/pwd for each magazine.
  • Select command: To run comands against a set of magazines.
  • Mantainence: Play nicely with git. Even backup code in git.
  • Staging: Define different contexts and let mOJO move magazines from test to production (and vice versam)
  • [DEV] Verbose or progress bar for "slow" operations.
  • [POSTPONED] In discussion: Migration from bash to PHP (as far as OJS is PHP)
  • ...

Known issues?

Unable to login if your OJS code is not "patched"

Some code in OJS (plugins classes) use absolute paths instead of relative and it breaks the system. This is partially fixed but still under discussion with PKP.

Better error checking

Doing mOJO operations in damaged OJS (for instance, without DBs, webdata folders...) could result in tar or mySql warnings. Reading the warning will be enough to figure it out what happened (try to remove an non existing DB) and fix it, but would be nice to clean it up with better error checking.

Arguments

Right now, params need specific order, could do not include quotes, are not seriously checked... Till now, parameter handling was not a priority and it must be improved ("getopt" looks like the right candidate).

Further questions

Here you can find some questions that raised up during the presentation of mOJO to the PKP Technical Committee. The discussion made obvious we were thinking in a little different contexts: an "institutional" one, and a "hosting" service. We refer to both in the explanations.

Some of those questions need a deeper analysis or cross work between interested parts. Feel free to extend this list.

How to deal with PKP recommended patches?

This is not implemented yet but it's something we have in mind (and it's probably connected with the GIT question).

If is possible, in our "institutional" context we like to keep all the magazines in the same version (the last stable one). It means that we like to apply the same patches for ALL our magazines at the same time. It's usually a win-win: Editor's are happy because their magazines are "up to date" and we (sysadmins) are happy because the complexity is reduced.

As explained mOJO creates links for every OJS instance to a "base-version" of OJS code (in our case I have 3 versions running: "current" with 2.3.6, "new" with 2.4, "dev" with 3.0...)

If you don't have "file customizations" (aka. "forks" with different templates, plugins...) we have at least two easy solutions here:

  • A BASE magazine: We can keep a fake magazine that links to the OJS base-version (pe: "current") and you can nicely play "pkp tools/download patch" or whatever so (as symlinked) the change will be applied to ALL the magazines.
  • Directly patch the code: mOJO can patch directly the source/versions/current folder (including a dry-run)

I suppose would be nice to run a "php tools/update.php" patching so mOJO could also be extended to help with this and do the job in every magazine that links to this base-version. Something like "update all the magazines symlinked to current version and warn if a magazine include "customizations".

I'm not sure if this approach will be good enough in a "hosting" context with needs as the one explained in the next question. Please, let us known.

Is it possible to keep different permissions?

I understand this question is not related with OS file permissions it's more about user permissions and apache security modules (as suExec, suPHP...)

In our installation we use "ojs:www-data" and as far as "mojo" is (and will be) a sysadmin script runs as root, permissions is not a big concern for us. Please, comment if you disagree.

Any way, some security modules will complain if the destination of a symlink is not in web folder (public_html...), users don't fit or file permissions are too permissive (777), etc. More than this, in a "hosting" context you will also like to offer ssh/FTP to your users, "chrooted" spaces, etc.

In this sense, the "hosting" scenario could be very variable and looks a little far from the "controlled" context we are building so we will need to detail to see if mOJO can fit.

Any case, in this wide context, seams that mOJO will need to be extend to:

  • keep permissions&users up to date.
  • build a different folder structure (place code in public_html...)
  • keep the destination of the symlinks open

Any way, as symlinks are against the security model those tools like, we will probably need to play a little with them (pe: Forcing apache 2.2.15 and suexec to allow symlinks)

Will Site Administration operations affect the rest of magazines?

Yes it is, and this is why we don't offer "site admin" permissions to our magazines.

There is an extra issue here with "plugins installation", as far as is an magazine level operation that work over a shared folder (plugins).

In practice it's not very usual to see magazine editors installing plugins, but it's something to fix in a "hosting" context.

We plan to patch OJS code to include a variable (in config or settings) to lock down those sensitive operations when you are running in multiOJS mode.

How to integrate mOJO with GIT?

It's one of the biggest questions to solve right now.

As far as we only 3 magazines with code customizations we were not very worried about this, but we like to migrate to OJS 2.4 soon and GIT looks like the way to do the job smoothly.

Last PKP-OJS GIT guide will be really useful here but if you have been working in it and you have any suggestion please let us know.

References

And the code is in github:

[1] https://github.com/marcbria/mojo

Documentation about how to install mOJO could be found here:

[2] https://github.com/marcbria/mojo#installation

All the development process was discussed in this thread:

http://pkp.sfu.ca/support/forum/viewtopic.php?f=8&t=7578