Formatting dates

[mjordan]

The current version of the Harvester (2.0.1) provides the following format templates strings in config.inc.php to pretty-print metadata values of type 'date' (as defined in the schema's plugin):

; Short and long date formats
date_format_trunc = "%m-%d"
date_format_short = "%Y-%m-%d"
date_format_long = "%B %e, %Y"
datetime_format_short = "%Y-%m-%d %I:%M %p"
datetime_format_long = "%B %e, %Y - %I:%M %p"

As I understand it, a metadata value that is of type date gets converted to a Unix-style timestamp for easy sorting, etc. This all works fine for date values that are harvested in YYYY-MM-DD format, since all the parsable segments can be mapped to the format templates.

For metadata values that do not conform to YYYY-MM-DD, the provided templates don't work well (for YYYY dates, for example, it appears that the values of the MM and DD on the day the metadata was harvested are added to the YYYY).

I've tried to add some additional date formatting templates to config.inc.php, but doing so also requires registering the new format in the TemplateManager.inc.php class file. An alternative, which I haven't tried because it is problematic for several reasons, is to convert the date's "raw" timestamp value into a date and check to see if it's just a year. This could be done in the schema plugin file or in the record.tpl and summary.tpl files themselves.

I think we can assume that the dates that are coming in are formatted in some way (and some metadata schemas allow attributes on date elements that indicate what encoding scheme is being used, not that it's trustworthy), but at a minimum we need a way to handle common date strings like YYYY and YYYY-MM for elemtents that describe publication dates, for example, in addition to YYYY-MM-DD.

Any suggestions on how to handle this problem?

[asmecher]

Hi Mark,

The first thing to check is how the dates are being stored in the database, and this will depend on the schema. If dates are being stored using a timestamp field in the database, there will be no indication of the "precision" of the date -- it will likely include defaults for the month and day, for example, if only the year is specified. If, however, the date is being stored as a string field, it'll be available to the template in its original precision (i.e. just as 2008), and it'll be easier to handle in the templates.

If you can check how the date is stored, I can provide more details.

Regards,
Alec Smecher
Public Knowledge Project Team

[mjordan]

In my case (i.e., using the MODS schema), assuming that the only copy of the values I am looking for are stored in entries.value, they are being stored as text (and looking at the values in that field confirm that they are stored as harvested).

In my original message, I think I was wrong about how the schema plugin defines the data type of an element -- I was looking at function getFieldType(), which defines the HTML form element type, not the database data type, correct?

[mjordan]

I've figured out my problem: in order to get the dates to display in the templates as stored, I removed selective calls to strtotime() in the MODS plugin file (i.e., where they were being processed for export to the templates), and removed 'date_format:$dateFormatShort' from the templates.

As a general approach to handling the formatting of metadata elements that would allow for overriding functions that currently exist in the schema *Plugin.inc.php files, could we consider using an external file containing 'preprocessor' functions similar to the ones used in other plugins? These functions, and the tokens in the templates, would use XPath-like expressions to identify specific element names (XPath since in MODS records, for example, you may want to format an element based on its context, e.g., abstract vs. relatedItem/abstract). These overrides should be applicable to a particular archive as well. The function signature of the preprocessor plugin function preprocessEntry() might serve as a model:

Code: Select all

function preprocessEntry(&$archive, &$record, &$field, &$value, &$attributes) {
                /*
                 * Add your regular expressions, and any conditional logic, here. You can use
                 * methods like $archive->getArchiveId() and $field->getName() in your code.
                 * This example removes periods from the ends of subject elements:
                 *
                 * if ($field->getName() == 'subject') {
                 *    $value = preg_replace('/\.$/', '', $value);
                 * }
                 */

                return false;
        }

Instead of expressions like '$field->getName() == 'subject'', we'd use the XPath-like expressions like '$field->getName() == "relatedItem/abstract"'. What I'm thinking is that these overrides could go into a file of a speicifc name, like template_preprocessor.inc.php, and if it's present, it is included and all entries run though it. On the template side, XPath expressions could be used as tokens so the template knows which overridden variables to use.

[mjordan]

I forgot to add, even though it might be obvious, that using XPath for overrides and tokens would let us access element attributes easily, which would be useful in an attribute-heavy schema like MODS.

[asmecher]

Hi Mark,

Excellent suggestions. I'm in the middle of writing a storage overhaul for the next development release of the harvester, which will permit a lot better smarts when it comes to record processing and display; this will give the schema plugins a lot more flexibility and permit things like using the Harvester as a data source.

Regards,
Alec Smecher
Public Knowledge Project Team