We are moving to Git Issues for bug tracking in future releases. During transition, content will be in both tools. If you'd like to file a new bug, please create an issue.

Bug 6876 - Galleys uploaded in Firefox not indexed for search
Galleys uploaded in Firefox not indexed for search
Status: RESOLVED FIXED
Product: OCS
Classification: Unclassified
Component: General
2.3.4
All All
: P3 normal
Assigned To: Michael Felczak
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-09-07 11:45 PDT by Michael Felczak
Modified: 2014-10-07 12:38 PDT (History)
2 users (show)

See Also:
Version Reported In: 2.3.3
Also Affects: OHS 2.3.2, OJS 2.3.7


Attachments
Patch against OCS 2.3.3-1 (536 bytes, patch)
2011-09-07 11:49 PDT, Michael Felczak
Details | Diff
Patch against OJS 2.3.6 (1.17 KB, patch)
2011-10-11 22:11 PDT, Michael Felczak
Details | Diff
OCS MySQL database update (867 bytes, text/plain)
2011-10-11 22:21 PDT, Michael Felczak
Details
OCS PostgreSQL database update (832 bytes, text/plain)
2011-10-11 22:22 PDT, Michael Felczak
Details
OJS MySQL database update (881 bytes, text/plain)
2011-10-11 22:23 PDT, Michael Felczak
Details
OJS PostgreSQL database update (846 bytes, text/plain)
2011-10-11 22:23 PDT, Michael Felczak
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Felczak 2011-09-07 11:45:28 PDT
Galley upload form does not report correct file type for PDFs.

External indexing app never invoked and galley remains unindexed.
Comment 1 Michael Felczak 2011-09-07 11:49:53 PDT
Created attachment 3633 [details]
Patch against OCS 2.3.3-1
Comment 2 Michael Felczak 2011-09-07 11:50:02 PDT
PDF galleys not indexed for search
https://github.com/pkp/ocs/commit/1bae4b11b82ec2983fba3ee4dd6610da773f9232
Comment 3 Michael Felczak 2011-09-07 12:50:02 PDT
PDF galleys not indexed for search
https://github.com/pkp/ocs/commit/ee865293da3790e14ca773e8bf30a6bb9adc62e4
Comment 4 Michael Felczak 2011-09-07 13:21:26 PDT
Looking into this a bit more it appears to be a browser issue.

Firefox (OSX, 5.0) reports the file type as application/download whereas other browsers correctly identify the file as application/pdf

We really shouldn't rely on the browser for this info, i.e.

$_FILES['userfile']['type']

    The mime type of the file, if the browser provided this information. An example would be "image/gif". This mime type is however not checked on the PHP side and therefore don't take its value for granted.

This should be fixed in all apps to use mime_content_type on the server.
Comment 5 Michael Felczak 2011-09-07 22:08:39 PDT
This problem applies to other versions of Firefox on other platforms as well:

https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/84880
http://forums.mozillazine.org/viewtopic.php?f=9&t=2241011
Comment 6 Michael Felczak 2011-09-08 09:33:52 PDT
Good explanation of the issue:

http://techblog.procurios.nl/k/n618/news/view/15872/14863/Mimetype-corruption-in-Firefox.html
Comment 7 Michael Felczak 2011-09-09 12:10:52 PDT
Looking at some real world data, this issue also applies to MS Word and PowerPoint galleys.

Rather than application/msword and application/vnd.ms-powerpoint these are sometimes saved as application/octet-stream and will not be indexed.

Need to manually fix these to the correct mime types.

.docx should be saved as
application/vnd.openxmlformats-officedocument.wordprocessingml.document

This apparently supersedes application/msword
Comment 8 Michael Felczak 2011-09-09 12:19:09 PDT
Need to manually fix:

* doc: application/msword
* ppt: application/vnd.ms-powerpoint
* pps: application/vnd.ms-powerpoint

* docx: application/vnd.openxmlformats-officedocument.wordprocessingml.document 
* pptx: application/vnd.openxmlformats-officedocument.presentationml.presentation
* ppsx: application/vnd.openxmlformats-officedocument.presentationml.slideshow
Comment 9 Michael Felczak 2011-09-09 15:45:02 PDT
Galleys uploaded in Firefox not indexed for search
https://github.com/pkp/ocs/commit/74879827587d508675503d3079db739f1c24d203
Comment 10 Michael Felczak 2011-09-09 15:45:02 PDT
Galleys uploaded in Firefox not indexed for search
https://github.com/pkp/pkp-lib/commit/a8fbae1cabc6251069bc1e5792e9bf465aee09a6
Comment 11 Michael Felczak 2011-09-09 15:48:22 PDT
Port to OCS master.

Check and port as needed to OJS, OMP, and OHS.
Comment 12 Michael Felczak 2011-09-28 21:55:02 PDT
PDF galleys not indexed for search
https://github.com/pkp/ocs/commit/baf880c2057c1aac43f66f19fc4f610009c5b316
Comment 13 Michael Felczak 2011-09-29 21:00:02 PDT
Galleys uploaded in Firefox not indexed for search
https://github.com/pkp/ojs/commit/7178b64b1aec39dcc94918ecd379ee9fe7f9c3de
Comment 14 Michael Felczak 2011-09-29 21:55:02 PDT
Galleys uploaded in Firefox not indexed for search
https://github.com/pkp/ojs/commit/e24789a97ffb52f9a8d0939b7a455689dd7afa0f
Comment 15 Michael Felczak 2011-09-29 22:15:01 PDT
Don't rely on Firefox mimetype for uploaded files
https://github.com/pkp/pkp-lib/commit/753aaf9ddc961cd78b92c6b5cd2768778cf582a5
Comment 16 Michael Felczak 2011-09-29 22:46:09 PDT
Ported to OCS master, OJS, and OHS. OMP not affected.
Comment 17 Michael Felczak 2011-10-11 22:11:40 PDT
Created attachment 3657 [details]
Patch against OJS 2.3.6

Patch should also apply cleanly to older versions of OJS 2.3.x
Comment 18 Michael Felczak 2011-10-11 22:21:02 PDT
Created attachment 3658 [details]
OCS MySQL database update
Comment 19 Michael Felczak 2011-10-11 22:22:35 PDT
Created attachment 3659 [details]
OCS PostgreSQL database update
Comment 20 Michael Felczak 2011-10-11 22:23:12 PDT
Created attachment 3660 [details]
OJS MySQL database update
Comment 21 Michael Felczak 2011-10-11 22:23:42 PDT
Created attachment 3661 [details]
OJS PostgreSQL database update
Comment 22 Michael Felczak 2011-10-11 22:29:27 PDT
The fix, database update, and search index rebuild will automatically be executed as part of the OCS 2.3.4 upgrade.

To manually fix on OCS 2.3.3-1 or older:

1) Apply the patch to your OCS install
2) Manually execute the database update statements
3) Rebuild your paper search index, i.e. php tools/rebuildSearchIndex.php
Comment 23 Michael Felczak 2011-10-11 22:30:23 PDT
The fix, database update, and search index rebuild will automatically be executed as part of the OJS 2.3.7 upgrade.

To manually fix on OCS 2.3.6 or older:

1) Apply the patch to your OCS install
2) Manually execute the database update statements
3) Rebuild your article search index, i.e. php tools/rebuildSearchIndex.php
Comment 24 Michael Felczak 2011-10-11 22:32:06 PDT
(In reply to comment #23)
> To manually fix on OCS 2.3.6 or older:

This should read "OJS 2.3.6 or older"
Comment 25 Alec Smecher 2012-09-24 09:08:52 PDT
This was already fixed.
Comment 26 Michael Felczak 2014-10-07 12:38:28 PDT
Galleys uploaded in Firefox not indexed for search
https://github.com/pkp/pkp-lib/commit/a8fbae1cabc6251069bc1e5792e9bf465aee09a6