OJS OCS OMP OHS

You are viewing the PKP Support Forum | PKP Home Wiki



Problems with characters not showing up correctly

Are you responsible for making OJS work -- installing, upgrading, migrating or troubleshooting? Do you think you've found a bug? Post in this forum.

Moderators: jmacgreg, btbell, michael, bdgregg, barbarah, asmecher

Forum rules
What to do if you have a technical problem with OJS:

1. Search the forum. You can do this from the Advanced Search Page or from our Google Custom Search, which will search the entire PKP site. If you are encountering an error, we especially recommend searching the forum for said error.

2. Check the FAQ to see if your question or error has already been resolved.

3. Post a question, but please, only after trying the above two solutions. If it's a workflow or usability question you should probably post to the OJS Editorial Support and Discussion subforum; if you have a development question, try the OJS Development subforum.

Re: Problems with characters not showing up correctly

Postby Kholodny » Fri Apr 19, 2013 9:05 am

Hi Alec,
thanks for reply.

asmecher wrote:Try turning off the "charset_normalization" setting. (This will speed up your installation and is safe to leave disabled; it was useful before UTF-8 was so widely supported but is now a little redundant and occasionally trips over valid UTF8.)


It was so by default, off course. Turning in On was one of other solutions described somewhere in the forum. Without any reaction in my case. It is very unusual situation: usually if there is a problem with cyrillic font support - they are wrong displayed in bulk - every character without an exception, but the case with only two disappearing - this is very strange. These two are not any special symbols - just normal ones like the others alphabet neighbours. And other strange thing is that one of them is uppecase and other lowercase and their opposite cases are displayed normally.

Vitaliy Kholodny
Kholodny
 
Posts: 10
Joined: Tue Apr 16, 2013 9:40 am

Re: Problems with characters not showing up correctly

Postby asmecher » Fri Apr 19, 2013 9:53 am

Hi Vitaliy,

I would suggest checking in the database to see if they're represented properly there. The "focus & scope" is stored in the journal_settings table and can be queried as follows...
select journal_id, locale, setting_value from journal_settings where setting_name = 'focusScopeDesc';
It's sometimes tricky to figure out character encodings because the terminal or interface you're using to connect to the database may interpret them before you have a chance to inspect them undoctored. The best way to do this may be to dump them to a file straight from the MySQL client and then use a good programmer's text editor to look at the resulting dump. (I can provide further instructions if that's gibberish; let me know what kind of access you have to the database, i.e. command-line or web-based.)

(With the charset_normalization setting turned off, have you tried correcting the setting in your Journal Setup? That may clear up the problem if the charset_normalization setting was causing data to be stored incorrectly in the database.)

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7717
Joined: Wed Aug 10, 2005 12:56 pm

Re: Problems with characters not showing up correctly

Postby Kholodny » Fri Apr 19, 2013 11:10 am

Thanks Alec.
I have access through phpMyAdmin 3.5.5. Tried some variants of exporting files - all of them transform all the symbols.
Kholodny
 
Posts: 10
Joined: Tue Apr 16, 2013 9:40 am

Re: Problems with characters not showing up correctly

Postby Kholodny » Mon Apr 22, 2013 6:34 am

Hi Alec!

asmecher wrote: checking in the database to see if they're represented properly there.


Unfortunately the database contain the non-correct symbols.
Entered into form: Иш АБВГД абвгд ABCDE abcde
The database contains: �?�? АБВГД абвгд ABCDE abcde

What else to check?

Vitaliy Kholodny
Kholodny
 
Posts: 10
Joined: Tue Apr 16, 2013 9:40 am

Re: Problems with characters not showing up correctly

Postby asmecher » Mon Apr 22, 2013 8:13 am

Hi Vitaliy,

Interesting. If you're using Firefox, could you install Firebug to inspect the transactions between your browser and the server? I wonder if e.g. TinyMCE is intervening. Submit a POST request to save the page's settings and inspect the data that's being sent to the server. This will help determine whether those characters are getting garbled on the client side or the server side.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7717
Joined: Wed Aug 10, 2005 12:56 pm

Re: Problems with characters not showing up correctly

Postby Kholodny » Tue Apr 23, 2013 9:47 am

Hi Alec!

asmecher wrote:Submit a POST request to save the page's settings and inspect the data that's being sent to the server.


First time using Firebug :)
If I found the correct info, it looks as follows:

I filled the same "Иш АБВГД абвгд ABCD abcd" into the focus and scope form:

Date: Tue, 23 Apr 2013 16:15:04 GMT
Server: Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 PHP/5.2.17
X-Powered-By: PHP/5.2.17
Location: http://cryo.org.ua/journal/index.php/pr ... tupSaved/2
Content-Length: 0
Connection: close
Content-Type: text/html

customAboutItems[en_US][0...
customAboutItems[en_US][0...
customAboutItems[ru_RU][0...
customAboutItems[ru_RU][0...
customAboutItems[uk_UA][0...
customAboutItems[uk_UA][0...
focusScopeDesc[en_US] Иш АБВГД абвгд ABCD abcd
formLocale en_US
lockssLicense[en_US] This journal utilizes the LOCKSS system to create a distributed archiving system among participating libraries and permits those libraries to create permanent archives of the journal for purposes of preservation and restoration. <a href="http://lockss.org/">More...</a>
mailSubmissionsToReviewer... 1
numWeeksPerReview 4
privacyStatement[en_US] The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.
restrictReviewerFileAcces... 1
reviewGuidelines[en_US]
reviewPolicy[en_US]
reviewerAccessKeysEnabled 1
reviewerDatabaseLinks[0][...
reviewerDatabaseLinks[0][...


http://cryo.org.ua/journal/index.php/pr ... %5Burl%5D=


Seems like the symbols were sent to server in normal shape.

Vitaliy Kholodny
Kholodny
 
Posts: 10
Joined: Tue Apr 16, 2013 9:40 am

Re: Problems with characters not showing up correctly

Postby Kholodny » Thu Apr 25, 2013 2:22 am

Hi Alec!

Could be the problem the following settings of the server:

character set client cp1251
(Session value) utf8
character set connection cp1251
(Session value) utf8
character set database cp1251
character set filesystem binary
character set results cp1251
(Session value) utf8
character set server cp1251
character set system utf8
character sets dir /usr/share/mysql/charsets/
collation connection cp1251_general_ci
(Session value) utf8_general_ci
collation database cp1251_general_ci
collation server cp1251_general_ci


So if all my DB are set to utf8_unicode_ci, but the server has cp1251 globally and utf-8 as session value (have not found waht is the difference between global and session value), could this be responsible for the strange transformation of these two symbols?

Thanks,

Vitaliy Kholodny
Kholodny
 
Posts: 10
Joined: Tue Apr 16, 2013 9:40 am

Re: Problems with characters not showing up correctly

Postby Kholodny » Thu Apr 25, 2013 4:57 am

I have reinstalled OJS manually (earlier it was done by Softaculos in Cpanel) and databases collation was set by server as cp1251_general_ci. Tried to fill the form with problem characters - and they appeared normally when viewing thereafter. If collation was changed to utf8_unicode_ci - the "wrong symbols" appeared again.
Does this mean that I can not use utf8_unicode_ci for cyrillic text at least on the server, which has general cp1251 character set? If I will set the collation for databases as cp1251, would this make any problem in further use of OJS? Or it is not predictable and I would be an experimental animal for this issue? :)
Kholodny
 
Posts: 10
Joined: Tue Apr 16, 2013 9:40 am

Re: Problems with characters not showing up correctly

Postby asmecher » Thu Apr 25, 2013 8:22 am

Hi Vitaliy,

I haven't got any specific experience with Cyrillic, but UTF-8 ought to support it fully. I wonder if there's some non-UTF-8 configuration left in the system somewhere, i.e. a table or column collation.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7717
Joined: Wed Aug 10, 2005 12:56 pm

Re: Problems with characters not showing up correctly

Postby Kholodny » Fri Apr 26, 2013 10:42 am

Hi Alec!
asmecher wrote:I wonder if there's some non-UTF-8 configuration left in the system somewhere, i.e. a table or column collation.

I applied the command to change collation described in this topic earlier and checked the databases one by one: all of them were with utf-8_unicode. So the problem probably was due to the conflict of general character set of the server (cp1251) and sent by OJS data in utf8. Don't know if it is possible, but I can't check other server to exclude this. Will try to use the cp1251 collation for my databases, hopefully this will not interrupt the function of OJS.

Thanks,

Vitaliy Kholodny
Kholodny
 
Posts: 10
Joined: Tue Apr 16, 2013 9:40 am

Previous

Return to OJS Technical Support

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 3 guests