OJS OCS OMP OHS

You are viewing the PKP Support Forum | PKP Home Wiki



Problems with characters not showing up correctly

Are you responsible for making OJS work -- installing, upgrading, migrating or troubleshooting? Do you think you've found a bug? Post in this forum.

Moderators: jmacgreg, btbell, michael, bdgregg, barbarah, asmecher

Forum rules
What to do if you have a technical problem with OJS:

1. Search the forum. You can do this from the Advanced Search Page or from our Google Custom Search, which will search the entire PKP site. If you are encountering an error, we especially recommend searching the forum for said error.

2. Check the FAQ to see if your question or error has already been resolved.

3. Post a question, but please, only after trying the above two solutions. If it's a workflow or usability question you should probably post to the OJS Editorial Support and Discussion subforum; if you have a development question, try the OJS Development subforum.

Problems with characters not showing up correctly

Postby ryandc99 » Wed Apr 23, 2008 3:19 pm

Hello Everyone,
I am having some issues with OJS and its not showing certain characters correctly. For example if I type the name Andrić it changes the last character to a ? Any idea what is causing this and what can be done to fix it? I have posted my configuration below as a point of reference. Any help would be greatly appreciated! I am running OJS 2.2.0.0
Thanks
Ryan

general
installed On
base_url http://nodoubt54.com
registry_dir registry
session_cookie_name OJSSID
session_lifetime 30
scheduled_tasks Off
date_format_trunc %m-%d
date_format_short %Y-%m-%d
date_format_long %B %e, %Y
datetime_format_short %Y-%m-%d %I:%M %p
datetime_format_long %B %e, %Y - %I:%M %p
disable_path_info Off
database
driver mysql
host mysql.nodoubt54.com
name ojs
persistent On
debug Off
cache
cache file
memcache_hostname localhost
memcache_port 11211
web_cache Off
web_cache_hours 1
i18n
locale en_US
client_charset utf-8
connection_charset utf8
database_charset utf8
charset_normalization On
files
public_files_dir public
umask 18
security
force_ssl Off
force_login_ssl Off
session_check_ip On
encryption md5
allowed_html <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <b> <i> <u> <img> <sup> <sub> <br> <p>
email
enable_attachments On
time_between_emails 3600
max_recipients 10
require_validation Off
validation_timeout 14
search
min_word_length 3
results_per_keyword 500
result_cache_hours 1
oai
oai On
repository_id ojs.nodoubt54.com
interface
items_per_page 25
page_links 10
captcha
captcha Off
captcha_on_register On
captcha_on_comments On
font_location /usr/share/fonts/truetype/freefont/FreeSerif.ttf
proxy
debug
show_stats Off
show_stacktrace Off
Server Information

Basic operating system and server software versions. Click on Extended PHP Information to view extended details of this server's PHP configuration.
OS platform Linux
PHP version 5.2.3
Apache version Not available
Database driver mysql
Database server version 5.0.45-log
Extended PHP Information
ryandc99
 
Posts: 8
Joined: Mon Mar 24, 2008 7:24 pm

Re: Problems with characters not showing up correctly

Postby ryandc99 » Wed Apr 23, 2008 4:02 pm

I have seen this bug fix when searching. But I am new to the system and unsure what to do? I see the diffrent colors on this page mean diffrent things. If this is the fix, how do I apply it? http://pkp.sfu.ca/bugzilla/attachment.cgi?id=452&action=diff Any explanation would be great!
Thanks
Ryan
ryandc99
 
Posts: 8
Joined: Mon Mar 24, 2008 7:24 pm

Re: Problems with characters not showing up correctly

Postby ryandc99 » Thu Apr 24, 2008 8:07 am

I looked over that update and it appears the code is all similar? Does anyone out there know anything that could help me?
Thanks
Ryan
ryandc99
 
Posts: 8
Joined: Mon Mar 24, 2008 7:24 pm

Re: Problems with characters not showing up correctly

Postby jmacgreg » Thu Apr 24, 2008 2:17 pm

Hi Ryan,

By any chance, are you copy&pasting from an MS Office doc, or are you writing these out from scratch?

Thanks,
James
jmacgreg
 
Posts: 4183
Joined: Tue Feb 14, 2006 10:50 am

Re: Problems with characters not showing up correctly

Postby ryandc99 » Thu Apr 24, 2008 3:33 pm

I was copying and pasting from a document, or from another website it does not make any difference. When they go into the box, they appear just fine with the accent, its only after you submit the info do they become question marks.
ryandc99
 
Posts: 8
Joined: Mon Mar 24, 2008 7:24 pm

Re: Problems with characters not showing up correctly

Postby mj » Fri Apr 25, 2008 2:18 pm

Hi Ryan,

The issue lies with your MySQL installation -- the collation for the tables in your database are set to the default "latin1_sewdish_ci" character set, which does not support full UTF-8 characters, only a small superset of the Latin-1 (ISO-8859-1) character set. As a result, characters such as "ć" cannot be stored in the database, so they are replaced by the "?" chararcter. This is a common issue with MySQL installations -- in order to fix it, you'll have to either delete and re-create your tables, ensuring that they are set to UTF-8, or you can use the ALTER TABLE syntax for each of the tables in your database:

Code: Select all
ALTER TABLE `my_table` DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;


You may notice that this has already been done with the "site_settings" table; I was able to troubleshoot your issue by logging in to your phpmyadmin account with the information you posted from your config file above. I've since removed the username/password information to protect your security; while posting full details are very helpful, I'd highly recommend ensuring that you don't include any such information in the future, just to be safe.

Hope this helps,
MJ
mj
Site Admin
 
Posts: 304
Joined: Fri Mar 26, 2004 9:32 am
Location: Toronto, Canada

Re: Problems with characters not showing up correctly

Postby ryandc99 » Sat Apr 26, 2008 10:15 am

I did what you said, and fixed the collation on all of the tables and that worked so they are all set to utf 8, however when i attempted to change the main page info about this site, I found that the same errors are occurring with the characters showing up as question marks. Then I attempted to modify one of the authors names i was having issues with, and when I clicked to save I got the following error... DB Error: Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '=' Any ideas what can be done here?
Thanks
Ryan
ryandc99
 
Posts: 8
Joined: Mon Mar 24, 2008 7:24 pm

Re: Problems with characters not showing up correctly

Postby mj » Sat Apr 26, 2008 2:05 pm

Hi Ryan,

My apologies. Digging into this further, I've discovered that you'll have to update the collation of all the columns in all of your tables. The syntax for this is:

Code: Select all
ALTER TABLE `table_name` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;


This should bring all the columns to the same collation type as the tables and database (UTF-8). The lesson here for others who may encounter similar problems, is to ensure that you create your entire database as UTF-8 from the very beginning! :-)

Hope this helps,
MJ
mj
Site Admin
 
Posts: 304
Joined: Fri Mar 26, 2004 9:32 am
Location: Toronto, Canada

Re: Problems with characters not showing up correctly

Postby ryandc99 » Tue Apr 29, 2008 8:17 am

Thanks soo much! That solved the problem... I am not sure what happend as far as the installation goes, when I created the tables and unpacked them they seemed to automatically go to that sweedish setting. Not sure why...
Ryan
ryandc99
 
Posts: 8
Joined: Mon Mar 24, 2008 7:24 pm

Re: Problems with characters not showing up correctly

Postby mj » Tue Apr 29, 2008 8:20 am

Hi Ryan,

Glad that solved it. You did everything correctly in your installation -- the issue is that MySQL, which was developed originally by a Swedish group, defaults to the Latin-1 Swedish character set unless another is explicitly specified. While this may have made sense at the time the original MySQL was developed, it has certainly been the cause of many headaches for MySQL users ever since.

Best,
MJ
mj
Site Admin
 
Posts: 304
Joined: Fri Mar 26, 2004 9:32 am
Location: Toronto, Canada

Re: Problems with characters not showing up correctly

Postby soj » Thu Jul 24, 2008 11:11 am

Hi I'm getting the same DB error when working with some of my galley files. Does one run the query on one table at a time, replacing the 'table name' with each table's name, or might there be a query that will change all the tables at once? TIA! soj
soj
 
Posts: 151
Joined: Fri Oct 28, 2005 1:53 pm
Location: Norman OK USA

Re: Problems with characters not showing up correctly

Postby mj » Thu Jul 24, 2008 11:29 am

Hi soj,

Unfortunately, I believe you'll have to run the ALTER TABLE query on each table, replacing 'table name' with the appropriate name. The easiest way to do this is some copy-pasting in a text editor to get all the SQL queries, and then just run them all at once.

MJ
mj
Site Admin
 
Posts: 304
Joined: Fri Mar 26, 2004 9:32 am
Location: Toronto, Canada

Re: Problems with characters not showing up correctly

Postby soj » Thu Jul 24, 2008 11:50 am

Okie dokie! Thanks so much for the quick response! soj
soj
 
Posts: 151
Joined: Fri Oct 28, 2005 1:53 pm
Location: Norman OK USA

Re: Problems with characters not showing up correctly

Postby soj » Sun Aug 03, 2008 5:49 pm

Hi, I successfully ran the following command on each of the DB tables (I still have a text file copy if you would like to see it):

ALTER TABLE `table name` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;

However, we are still getting the following DB error when attempting to use the email function, including the "reset password" request.

DB Error: Illegal mix of collations (utf8_unicode_ci,IMPLICIT) and (utf8_general_ci,IMPLICIT) for operation '='

Any idea what else could be wrong? It's curious to me also that this something we did not encounter using the same server and database prior the 2.2.1 upgrade.
TIA! Soj
soj
 
Posts: 151
Joined: Fri Oct 28, 2005 1:53 pm
Location: Norman OK USA

Re: Problems with characters not showing up correctly

Postby soj » Mon Aug 04, 2008 2:25 pm

Well, I'm not the most database savvy person, that's for sure. I ran the command on all the tables (some 85) listed in phpMyAdmin under the DB name in the left hand sidebar. But that is not apparently all the tables, correct? Each of these contain further tables that also should also be collated utf8_unicode_ci, is that right? So, if I open each linked table in sidebar, I can then edit all the sub-tables or whatever they are called. I just want to make sure that every table listed in the database must be utf8_unicode_ci. Is that correct?

TIA, soj
soj
 
Posts: 151
Joined: Fri Oct 28, 2005 1:53 pm
Location: Norman OK USA

Next

Return to OJS Technical Support

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 3 guests