Utf-8 and long subject lines

[shubhash]

Emails with long subject lines seem to generate invalid email headers giving strange results such as some headers being pushed down to the body of the email. After doing some research on the Internet, it appears that the php function "mb_encode_mimeheader" (used in OJS) seems to be buggy and has issues with utf-8. So I am wondering whether there is any workaround for this other than never using long subject lines.

[asmecher]

Hi Shubhash,

We haven't encountered this bug; if it is a bug in PHP, have you tried a newer release? I can investigate this further if you forward me an email that contains the buggy headers. (Please remove any incriminating content from the email first.) Do you know of a relevant PHP bug report?

Regards,
Alec Smecher
Open Journal Systems Team

[shubhash]

Hi Alec,

After looking deeper into the problem, I found that it is not related to utf-8, because it happens even with iso-8859-1.

The bug reports that I was reading are quite old, so it is possible that the problem may not exist in newer versions of php. However, I am unable to test it at present because we only use PHP 4.3.2. I was reading here (especially #23192)
http://bugs.php.net/search.php?cmd=disp ... mimeheader

but you can see that the problem report is quite old and the problem was not even considered a bug.

Here are the relevant mail headers:

Code: Select all

Date: Mon, 02 Oct 2006 11:11:24 -0600 (MDT)
Date-warning: Date header was inserted by xxxx.xxxx.xxxx
From: xxxxxxx
Subject: [TJ] Submission Acknowledgement - This is test of very long subject
 line,
To: xxxxxxx <xxxxxxx@xxxxxxx>
Message-id: <0J6I0PN3AQF0IA@xxxxx>
X-Envelope-from: xxxxxxx
Content-type: TEXT/PLAIN

 test of very long subject line, test of very long
Content-Type: text/plain; charset="utf-8"
X-Mailer: Open Journal Systems v2
X-Originating-IP: xxxxxxx
From: xxxxxxx <xxxxxxx@xxxxxxx>
Message-Id: <20061002171124.79BCE698001@xxxxxxx>
Date: Mon,  2 Oct 2006 11:11:24 -0600 (MDT)


The subject line was probably broken with extra line break (it may be OS specific I guess), so it was pushed down to the body of the email. So the mail server thought "message-id", "date", "content-type" were missing and created them again.

[asmecher]

Hi Shubhash,

This is how SMTP multi-line headers should work -- if a header is more than 75 characters in length, it should be split onto a second line by inserting a line break and a space (which appears to be the case here). Could you try a couple of slightly different configurations -- e.g. reading with a different mail client, and using SMTP as opposed to the mail() function (by configuring the SMTP section in config.inc.php) to try to narrow down the problem?

Regards,
Alec Smecher
Open Journal Systems Team

[shubhash]

Hi Alec,

If I use smtp mailer, it works fine. But I prefer sendmail for logging purpose.

Now I have found the cause of the problem. The problem is due to the inconsistency in the use of newlines.

As you know, Windows uses "\r\n" while Linux uses "\n" for a new line. This issue has been taken care in the file "classes/mail/Mail.inc.php" by defining a constant MAIL_EOL. Same needs to be done in "classes/core/String.inc.php". By default, the function uses "\r\n" for newlines. The newline character can be changed by supplying an extra argument.
http://ca3.php.net/manual/en/function.m ... header.php

See the bugzilla entry:
http://lib-pkp.lib.sfu.ca/bugzilla/show_bug.cgi?id=2453

[asmecher]

Hi Shubhash,

Thanks, and good catch! I'll look into this for correction in the next release.

Regards,
Alec Smecher
Open Journal Systems Team