Fake email problems

[ramon]

Hello all,

I'm posting this message here as it would apply to all of PKP Software.
We are hosting journals from other institutions, usually temporarily.
We are having a problem with fake emails, that is, a user is sending an email but the SPF header produces conflicting information, as the server is from one institution but the email address is from another.
Some services allow this, either to receive or send, but others do not.
How do you solve this problem with your hosting service, as we know users have personal emails they use?
SMTP is not always available since many institutions do not open this service to outside servers, but we will try this approach while you respond.

Hotmail, Yahoo! and UOL, for example, always send OxS messages to the SPAM box, and now we suspect this is the main reason.

Last year it was discussed to have an internal message system within OxS.
This is pretty much the standard in most CMS, in that a user sends a message, but the recipients get notified by the System Administrator that they have a message in their mailbox.
Is this still in development?

[asmecher]

Hi Ramón,

An internal mail system wouldn't solve this problem because OJS still needs to send notifications, and those will potentially suffer the same fate. To be honest, we're avoiding an internal mailing system because it would be an enormous amount of work for limited benefit; there are thousands of PHP apps that send email without needing to implement an internal email system, and hosts should to be able to support them.

Have you confirmed that the sender address is the problem? I think it's worth putting some time into asking directly why email is not getting through before embarking on a big development effort.

Regards,
Alec Smecher
Public Knowledge Project Team

[ramon]

Hello Alec,

The issue is that the email server is myinstitution.org, while the sender is another.institution.com, for example.
While Google accepts this type of situation, other mail services don't and are widely used.

We thought about creating accounts for the Editorial Team and problematic reviewers for example, but it won't solve the problem in the long run.
An author with a personal e-mail and sends a message from within OxS may run into the same problem, where his message is sent from a server he doesn't belong to.

How do you solve this situation in you hosting service?
Hasn't anyone run into this situation?

[asmecher]

Hi Ramón,

This is a pretty common mail relaying setup; for example, I send email from my own email address through my ISP's SMTP relay, but my email address domain isn't the same as my ISP's. Have you checked the headers of the messages getting flagged as spam? There's often an indication of what's happening there.

Regards,
Alec Smecher
Public Knowledge Project Team

[asmecher]

Hi Ramón,

As I understand it, SPF gets triggered not based on the sender email address but on the return path. The return path is specified in OJS optionally, when the "allow_envelope_sender" option is enabled in config.inc.php, in the "Bounce Address" setting on Journal Setup page 1. (A site-wide default can also be set in config.inc.php in the "default_envelope_sender" setting.) If you don't want to trigger an SPF failure, the DNS record for the domain of the bounce addresses you enter must include your mail server (either local or SMTP, depending on your OJS configuration).

For example, if you specify a gmail address as an envelope sender, gmail's DNS record must specify your mail server using an SPF rule. This is unlikely in the case of gmail, of course. However, if you have an institutional email account that you can use as your envelope sender, you'll probably have better luck.

In any case, an internal mail system shouldn't be necessary to avoid triggering SPF failures -- if you can't use a local mail account for your bounce address, don't use the allow_envelope_sender option.

Regards,
Alec Smecher
Public Knowledge Project Team

[ramon]

Hello Alec!

Thanks for the amazingly fast response.
We performed quite a few tests and here our results.
The original situation of the config.inc.php file was allow_envelope_sender and default_envelope_sender commented.
No bounce address was defined in Step 1 of the Journal Config.

Our email sending process was the following:
Log in as the journal manager (with an institutional email from a different domain).
Enter the user list, select our user and send a basic email, adding other recipients (gmail, hotmail and our local domain).
We also tested sending message to the same user logged in.
We added another recipient with a local domain address to view the message headers.

All GMAIL messages were received without problems so far.
Hotmail and our local domain send messages to the SPAM box depending on the situation.

Our first test was to keep everything as default.

1. Sending email from a user to himself causes the message to go to the SPAM box in Hotmail.
2. Our local domain sends emails from the admin user to his account at the local domain end up in the SPAM box with a ****SPAM*** in the subject.
3. Sending from one user to the admin and another recipient has no problems.
4. Return path is the sender's email

Our second was to enable and set the allow_envelope_sender to Off, leaving default_envelope_sender commented.

1. Sending email from a user to himself the message is received fine in Hotmail.
2. Our local domain sends emails from the admin user to his account at the local domain end up in the SPAM box with a ****SPAM*** in the subject.
3. Sending from one user to another has no problems.
4. Return path is the sender's email

Our third test was to keep allow_envelope_sender to Off and enable the default_envelope_sender with a local domain address

1. Sending email from a user to himself the message is received fine in Hotmail.
2. Our local domain sends emails from the admin user to his account at the local domain end up in the SPAM box with a ****SPAM*** in the subject.
3. Sending from one user to another has no problems.
4. Return path is the sender's email

Our fourth test was to enable and set keep allow_envelope_sender to On and comment default_envelope_sender

1. Sending email from a user to himself the message is received fine in Hotmail.
2. Our local domain sends emails from the admin user to his account at the local domain end up in the SPAM box with a ****SPAM*** in the subject.
3. Sending from one user to another has no problems.
4. Return path is the sender's email

Our fifth test was to enable and set keep allow_envelope_sender to On and enable and set default_envelope_sender to a local domain

1. Sending email from a user to himself the message is received fine in Hotmail.
2. Our local domain sends emails from the admin user to his account at the local domain end up in the SPAM box with a ****SPAM*** in the subject.
3. Sending from one user to another has no problems.
4. Return path is the default_envelope_sender

Our sixth test was to enable and set keep allow_envelope_sender to On and enable and set default_envelope_sender to a foreign domain

1. Sending email from a user to himself the message is received fine in Hotmail.
2. Our local domain sends emails from the admin user to his account at the local domain end up in the SPAM box with a ****SPAM*** in the subject.
3. Sending from one user to another has no problems.
4. Return path is the default_envelope_sender

Since we started testing, the return path has been either the sender or the default_envelope_sender as far as we can understand.
Setting allow_envelope_sender to On seems to fix the problem of messages being received in the SPAM box when sent from the same user, when sending to Hotmail.

Any ideas to what further testing can we do, or what configuration to define in the config.inc.php?

[asmecher]

Hi Ramón,

The best way to know for sure what happened to an email that was never received is to look in your mail server logs; if you can identify the request in your web server log that resulted in the message being sent, you can try to correlate that date and time with your email server log to see if the message was accepted for delivery or not.

Regards,
Alec Smecher
Public Knowledge Project Team