First of all I’m not really a Magento developer nor its power user or any sort of user for that matter. My primary function is server administration and configuration for hosted web sites, Magento included. Like any good developer you will be handling all the nifty custom code and the web site itself. However, when it comes to mailing, your application will format the mail message and forward it to the system based software to transfer your message via SMTP to its destination.
Whatever your mail is containing, I’m pretty sure it is important that this mail reach its destination regardless who you are; a web site owner, developer or its end user.
From my perspective, and years of mail delivery debugging I will try to share some helpful tips on how to properly configure your Magento and server environment so that your mails do arrive on its destination.
Configuring Magento
Ok, let’s start with easy part, the part that you as a developers are most comfortable and familiar with. Magento by default is using built in PHP mail() function to send e-mails. Commonly some important settings are left at their default values in Magento admin, and are often overlooked or forgotten. By default most people are just configuring default mail sending address in Magento, and stop at that. This will only set the <from: > e-mail address in $additional_headers field for mail() function. This will indeed put a mail from address in recipients inbox to your mail address but if we inspect the mail headers and server logs actual sending e-mail address will be something like apache@servername.com. This alone will be reason enough for some spam filters to classify that mail as spam since mail-from and envelope-from will be apache@servername.com and <from:> headers will be your configured e-mail address.
In this case I’ve used apache@servername.com as a “true” senders address but this may vary, depending on how your web server is configured.
Additionally, you will not receive bounced undeliverable mails. They will be forwarded to the actual sending user, and you will not know what mails you need to remove from your mailing list, what mails you should resend etc. In the former case if you continue to send mails to undeliverable e-mail addresses, some systems may interpret this as a dictionary spam attack attempts and may stop receiving even legitimate mails from your servers IP.
To correct this problem PHP mail() function is accepting $additional_parameters in which we can set mail-from and envelope-from mail headers to same address as <from:> mail headers. Magento has this configuration option in its admin interface. Just log-in to your admin and navigate to advanced -> system config and look for option set return path just set it on yes and Magento will add in $additional_parameters your configured mail address and this will corrects the mail-from and envelope-from mail headers.
Configuring the server – part 1 (trusted users)
Depending on your server setup, your Magento web may be running under apache/nobody user or under Unix user you have been assigned by your server admin. In either case you must ensure that this user is trusted by your mail server software.
What do I mean by this? Well just sending the $additional_parameters to mail() function is not enough. Most mail servers have built in protection for mail forgery and are by default disallowing overrides on mail-from and envelope-from headers via $additional_parameters. You must make sure that user under which Magento is run is in mail servers trusted users list. Only then this method of setting return path in Magento admin will work.
If you are by some chance on some cPanel powered server, this will be by default turned on. But if you are hosted on somewhat custom configured dedicated server this option must be set by your system administrator.
I’m not saying necessarily that cPanel hosting is in this case better choice for your Magento, some custom dedicated server setups can be and are more configurable and tunable for performance.
In any case, if you are running Exim as your MTA you will need to configure:
trusted_users = apache
in /etc/exim/exim.conf. You can add additional trusted users by separating them with:
trusted_users = apache : unixuser : unixuser2
For Sendmail MTA you can add trusted users in /etc/mail/trusted-users
Your mail server may vary so please consult your manual or your system administrator. At this point you should have mail headers sorted out.
Configuring the server – part 2 (mail limits)
On most hosting systems or rented dedicated servers you will by default have some sort of hourly mail limit. This is nowadays common practice among system administrators, especially for shared hosting servers, to limit the bulk spam mailing either from users itself or from malicious scripts which are unauthorized on the system. Limits work on principle of limiting each user account on how many mails they can send in one hour. If for example this limit is set to 100 mails/h and you have 200 purchases/h, last 100 of your users will not get their mail. This will be instead bounced to your sending address (now this is why we need to configure return path correctly). Try to find out with your provider and/or system administrator if there is such limit in place, and if it can be customized for your site. This should be pretty trivial thing to do. Further more keep track of your mails sent per hour and if you are close to the hourly limit, talk to your administrator again.
Configuring the server – part 3 (sending address)
Image may be NSFW.
Clik here to view.
Ensure that e-mail address from which you are sending your mails actually exists on your server. Some of the remote mail servers will try to perform “Sender verification call-out“.
What does this means? When your mail arrives on its destination mail server, the receiving server, before accepting that mail, will communicate with your sending server and will try to determine if your mail address actually exists. If it does not, remote server will just discard this mail and it will bounce back to your address. Well, if it doesn’t exists, you won’t get this bounce and will not even be aware there is a problem until some end user reports this as an issue.
Configuring the server – part 4 (server domain)
As one of the spam filtering rules, most remote mail servers will check your server’s IP address and its DNS records.
Make sure your server’s FQDN is properly configured, both in your mail server and in your DNS. Additionally make sure that your server’s PTR records are set strait.
PTR records are reverse IP to hostname DNS records and should point to your hostname. Your hostname DNS record should also point to the same IP address so that circle is complete. Some remote mail servers which cannot verify your PTR and hostname will just refuse to accept your mails.
Configuring your domain
In recent years as additional spam protection several new methods are implemented to verify sender’s identity. SPF domain records are one of them and most of the biggest mail providers (including Google, Yahoo and Hotmail) are checking them.
Not having the SPF records will not hurt your mail delivery, but having them configured will help in delivering your mails to inbox and not SPAM folders.
SPF is just a TXT DNS record which looks something like this (in a single line):
google.com descriptive text "v=spf1 include:_netblocks.google.com ip4:216.73.93.70/31 ip4:216.73.93.72/31 ~all"
In this TXT record is a list of permitted mail servers for your domain. Meaning if mail claiming to be from your domain is not coming from this trusted list of mail servers, it’s not valid mail and is discarded. But if mail claiming to be from your domain is coming from the listed servers, then this mail is verified as authentic and will not be classified as SPAM.
For some help in how to configure properly your SPF records you can use the online wizard at: http://spfwizard.com/
Besides SPF you may consider publishing DKIM records, but configuring them are is a topic for another post.
Tracking your mail server reputation
In addition to all above mentioned it is a good practice to track your mail server’s IP reputation at various RBL blacklists and respond quickly to any listings.
Having your mail server listed in some RBL lists will most definitely classify your mails as spam or even bounce them to the sender address.
You can query the RBL list database for your mail server IP at: http://www.dnsbl.info/dnsbl-database-check.php