Sending e-mails from the cloud based web application

by DmitryKirsanov 10. June 2016 06:26

 

 

This sounds crazy. But when you move your website to the cloud, you get problems in things you were previously taking for granted. Like sending e-mails. Basically, the problem is that many e-mail servers, usually ones of big providers, have Azure, Amazon and other cloud provider IPs blacklisted. When you attempt to use SMTP service from your virtual machine in Azure – in many cases it fails to deliver. This means, that your Azure machine can not act as mail server and shouldn’t attempt to deliver messages to recipient SMTP server directly.

Why would you use SMTP service at all? Well, mainly for the sake of performance. Your web application, be it ASP.NET, PHP, Ruby or whatever, will benefit from saving the outgoing e-mail message as text file somewhere on local hard drive, instead of trying to deliver it using TCP/IP, even if that’s done in asynchronous method.

This method of sending e-mails is called Pickup Directory – your application saves outgoing e-mail as well formed .eml file, and SMTP service picks it up (hence the name). In Windows Server, it’s done by system service called SMTP Service. You install it separately using Windows Server Manager. And it is perhaps the fastest way your app can send e-mail and return to serving client’s request, no matter what is the size of the message and whether external SMTP server is down.

So if we are about to solve this problem, we should try to keep using the pickup directory delivery method, but at the same time try to use our external SMTP account to send the accumulated messages. Unfortunately, the standard SMTP Service can’t do that, it may only connect to recipient’s SMTP server directly and act as SMTP server. And this means – we should replace the SMTP Service, but keep using the directory.

For that purpose I wrote an application, which works as both command line utility and windows service, which does exactly that – it polls the specified pickup directory and sends everything it finds to the SMTP account of your choice – be it your own e-mail server or Google / Office365, you name it. The Windows Service part of solution can even poll a number of directories, while command line utility is taking one directory at time.

Mail Processing Unit

Or MPU, for short. It’s compact, only 5Mb when installed. It’s flexible, and so far – reliable, as it’s working in a few production servers already with no problems.

How it works

There are two executable files – MPU.exe and MPUService.exe. The first one is a command line utility (CLU) – you can execute it with command line parameters, e.g.
mpu.exe c:\inetpub\mailroot\pickup /p
This will process the mentioned directory, and send all .eml files that are found in that directory. The parameter /p means “process”, it was implemented as security measure to prevent accidental processing when it’s not desired.

You can simply run the mpu.exe to see the list of parameters.

The MPUService.exe is a windows service executable, it’s not something you run from the command line. You can run it from the same Services management console as all other services.

The difference between the two is that you can script command line utility to run when you want using the Windows Task Scheduler, and process multiple directories simultaneously (or the same directory by more than one process). Windows Service has parameters, stored in registry, where you can specify more than one directory – they will be processed in order, one by one.

Each time either of these apps finds the .eml file to send, it renames it to .eml.lck and then processes. This allows to avoid double-processing of message, so potentially more than one application can poll the same directory. Then, MPU is using the SMTP account settings in [filename].exe.config file to send the message. The [filename] depends from what tool we are talking about – it might be either mpu.exe.config or mpuservice.exe.config.

Once message is sent, the file is deleted. If file cannot be sent, it’s renamed back for future processing. Since we are using SMTP account, there is little chance that only one message will be rejected while others not. If we would attempt to deliver message directly, such situation would be normal.

System Requirements

This application is supposed to work on servers, hence there was no point in making it 32-bit or compatible with older frameworks. Hence, it is 64-bit app, requiring .NET Framework 4.5, and it will work on any 64-bit edition of Windows Server starting from Windows Server 2008. Note, that all server editions starting with Windows Server 2008 R2 are 64-bit. It should also work fine on any other 64-bit machine that has .NET Framework 4.5 installed.

Installation

Simply download this file and run it. It will check for prerequisites (.NET Framework) and if they won’t be found – will offer to download and install it from Microsoft website. Then it will execute the MSI (Microsoft Installer) package, which will copy files and set up the Windows Firewall exceptions for both executable files. If Windows Firewall service is disabled in your machine, the installation may fail.

It also installs windows service, which is set up for Manual start up (i.e. will not run automatically unless you will change the start up method manually).

During the installation, you will be able to set up your SMTP account configuration, but if you’ll skip that step, you’ll have to edit your .config files for both CLU and Service apps. And, if you will be using Windows Service, you will need to set up the pickup directory and perhaps change the timer interval – by default service will attempt to search for new messages every minute. Perhaps you will want to increase that interval.

Configure Settings

There are two settings – one in .exe.config file (SMTP account settings) and one – only if you are using windows service – in registry. You can set up everything by running mpu.exe /setup .

  1. Run command line console As Administrator. Or use Far, which is way more convenient command line shell.
  2. Change directory to where MPU is installed.
  3. Run mpu.exe /log
    This will prepare windows event log for MPU, so any errors can be logged in the future and will be available in Windows Event Log. You only need to run this command once.
  4. Run mpu.exe /setup
    This will ask you two questions – where your pickup directories are and what would be the preferred timer interval. The results will be saved in HKLM\Software\DMKI\MPU part of your windows registry.
You may want to consult the Microsoft documentation to see how you can configure the .config files to work with your environment. I.e. for Office365 you may need to include additional settings, like below:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
  <appSettings>
    <add key="UseCredentials" value="true" />
  </appSettings>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5"/>
  </startup>
  <system.net>
    <mailSettings>
      <smtp from="address@example.com">
        <network defaultCredentials="false" host="smtp.office365.com" password="MyPassword" 
        enableSsl="true" userName="address@example.com"/>       </smtp>     </mailSettings>   </system.net> </configuration>
 

Usage

You you are using windows service, then you’re set. Just make sure you configure service for automatic startup (preferably with delayed start) and I would also recommend using windows service safeguard tool like Service Protector. Windows service will poll your pickup directory every minute or so (depending from what timeout you’ve set up earlier), but for better security I would also configure the CLU for running once a day using Windows Task Scheduler. Personally, I don’t do that, as windows service is stable in my environments, but you can do it.

In order to configure your ASP.NET application to send e-mails using pickup directory, you need to configure your app first.

Future Enhancements

Although I am quite happy with how it works at the moment, what I am going to do is to implement multi-threading, so multiple SMTP accounts could be used to send multiple messages at once, and then perhaps even batch processing – when message queue could be sent to another machine for batch processing without having to install any software there. That might be useful for high-load processing, but is rather fun than practical for me, as in current state the MPU is able to send approximately one message per second, especially when your SMTP server is in the same country.

Download

MPUSetup.exe [3 Mb]

(updated 10/09/2019, version 1.2.0)

 

 

blog comments powered by Disqus