3

Can anyone point me to a guide for using Postfix (or similar) to send email as an SMTP server for Gmail. I've had a solid search and come up empty handed.

I have multiple domains and all the email terminates in Gmail, but Google changed their policy a while back to stop letting users send email from multiple domains - now they insist you have your own SMTP server. I have a Ubuntu VPS I use for web hosting, is there any reason I couldn't use it as an SMTP server for Gmail?

  • 1
    I'm not sure what you mean "from multiple domains" here. I'm able to use my standard GMail address to send 'from' multiple websites. As for SMTP Server for GMail, no you can't use it as a GMail SMTP server; you have to set up an SMTP server for your specific domains, set it up in DNS, etc. and then send mail for that given domain through that SMTP server. – Thomas Ward Sep 5 '17 at 15:46
  • This isn't sending from websites. I have email from domaina.com and domainb.com forwarding to myaddress@gmail.com. I want to be able to send an email from Gmail from myname@domaina.com or myname@domainb.com from the Gmail interface. Up until about 18 months ago Gmail would let me send from any email address I could prove I owned. They changed the policy to only let you send from other email addresses if you could supply an SMTP server willing to send the email. That's what I want to set up, an SMTP server that will just send email for domaina.com and domainb.com. – Jonno Sep 5 '17 at 16:08
  • that's what I mean by "sending from a domain". You need to set up the SMTP portion of the GMail account section for that 'send mail as' that you added. You need to set up a Postfix server, configure it with authentication (headachey!), and then set up the corresponding DNS, SPF, and rDNS records (for antispam reasons), as well as authentication to the SMTP server. (This is not a small task) – Thomas Ward Sep 5 '17 at 18:19
  • I'll try and write up something for this, though I caution you that you might need Dovecot installed so it can handle the system level authentication... (I'll have to do some tests) – Thomas Ward Sep 5 '17 at 18:24
  • Thanks, appreciated. What you're saying is what I was expecting, authentication is the bit I'm new to. – Jonno Sep 5 '17 at 19:45
4

NOTE: This answer is written for a 16.04 server, but can be adapted for 14.04 if necessary (replacing systemctl ... commands with proper service commands.)

This is a very complex task to set up. There's a ton of separate important parts to this, but there's four primary components to a setup here, that is also 'secure':

  1. The ISP of the system being used (residential IPs are typically "bad" for this; you're going to want something at a datacenter with either a VPS or an AWS cloud EC2 instance, or something similar)

    Basically, do not use your own network for this. Set up your SMTP server somewhere else, like Amazon EC2.

    However, I strongly recommend getting a separate VPS from somewhere, where you can set the rDNS for the IP address to match the forward DNS you'll have to set. We'll touch on the DNS later.

  2. The system itself (locked down SSH, firewall, etc.)

    We need to pretty much configure the system to prohibit connections on port 25 and without SSL. We also need to just lock down the system. We'll do this step last, after we make sure the rest works.

  3. Postfix MTA

    This provides SMTP. We ultimately don't need it to deliver mail anywhere, so we don't need this as an MX record in your site's DNS, but we do need some things set up; we need an A record for the hostname, and we need a reverse DNS record to match (hence a suggestion of an actual VPS instead of an EC2 instance).

  4. Dovecot Core

    Normally we consider DoveCot to be a provider of POP3 or IMAP services, but dovecot-core on its own can be used to provide the SASL backend that works without many issues, and interacts with the system users. That's all we're using Dovecot for, but we'll need to adjust some configurations.

I will not touch on point #1, because I can't control that. I can however give you guidance on how to set things up. We're going to do it in a different order from the above list, but we'll still get this all done.


Dovecot Core

dovecot-core is going to be used for the SASL backend. There's a few things later we'll need to work with for 'secure' sending of mail, but we can start setting up the Dovecot backend.

First, install dovecot-core:

sudo apt-get install dovecot-core

Next, we need to make some modifications to /etc/dovecot/conf.d/10-auth.conf.

Let's disable plain-text authentication, and require SSL for the credentials. Ultimately doesn't matter from postfix -> dovecot, but this is a good step to do.

FIND: #disable_plaintext_auth = yes

REPLACE WITH: disable_plaintext_auth = yes

Next, we need to enable the 'login' auth mechanism.

FIND: auth_mechanisms = plain

REPLACE WITH: auth_mechanisms = plain login

Then, we need to edit the /etc/dovecot/conf.d/10-master.conf file.

FIND:

 # Postfix smtp-auth
 #unix_listener /var/spool/postfix/private/auth {
 #  mode = 0666
 #}

MODIFY TO BE:

 # Postfix smtp-auth
 unix_listener /var/spool/postfix/private/auth {
   mode = 0660
   user = postfix
   group = postfix
 }

This enables the Postfix authentication backend of SASL in Dovecot.

Finally, for Dovecot, we need to edit /etc/dovecot/conf.d/auth-system.conf.ext:

FIND:

passdb {
  driver = pam
  # [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=<n>]
  # [cache_key=<key>] [<service name>]
  #args = dovecot
}

MODIFY TO BE:

passdb {
  args = %s
  driver = pam
  # [session=yes] [setcred=yes] [failure_show_msg=yes]
  # [max_requests=> <n>]
  # [cache_key=<key>] [<service name>]
  #args = dovecot
}

Once this is done, we can restart the Dovecot process:

sudo systemctl restart dovecot

Postfix

Now, for the actual SMTP stuff. This isn't as hard to configure as Dovecot heh.

First, install Postfix and the ssl-cert package which will create a basic self-signed certificate that is just enough to work with SSL-secured SASL auth for SMTP clients:

sudo apt-get install postfix ssl-cert

When prompted for Postfix site types, just select "Internet Site".

Once that's done, we need to make some modifications to the Postfix files.

Find this line:

smtp      inet  n       -       y       -       -       smtpd

Add below those lines, so that the line looks like this with two additional lines below it:

smtp      inet  n       -       y       -       -       smtpd
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth

There is also a huge section starting with submission that is commented out. Take the entire section, uncomment that section, and make it look like this:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_wrappermode=yes
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination,check_policy_service unix:private/policy-spf,reject
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o smtpd_reject_unlisted_recipient=yes
  -o milter_macro_daemon_name=ORIGINATING
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth

With this now configured, we need to now edit the core settings, by editing /etc/postfix/main.cf.

First, move /etc/postfix/main.cf to /etc/postfix/main.cf.dist:

sudo mv /etc/postfix/main.cf /etc/postfix/main.cf.dist

Next, put the following into a new file. Make sure to update domain names and hostnames for at least one of your domains:

# See /usr/share/postfix/main.cf.dist for a commented, more complete version


# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_use_tls=yes
smtpd_tls_security_level=may
smtpd_tls_auth_only=no
smtp_tls_note_starttls_offer=yes
# Use an actual certificate.
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
# Use an actual key.
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
#smtpd_tls_session_cache_database=btree:${data_directory}/smtpd_scache
smtpd_tls_mandatory_protocols = !SSLv2,!SSLv3
smtpd_tls_protocols = !SSLv2,!SSLv3
smtpd_tls_mandatory_ciphers = medium
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
tls_random_source = dev:/dev/urandom
#tls_medium_cipherlist = AES128+EECDH:AES128+EDH

smtpd_client_connection_rate_limit = 50
anvil_rate_time_unit = 120s

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination, check_policy_service unix:private/policy-spf

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
myhostname = mail.somedomain
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname somedomain mail.somedomain localhost.somedomain localhost
relayhost = 
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all

local_recipient_maps = proxy:unix:passwd.byname $alias_maps

milter_protocol = 2
milter_default_action = accept

queue_directory = /var/spool/postfix

Note that a good portion of the configuration options here are used primarily for a full blown SMTP/IMAP setup with a Dovecot backend. You can still use this config, as this is pulled from an actual mail server (with some data obfuscated lol), as a base for setting this up to use just the SMTP portion of this for your things. So ultimately, don't worry about anything else.

Note that I recommend you get an SSL certificate just for the mail server, or if you have a wildcard cert for one of your domains use that and set the mailname to a domain in that certificate. This way, you aren't using self-signed SSL certs for the SSL/TLS encryption on the connection. Just my two-cents, you can still use the self-signed for most things if all you're doing is submitting email from an SMTP client like Google's Send Mail As function.

Once you're done here, we need to stop then start the Postfix process:

sudo systemctl stop postfix
sudo systemctl start postfix

We should now be good to go with Postfix and Dovecot SASL.


We need a user to auth with though.

We didn't create a user, and you have only a few options. In our case, we're going to create a dedicated user for this purpose of SMTP authentication.

We're going to create the user with the following:

sudo useradd --create-home --user-group --shell /usr/sbin/nologin authsmtpuser

We then need to set a password. You can do this with the following command, and enter the password accordingly. (Characters won't show up, nor will placeholders when it prompts unfortunately)

sudo passwd authsmtpuser

If you don't do this, and have another username on the box (NOT root) you want to auth with, use that in the script below.


Now we can test things.

Let's test this. This is a very simple Python 3 script that you can use to 'test' that everything works as we expect it to. Make sure to update DESTINATION_ADDRESS, SOURCE_ADDRESS, SMTP_USERNAME, SMTP_PASSWORD, and SMTP_SERVER_ADDR here to make sure that this script will properly work. Save this as test_smtp_server_tls.py somewhere, and remember where we saved this. (You can create it in /tmp/ if you want so it doesn't persist on your computer after a reboot)

#!/usr/bin/env python3

import email.utils as emailutils
import smtplib
from email.mime.text import MIMEText

DESTINATION_ADDRESS = 'your-address@gmail.com'
SOURCE_ADDRESS = 'smtp-test@yourdomain.com'

SMTP_USERNAME = 'authsmtpuser'
SMTP_PASSWORD = 'your password'
SMTP_SERVER_ADDR = 'ip.add.re.ss'
SMTP_SERVER_PORT = 587  # Don't change this.

# Construct a test message.
msg = MIMEText("This is a test message from your custom SMTP server!")
msg['To'] = DESTINATION_ADDRESS
msg['From'] = SOURCE_ADDRESS
msg['Date'] = emailutils.formatdate(localtime=True)
msg['Message-ID'] = emailutils.make_msgid()
msg['Subject'] = "SMTP Server Mail Testing"

try:
    print ("Establishing SMTP connection with SSL...")
    conn = smtplib.SMTP_SSL(SMTP_SERVER_ADDR, SMTP_SERVER_PORT)
    print ("Established connection.")
except Exception as e:
    print ("Could not establish SMTP connection.  Error was:\n%s" % str(e))
    exit(1)

try:
    print ("Attempting to authenticate...")
    (_, response) = conn.login(SMTP_USERNAME, SMTP_PASSWORD)
    if 'Authentication successful' in response.decode('utf-8'):
        print ("Authenticated with SMTP server successfully.")
    else:
        print ("Could not authenticate with server, response was: %s" % response.decode('utf-8'))
        exit(2)
except Exception as e:
    print ("Could not authenticate with SMTP.  Error was:\n%s" % str(e))

try:
    print ("Sending test message to [%s] from [%s] with subject [%s]..." % (DESTINATION_ADDRESS, SOURCE_ADDRESS, msg['Subject']))
    conn.sendmail(SOURCE_ADDRESS, DESTINATION_ADDRESS, msg.as_string())
    print ("Message queued for sending, please check your destination email inbox and junk folders for the message.")
except Exception as e:
    print ("Could not send email.  Error was:\n%s" % str(e))
    exit(3)

exit(0)

Execute this script with python3 /tmp/test_smtp_server_tls.py (assuming you saved it in /tmp/; change the path if you saved it elsewhere.

Check the destination email address inbox and such. If all worked, you'll get an email shortly from your server and the custom domain. It may go to your junk mailbox, so check there if you don't get the message initially.


The system - time to lockdown!

This assumes all the above steps worked, of course.

Time to setup the firewall to prohibit unauthorized access. Note that I hate ufw, it's not complex enough for my needs, so I have written this answer with iptables - you can do sudo ufw disable to turn off UFW and use direct iptables manipulation. I also only assume you have IPv4 on the system.

If you already have firewall rules in place, then just add a rule to accept new connections on port 587.

If you don't have a firewall enabled, execute these commands (add additional iptables -A INPUT -p tcp --dport 123 -m conntrack --ctstate NEW -j ACCEPT lines, replacing 123 with the actual port number of any other services on your system you need to accept connections to, such as tcp/443 and tcp/80 for HTTPS and HTTP accordingly, before you do the -j DROP line):

sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m comment --comment "Allow SSH access remotely." -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 587 -m conntrack --ctstate NEW -m comment --comment "Allow TLS-protected SMTP connections, for sending mail." -j ACCEPT
sudo iptables -A INPUT -j DROP

Once this is all set up, run this command: sudo apt-get install iptables-persistent. When it asks to save the current rule-sets, hit "Yes" for each.

Make sure your test script above and SSH access still works before you close the SSH connection or the connection to your server! If all works fine, then the last step is to set up the Google alias and tell it to send SMTP from that domain with a different server.


Google Configuration

If your GMail is under a Google Apps system, and you aren't on a Google Apps plan that lets you use external SMTP servers for mail sending, then you will not be able to do this part. This is a Google-imposed restriction, and I can't help you there.

From the GMail page, we need to configure things. Firstly, go into your GMail settings.

GMail Settings Button

Once in your settings, go to the Accounts and Imports tab.

You'll need to 'remove' any set aliases so far, in the "Send mail as", section, then hit "Add another email address".

enter image description here

Put in your name and the other email address for the other domain in the window that pops up.

enter image description here

Once you hit "next step" you'll get a page like this. Make sure to enter the IP address of your server properly, and also set the username and password to the ones we created earlier. Also make sure to select the "Secured connection using SSL" option. This enforces the need of SSL connectivity to happen before sending mail. This should look something like this, except with the IP address of your server in the blank space:

enter image description here

Save the setting with "Add Account". You'll receive an email indicating that you need to verify you want GMail to send mail with that address; it'll send you a link to click, with a verification code. You can just enter the verification code once the email arrives in the last window that opens during this process.

Once that's done, use GMail to send email to somewhere with the other "From" address selected, and it should send via the SMTP server we just set up! :)

  • This is brilliant, I'll give it a whirl. Thanks so much for the effort! – Jonno Sep 6 '17 at 19:40
  • @Jonno You're welcome! Let me know how it works, and let me know if there's anything else we can help you with. :) – Thomas Ward Sep 6 '17 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.