Skip to content
In this article9 sections

Running your own mail server gives you full control of your mailboxes and your data. It also means your messages must be accepted by Gmail, Outlook and every other provider, and the hardest part of that is not the software. Most self-hosted mail fails to reach inboxes because of the IP address, the reverse DNS record or missing authentication, not because the mail server is misconfigured. This guide starts with the decision, then shows the setup in the order that keeps mail flowing, with the checks that confirm each step.

Decide first

Self-hosting makes sense when you need control that a hosted provider cannot give you: mail stored on your own machine, custom routing, or a mailbox policy that a provider does not offer. It is a poor choice when mail is business-critical and you cannot afford an outage, or when nobody will watch the server.

Be ready for these ongoing jobs:

  • Deliverability. Your sending IP needs a good reputation. A new or shared IP can be blocked for weeks, and you will not get a warning email when it happens.
  • Authentication records. SPF, DKIM and DMARC must be correct and kept current. Large providers now require them for bulk senders.
  • Security. Mail servers are common targets for password guessing and for abuse. Fail2ban, strong TLS and updates are not optional.
  • Backups and restores. A mailbox you cannot restore is a loss, not a backup.

If the answer is still yes, the rest of this guide applies. If you want the full mail stack with a web interface, the Zimbra installation guide covers a complete package. A hosted email plan on SiteHarbour avoids the IP and maintenance work entirely, and it is the better choice for most small businesses.

Check the IP address before you buy the server

Your server’s IP address decides whether your mail is delivered. Check these before you install anything:

  1. Outbound port 25 is open. Many cloud providers block outbound port 25 by default and may need a request to unblock it. Test from the server:
    nc -vz gmail-smtp-in.l.google.com 25

    If the connection is refused or times out, ask the provider to unblock it before you go further.

  2. The IP is not on a blocklist. Check it at a public blocklist lookup, and check the history if the provider reused the address from another customer.
  3. The reverse DNS (PTR) record can be set. Receiving servers check that the IP’s reverse name matches your mail host name. Ask the provider to set the PTR record to your mail host, for example mail.example.com, and check it:
    dig -x 203.0.113.10 +short

    The answer must be the host name you chose, and that host must resolve back to the same IP.

If the reverse name cannot be set, do not run a mail server from that address. Mail sent from it will be filtered, and you cannot fix that with DNS records.

Set the DNS records

These records tell other servers that your mail is legitimate. Replace the examples with your domain and IP:

example.com.          IN MX 10 mail.example.com.
mail.example.com.     IN A  203.0.113.10
example.com.          IN TXT "v=spf1 mx -all"
_dmarc.example.com.   IN TXT "v=DMARC1; p=none; rua=mailto:[email protected]"
  • SPF lists the servers allowed to send for your domain. mx means your MX host, and -all means anything else should fail. Start with this, and add any other sending service, such as a newsletter tool, before you switch to -all.
  • DMARC starts with p=none, which reports problems without blocking mail. Read the reports for a few weeks, then move to quarantine and reject when every legitimate sender passes. The full method is in how to set up DMARC without losing email.
  • DKIM signs outgoing mail. The key is set up after the software is installed, in the next section.

Check that the records are visible from the public DNS before you test mail:

dig MX example.com +short
dig TXT example.com +short
dig TXT _dmarc.example.com +short

Install the software

On Ubuntu, Postfix sends and receives mail, and Dovecot serves mailboxes over IMAP and handles authentication. Install both, and choose “Internet Site” in the Postfix prompt:

sudo apt update
sudo apt install postfix dovecot-core dovecot-imapd dovecot-lmtpd opendkim opendkim-tools

Set the host name and the TLS certificate in /etc/postfix/main.cf. This is a starting point, not a finished configuration, so check the effective values with postconf -n after each change:

myhostname = mail.example.com
myorigin = /etc/mailname
mydestination = $myhostname, localhost
inet_interfaces = all
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level = may
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

The certificate must exist before Postfix is reloaded. The steps for it are in setting up free SSL with Certbot. The reject_unauth_destination line stops your server from being used to relay spam to strangers. Do not remove it.

Reload after each change, and check the configuration first:

sudo postfix check && sudo systemctl reload postfix

Add DKIM signing

Generate a key for the selector mail, then publish the public half in DNS:

sudo mkdir -p /etc/opendkim/keys/example.com
sudo opendkim-genkey -s mail -d example.com -D /etc/opendkim/keys/example.com
sudo chown -R opendkim:opendkim /etc/opendkim/keys/example.com
sudo cat /etc/opendkim/keys/example.com/mail.txt

Publish the contents of mail.txt as a TXT record named mail._domainkey.example.com. Then connect OpenDKIM to Postfix in /etc/opendkim.conf and main.cf, following the OpenDKIM documentation for your Ubuntu release, and check the signature on a test message:

dig TXT mail._domainkey.example.com +short

Test before you trust it

Send test messages to a mailbox you control at a large provider, and to a dedicated test address from a deliverability checker. Then check the headers of the received message. You want to see spf=pass, dkim=pass and dmarc=pass. You can also test the TLS on the submission and IMAP ports from outside:

openssl s_client -starttls smtp -connect mail.example.com:587 -crlf
openssl s_client -connect mail.example.com:993

A failed test is useful. Read the bounce message: it usually names the check that failed, and the fix is almost always a DNS record or the reverse name.

Security and backups

  • Limit login attempts. Install Fail2ban with the Postfix and Dovecot jails, so repeated password guesses are blocked.
  • Open only the ports you need. Port 25 for inbound mail, 587 for submission with login, and 993 for IMAPS. Close plain IMAP on port 143 if you do not use it.
  • Back up the mailboxes and the configuration. Copy /var/mail or the Dovecot mail store, and the Postfix and DKIM configuration, to another machine every day. Test a restore once a month.
  • Apply updates. Mail software and TLS libraries are updated for security reasons. Enable unattended security updates, and read the release notes for anything that needs manual changes.

Common mistakes

  • Sending from a shared or reused IP without checking it. The reputation belongs to the previous owner until it improves.
  • Setting DMARC to reject on day one. A missed sender, such as a newsletter service, will be blocked. Start with p=none.
  • Forgetting the reverse name. It is the most common cause of mail going to spam or being refused.
  • Opening relay to everyone. An open relay is abused within hours, and the IP is blocklisted.
  • Treating it as set-and-forget. Mail problems often appear weeks later, when a record expires or a certificate is not renewed.

Frequently asked questions

Can I host email on a home internet connection?

Usually not. Home connections often block port 25, use dynamic IP addresses, and have a poor reputation. A server with a static IP and a reverse name is the realistic minimum.

Is Postfix and Dovecot better than Zimbra?

Postfix and Dovecot are lighter and more flexible, but you assemble and maintain the parts yourself. Zimbra is a complete package with a web client and admin console, and it needs more resources. Choose by how much you want to maintain, not by the features list.

Will my mail go to spam at the start?

It can. A new IP has no reputation, so send small volumes to people who reply, and watch the DMARC reports. Reputation builds slowly over weeks.

How do I move existing mail to my new server?

Use a sync tool that copies mailboxes without losing messages. The method is in how to migrate email with imapsync.

Self-hosted email works when you can keep the IP clean, the reverse name correct, the authentication records accurate, and the server maintained. Check those four things first. If any of them cannot be guaranteed, a hosted mail plan is the safer choice.

Share this guide