Self-hosted mail server: what it really takes

Running your own mail server is entirely possible in 2026, and it is less work than its reputation suggests - but the work is in different places from where people expect. Installing the software takes an afternoon. Receiving mail is easy once port 25 reaches you. Sending mail that lands in the inbox at Gmail and Outlook is the hard part, and it depends almost entirely on things outside the server: DNS records you publish, the reputation of the IP address you send from, and its reverse DNS. Get those right and a small self-hosted server for a company or a family delivers as well as anything you pay per mailbox for. Get them wrong and the server works perfectly while your mail quietly goes to spam.

This guide is the honest overview: what a mail server is made of, what it needs from the network and from DNS, what the ongoing job looks like, and when you should not do it at all.

What a mail server actually does#

"Mail server" is shorthand for several services that happen to share a process and a disk. It helps to name them separately, because each one fails in its own way.

• Inbound SMTP on port 25. Other mail servers on the internet connect here to deliver mail addressed to your domain. This is the only way mail reaches you, and it has to be reachable from anywhere.
• Outbound SMTP. Your server connecting to other servers' port 25 to deliver what your users send. Many networks block this direction, which is the subject of port 25 and outbound mail blocks.
• Submission on port 587 (STARTTLS) or 465 (implicit TLS). Your own users' mail clients hand outgoing mail to the server here, after logging in. It is not the same as port 25, and mixing the two up is behind a large share of "I cannot send" tickets.
• Mail access: IMAP on 993, POP3 on 995, or JMAP over HTTPS. This is how clients read mail that has already arrived. IMAP vs POP3 vs JMAP compares them.
• Storage: the mailboxes themselves, with quotas, folders and search indexes.
• Filtering: spam scoring, authentication checks on incoming mail, and user rules (Sieve).
• Signing: DKIM signatures on outgoing mail, so receivers can verify it came from you.

Older setups assembled these from separate programs - Postfix for SMTP, Dovecot for IMAP, Rspamd for spam, OpenDKIM for signing - glued together with a dozen configuration files. Modern all-in-one servers such as Stalwart, Mailcow (a bundle of the classic components in containers) and Maddy do the whole job in one piece. That is a large part of why self-hosting got easier: the moving parts are the same, but you configure them in one place.

What it needs from the network#

Before you think about software, check the network the server will live on. Three things decide whether self-hosted mail is viable there at all.

Home connections fail most of these. Residential address ranges are on Spamhaus's Policy Block List by design, most ISPs block outbound 25, and you cannot set reverse DNS on a consumer line. That is why self-hosted mail lives on a server in a data centre rather than on a box under the desk.

Even in a data centre, outbound 25 is often blocked by default, because an open port 25 on a cheap server is exactly what spammers rent. The answer is not to fight that: send through a relay (a smarthost) that already has the reputation, while your own server still receives directly and holds the mailboxes. SMTP relay for outgoing mail covers how that works and what it costs.

The DNS records you will publish#

Mail runs on DNS more than any other service. For a domain example.com served by a mail host mail.example.com at 203.0.113.25, the minimum set looks like this:

mail              3600  IN  A     203.0.113.25@                 3600  IN  MX    10 mail.example.com.@                 3600  IN  TXT   "v=spf1 mx -all"s1._domainkey     3600  IN  TXT   "v=DKIM1; k=rsa; p=MIIBIjANBg..."_dmarc            3600  IN  TXT   "v=DMARC1; p=none; rua=mailto:dmarc@example.com"

What each one does:

• A for the mail host: the address of the server. The MX points at this name, never at an IP address and never at a CNAME.
• MX: tells the world which host accepts mail for the domain. MX records explained covers priorities, backup MX and the null MX for domains that never receive mail.
• SPF: lists which servers may send with your domain in the envelope. mx here means "the hosts in my MX records". If you send through a relay, its include: goes in this record too.
• DKIM: the public key receivers use to check your signatures. The mail server generates the key pair; you publish the public half under a selector. DKIM keys and rotation goes deep on selectors and key sizes.
• DMARC: the policy that ties SPF and DKIM to the visible From: address and asks receivers to send you reports. Start at p=none; DMARC reports and policy rollout is the path to p=reject.

Then the optional but useful ones: MTA-STS and TLS-RPT to require TLS on mail sent to you, autoconfig and autodiscover records so mail clients find their settings, and SRV records for the same purpose. MTA-STS and TLS for email explains the first two. None of them is required on day one.

The background for all of this - why alignment matters, why two SPF records break everything - is in SPF, DKIM and DMARC explained. If you have not read it, read it before you publish anything.

Deliverability is the real job#

Receiving is mechanical. Sending is a question of trust, and trust is earned slowly.

When your server connects to Gmail, the receiving side asks, in roughly this order: is this IP on a blocklist; does it have reverse DNS that matches forward DNS; does the EHLO name make sense; does SPF pass; does DKIM pass; does DMARC align; what is the reputation of this IP and this domain; and only then, what does the message look like. A new server on a new IP with a new domain starts with no reputation at all, and "no reputation" is treated cautiously.

What builds reputation:

1. Authentication complete and aligned. SPF, DKIM and DMARC all passing for the domain in From:. This is the entry ticket, not the prize.
2. Correct reverse DNS. The IP's PTR record resolves to a name, and that name resolves back to the IP. See reverse DNS and PTR for mail servers.
3. Low volume, steady pattern. A company of ten sending a few hundred messages a day to people who reply is close to the ideal sender. A server that sends nothing for a month and then 5,000 newsletters in an hour is not.
4. Recipients who want the mail. Replies, moves out of spam, and adding you to contacts all count. Complaints count heavily against you.
5. Time. Weeks, not days.

The full catalogue of reasons mail lands in spam, and how to diagnose each one, is in why email goes to spam. The short version for a self-hoster: if you only send person-to-person mail, a correctly configured server on a clean IP is usually fine within a few weeks. If you send bulk or marketing mail, use a dedicated sending service for that and keep your own server for conversation.

The ongoing work, honestly#

People who give up on self-hosted mail rarely give up during setup. They give up on month three when something odd happens and they do not know where to look. Budget for these:

• Watching the queue. Mail that cannot be delivered waits in the outbound queue and is retried for days. A growing queue is the first sign of a block or a broken relay. Look at it weekly.
• Reading DMARC reports. They tell you who is sending as your domain, including the forgotten invoicing tool and the spammer spoofing you.
• Checking blocklists when a delivery fails with a 5xx error mentioning one. Most delistings are a form and a wait.
• Spam filter tuning. Training on misclassified mail, adjusting thresholds, whitelisting the supplier whose invoices keep being flagged. Mail server spam filtering covers how the scores work.
• Updates. A mail server is permanently exposed to the internet on several ports. Keep the software current.
• Backups. Mailboxes are irreplaceable in a way most data is not. A snapshot that has never been restored is a hypothesis.
• Certificates. Clients connect over TLS to the mail host name; the certificate must be valid and renewed before it expires, or every phone in the company starts showing warnings on the same morning.
• Account hygiene. A compromised mailbox password turns your server into a spam source within hours and destroys the reputation you built. Strong passwords, and ideally app passwords per device.

None of this is a full-time job for a small organisation. It is perhaps an hour a month when nothing goes wrong, and an afternoon when something does.

Sizing: memory, disk and users#

Mail is light on CPU and memory and heavy on disk over time. Rough figures for an all-in-one server like Stalwart:

The thing that grows is storage. Attachments are most of it, and people never delete mail. Set quotas per mailbox from the start, because imposing them later on a user with 9 GB of mail is a conversation nobody enjoys. Search indexes add a percentage on top of the raw mail, and a backup needs room too if it is staged locally.

CPU spikes come from spam filtering on bursts of inbound mail and from indexing large mailboxes after a migration. Neither lasts long.

When not to self-host#

Saying this plainly saves people a lot of grief. Do not run your own mail server if:

• You need guaranteed delivery from day one - a new business whose invoices must arrive this week. Reputation takes time to build.
• Nobody will look after it. A mail server that nobody checks is a liability, not an asset.
• You send bulk or marketing mail. Use a service built for it, with its own IP pools and complaint handling. Keep your domain's day-to-day mail separate.
• You need a compliance regime - legal holds, eDiscovery, retention policies with audit trails. Those exist in self-hosted software, but you become responsible for proving them.
• Your network cannot meet the requirements in the table above and you are unwilling to use a relay.

And do self-host if you want control of your data, predictable cost that does not scale per mailbox, many domains or aliases without a price on each, or simply to understand the system your business runs on. A small team on its own server is cheaper than per-user plans after only a handful of mailboxes, and the knowledge pays for itself the first time something goes wrong with mail anywhere.

How this works on a RE:NODE Mail Server plan#

The Mail Server line is a whole Stalwart server that you run, not mailboxes sold one at a time. It comes with SMTP, IMAP and JMAP, and a web admin where you create domains and mailboxes, get the DKIM keys, and set up spam filtering. Stalwart mail server setup walks through the first hour.

The parts that remain yours: you publish the MX, SPF, DKIM and DMARC records at whoever runs your DNS - there is no zone editor on our side. Receiving mail from the internet needs port 25, which a container cannot open by itself, so you open a ticket and support forwards it to your server; it is one mail server per address, because only one server can own port 25 on an IP. If a network refuses your outgoing port 25, you send through a relay configured in the web admin. Plans carry five ports, so choose which services you expose, and backup slots from one to four depending on the tier. Whether a PTR record can be set for your server's address is not something to assume - ask support before you rely on it, and use a relay if the answer does not suit you.

FAQ#

Is self-hosting email still realistic for a small business?

Yes, for person-to-person mail on a server with a clean IP, correct DNS and someone who checks on it monthly. It is not realistic for bulk sending, and it is not realistic if nobody will own it. Most of the effort is in the first few weeks.

Why does mail from my new server go to spam when everything passes?

Because passing authentication only proves who you are, not that you are trustworthy. A new IP and domain have no reputation. Keep volume low and steady, make sure reverse DNS is correct, and expect it to improve over a few weeks rather than days.

Can I run a mail server on my home connection?

Not well. Residential ranges are on blocklists by policy, outbound port 25 is usually blocked, and you cannot set reverse DNS. You can receive at home with some effort, but sending will need a relay, at which point a hosted server is simpler.

How much disk does a mailbox need?

Ordinary office use grows by 1-3 GB per person per year, almost all of it attachments. Set a quota per mailbox from the start and plan disk for the next two years, not the next month.

Do I still need SPF if I sign everything with DKIM?

Yes. DMARC needs only one of them to pass and align, but SPF fails on forwarding and DKIM fails when a mailing list edits the message. Publishing both means one usually survives.