RE:NODE

App hosting12 min read

Migrating email to your own server without losing mail

Move mailboxes from Gmail, Microsoft 365 or a host to your own mail server: copy with imapsync, cut the MX over safely and keep every old message.

0 readers

Moving email to a new server is safe if you do it in the right order: create the mailboxes on the new server, copy the existing mail across with imapsync while the old provider is still receiving, lower the MX record's TTL, switch the MX, run imapsync a second time to pick up whatever arrived during the switch, and only then move people's mail clients over. Keep the old accounts for a few weeks afterwards. Done in that order, no message is lost and nobody has a day without mail. Done in any other order - switching DNS first and copying later is the classic - mail goes missing during propagation and there is no way to get it back.

This post walks through the whole move: what transfers and what does not, the imapsync commands, the quirks of Gmail and Microsoft 365, the DNS cutover, and keeping the old archive.

What moves over IMAP, and what does not#

IMAP carries mail and folders. Everything else a mail provider holds for you travels some other way, or not at all.

ItemMoves with imapsyncHow to move it otherwise
Messages and attachmentsYes-
Folder structureYes-
Read, flagged and answered flagsYes-
Original received datesYes-
Gmail labelsAs folders, with duplicatesSee the Gmail section
ContactsNoExport as vCard (.vcf)
CalendarsNoExport as iCalendar (.ics)
Server-side filters and rulesNoRewrite as Sieve rules
Aliases and forwarding addressesNoRecreate in the new server's admin
PasswordsNoSet new ones, tell the users
Signatures, client settingsLive in the clientUsually survive if the client is reused

Write the list of non-IMAP things down for your organisation before you start. The mail itself is the easy part; the alias that every invoice is sent to, which nobody remembered existed, is the part that breaks.

Before you start: inventory and sizing#

Collect, for every mailbox: the address, every alias that delivers to it, its size, and who uses it on which devices. Shared mailboxes and distribution lists need the same treatment. Most providers show mailbox sizes in their admin console; if not, imapsync --justfoldersizes against the old server prints them.

Then check three things on the new side:

  • Storage. Add up the mailbox sizes and leave room for growth and for the server's own indexes. A domain with 18 GB of mail does not fit on a 10 GB plan, and finding that out halfway through a copy is unpleasant.
  • Accounts. Create every mailbox and alias on the new server before copying. imapsync copies into accounts; it does not create them.
  • DNS access. You need to edit the domain's MX, SPF, DKIM and DMARC records. Find out where the domain's DNS is actually hosted - nameservers vs DNS records explains how to tell - and lower the MX record's TTL to 300 seconds at least a day before the switch, so the change propagates in minutes instead of hours.

Copying mailboxes with imapsync#

imapsync is the standard tool for IMAP-to-IMAP copies. It logs in to both servers, walks every folder, and copies messages that are not already on the destination, with their flags and dates. Running it a second time only copies what is new, which is what makes the two-pass cutover work. It is a Perl script published by its author, Gilles Lamiral; the simplest way to run it without installing Perl modules is the official Docker image, gilleslamiral/imapsync.

Run it from a machine with a good connection to both servers - your own computer, or a small VDS for a large migration - not from the mail server itself.

A first, harmless run that only reports what would happen:

bash
$ imapsync \    --host1 imap.oldprovider.example --user1 anna@example.com \    --passfile1 ./old-anna.txt --ssl1 \    --host2 mail.example.com --user2 anna@example.com \    --passfile2 ./new-anna.txt --ssl2 \    --automap --dry

Then the real one: the same command without --dry. The options that matter:

OptionWhat it does
--host1, --user1, --passfile1Source server and login; the password is read from a file
--host2, --user2, --passfile2Destination server and login
--ssl1, --ssl2Implicit TLS on each side
--port1, --port2Ports, when they are not the default 993
--automapMaps special folders (Sent, Drafts, Trash, Junk) by their role, not their name
--dryShows what would be done, changes nothing
--justfoldersCreates the folder tree only
--exclude 'regex'Skips folders whose names match
--maxage NOnly messages younger than N days, for a quick first pass

Using --passfile1 and --passfile2 instead of --password1 and --password2 keeps passwords out of your shell history and out of the process list on a shared machine. Delete the files afterwards.

For more than a handful of users, put the logins in a file and loop:

migrate.sh
#!/bin/sh# users.csv: old login;old password file;new login;new password filewhile IFS=';' read -r u1 p1 u2 p2; do  imapsync --host1 imap.oldprovider.example --user1 "$u1" --passfile1 "$p1" --ssl1 \           --host2 mail.example.com --user2 "$u2" --passfile2 "$p2" --ssl2 \           --automapdone < users.csv

A large mailbox takes hours. The bottleneck is almost always the source provider's rate limits rather than bandwidth, and imapsync resumes cleanly if interrupted - just run it again.

Gmail, Google Workspace and Microsoft 365#

The big providers each have one quirk that catches people.

Gmail and Google Workspace do not have folders; they have labels, presented over IMAP as folders. A message with three labels appears in three folders, and every message also appears in [Gmail]/All Mail. Copied naively, a 5 GB mailbox becomes 15 GB of duplicates. imapsync has a preset for this, --gmail1, which sets the right host and excludes the virtual folders such as All Mail and Important; read its documentation for exactly what it excludes, and decide whether you want messages that only exist in All Mail (archived without a label) copied into an archive folder. Google also limits IMAP downloads per account per day - its published Workspace limit is around 2,500 MB - so a large account takes several days, and the copy has to start well before the switch. IMAP access needs to be allowed, and with two-step verification on, the login needs an app password rather than the normal one.

Microsoft 365 / Exchange Online no longer accepts plain password logins over IMAP for most tenants; basic authentication was retired. imapsync supports OAuth2 access tokens for this (--oauthaccesstoken1), and there is an --office1 preset, but obtaining a token requires registering an application in the tenant. If that is more than you want to take on, the fallback is exporting each mailbox from Outlook to a .pst file and importing it from a client connected to both accounts, which is slower and loses less than people expect.

Other hosts and cPanel-style providers usually accept plain IMAP logins on 993 and need nothing special. Check whether the old host counts mail in a INBOX. namespace (folders named INBOX.Sent, INBOX.Archive); imapsync translates separators and prefixes automatically in most cases, and --dry shows you the folder mapping before anything is copied.

The DNS cutover#

When every mailbox has had a full first pass, switch mail delivery. The order matters.

  1. Make sure the new server can receive. Port 25 reaches it, the domain and all addresses exist, and a test message sent to the new server directly arrives.
  2. Add the new server to SPF alongside the old provider, for example v=spf1 include:_spf.oldprovider.example ip4:203.0.113.25 ~all. Both send for a while.
  3. Publish the new server's DKIM key under its own selector. The old provider's selector stays in place; they do not conflict. DKIM keys and rotation covers selectors.
  4. Change the `MX` record to the new server.
  5. Wait out the old TTL, plus a margin. Some senders cache longer than they should.
cached old MXnew MXSending serversMX recordTTL 300Old providerstragglersimapsyncsecond passNew serverport 25
Where mail lands during the switch

Your DMARC record does not change, as long as both the old and new senders pass SPF or DKIM in an aligned way during the overlap. MX records explained covers checking what the world sees with dig MX example.com +short and querying the authoritative server to bypass caches.

The second pass and the overlap period#

For a day or two after the switch, some mail still reaches the old provider: senders with cached DNS, and retries queued before the change. Run the same imapsync commands again. They copy only the messages that are new on the old side, and leave alone what is already on the new server - including mail delivered there directly since the switch.

Then repeat once more a few days later, and keep the old accounts open for two to four weeks. That window catches the last stragglers, and it gives you a fallback if something on the new server turns out to be wrong. When the old provider's inboxes stay empty for a week, remove its entry from SPF, delete its DKIM selector once you are sure nothing sends through it, and close the accounts.

Moving the mail clients

Clients are the last step, after the second pass, so that people open their mail client to a mailbox that already holds everything.

  • Desktop clients (Thunderbird, classic Outlook): add the new account alongside the old one rather than editing the old account's server settings. The client builds a fresh local cache from the new server, and the old account stays readable until you remove it. Editing the server field in place often confuses the local cache into showing duplicates or empty folders.
  • Phones: delete the old account and add the new one. Phones keep little locally, so there is nothing to lose.
  • Contacts and calendars: import the .vcf and .ics exports into wherever they now live.
  • Signatures and rules: client-side rules have to be recreated; server-side rules become Sieve rules on the new server.

Email client setup has the exact settings for each client, and is worth sending to users as the instruction sheet.

Keeping the old archive#

Years of mail is often the reason for the move and also the biggest mailbox in it. Options, in order of preference:

  1. Copy it all. If storage allows, everything lives on the new server and searching works as before.
  2. An archive account. Copy old years into a separate mailbox that a few people can open, keeping day-to-day mailboxes small and fast to sync to phones.
  3. An offline export. imapsync can copy to any IMAP server, but for a cold archive an mbox or Maildir export kept on separate storage is simpler. Store it twice, in two places; an archive kept in one copy is not an archive.

Whatever you keep, the new server now holds the only live copy of the organisation's mail, so set up backups on day one. Backups that actually restore explains why the restore test matters more than the backup schedule.

Troubleshooting a migration#

`imapsync` fails to log in to the old provider. IMAP may be disabled for the account or the tenant, the provider may require an app password because two-step verification is on, or the provider may have retired password logins entirely. Log in with a desktop client using the same credentials; if that fails too, the problem is the account, not the tool.

The copy is far larger than the mailbox. Duplicates from labels, almost always Gmail. Stop, delete the copied folders on the new server, and start again with --gmail1 or explicit --exclude rules for the virtual folders.

Folders arrive with odd names, such as `INBOX.INBOX.Sent`. A namespace or separator mismatch between the two servers. Run with --dry --justfolders and read the mapping it prints; imapsync has options to adjust prefixes and separators, and --automap takes care of the special folders.

The run stops halfway with timeouts or "too many connections". The source is rate-limiting you. Run one mailbox at a time instead of several in parallel, and simply run the command again; it continues where it left off.

Messages show today's date in the new mailbox. The client is sorting by the date it received the message from the server rather than the message's own Date: header. imapsync preserves the server's internal date by default; switch the client's sort column to the sent date, or rebuild its cache.

Mail sent to an alias bounces after the switch. The alias was never recreated on the new server. Check the inventory list against the new server's addresses - and check the bounce, which names the exact address.

Moving to a RE:NODE mail server#

The Mail Server line runs Stalwart, with SMTP, IMAP and JMAP and a web admin where you create the domain, the mailboxes and the DKIM key before the first copy. Plans start at 10 GB of mail storage and go up to 80 GB, so check your inventory total against the plan before ordering. Ask support to forward port 25 before you change the MX - one mail server per address - since a server that cannot receive is not ready for step four. Use the server's address and the IMAP port shown in the panel as --host2 and --port2. Backup slots come with every Mail plan; take one after the first full copy, and restore it somewhere once so you know it works.

FAQ#

Can I migrate email without any downtime?

Yes. With the two-pass method, mail is always delivered somewhere - to the old provider before and during propagation, to the new server after - and the second imapsync pass gathers the stragglers. Users notice only the moment they switch clients, which you control.

How long does an imapsync migration take?

Mostly as long as the source provider allows. A few gigabytes from an ordinary host takes an hour or two. Gmail's daily IMAP download limit means a large Google mailbox can take several days, so start the first pass well before the planned switch date.

Will moving my email break my website?

No, as long as you only change the mail records. The MX, SPF, DKIM and DMARC records are independent of the A or CNAME records that serve the website. Edit the mail records in place and leave the others alone.

What happens to mail sent during the DNS switch?

It is delivered to whichever server the sender's cached DNS points at. Mail that lands on the old provider is not lost; it is picked up by the second imapsync pass. That is why the old accounts must stay open for a while after the switch.

Can I run imapsync on the new mail server itself?

You can, but it is better not to. A migration is a heavy, long-running job, and the mail server has better things to do with its memory during the first week. Run it from your own computer or a separate machine with a stable connection to both servers.


Comments

Completely anonymous: no account, no email, no cookie. We store the name you type, the text and the time - nothing else. Links are limited and markup is not rendered.

0/2000