You installed Postfix and Dovecot, created a mailbox, sent a test message to your Gmail address — and it went to spam. Or it vanished. The software works: the message was accepted, queued and delivered. What failed is trust. Gmail, Outlook and Yahoo decided your server was not worth believing yet.
This post is about the part of self-hosted mail that no panel, including Vexlyx, can do for you: proving to receiving servers that your mail is legitimate, and then not burning the reputation you build.
Two reputations, not one
Receivers score every message against two separate reputations.
Domain reputation follows the domain in the From: header. It is built from how recipients treat mail from that domain: opened, replied to, marked as spam, deleted unread.
IP reputation follows the sending IP. A brand-new VPS IP has no history, and shared ranges are often already tainted by whoever used them before.
You cannot transfer reputation. A domain that sent 50,000 messages a month through a provider starts from zero on a fresh server. That is why even a perfectly configured new server will land in spam for the first few weeks if you immediately send a newsletter to your whole list.
Authentication records are how you get a fair hearing. They do not make receivers trust you; they stop receivers from rejecting you outright. For the record-by-record walkthrough — exact values, the OpenDKIM tables, and how to verify each one — see SPF, DKIM and DMARC on a self-hosted server, step by step.
The five records that decide your first impression
1. MX — where mail goes
The MX record tells the world which host accepts mail for your domain. Nothing else matters if this is wrong: SPF and DKIM are only checked after a receiver has accepted the connection.
Point MX at your server's hostname, not its IP address, and make sure that hostname resolves back to the server — the next record covers why.
2. Reverse DNS (PTR) — who owns the IP
PTR is the DNS record that maps an IP address back to a hostname. It lives with your hosting provider, not at your registrar, and you usually cannot edit it yourself — you open a ticket and ask.
Receivers check that the PTR name matches, or at least plausibly relates to, the hostname your server announces when it says EHLO. A generic PTR such as vps-1234.provider.net is not fatal, but a PTR that points to a different domain, or none at all, signals a server that nobody is looking after.
This is the record most often missed when moving mail to a VPS, because it cannot be set from inside the mail server. Vexlyx generates the mail-facing DNS records for a domain, but the PTR belongs to your host — the panel shows you the value you should be seeing, and your provider sets the real one.
3. SPF — which servers may send as you
SPF lists the hosts allowed to send mail claiming to be from your domain. It is checked against the envelope sender and the HELO name, not the visible From: header — an important detail that trips people up when they add a third-party tool later.
v=spf1 mx ~all
~all is a soft fail: receivers treat a mismatch as suspicious but not proof of forgery. It is the right starting point, because a hard fail (-all) combined with a forgotten sender — a website contact form, an invoice tool — silently kills real mail.
There is a second trap: SPF allows one record per domain, and each lookup inside it costs a DNS request. Ten lookups is the practical limit, and forwarding breaks SPF entirely because the forwarder sends from an IP you never listed. That is why SPF alone is never enough.
4. DKIM — a signature on every message
DKIM signs selected headers and the body with a private key; the matching public key sits in DNS at selector._domainkey.yourdomain.com. A receiver verifies the signature, which proves the message really came from a server that holds the key, and that nothing was altered in transit.
DKIM survives forwarding, which SPF does not, and it is the record that makes DMARC pass most reliably. Use a selector per service or purpose (mail, 2026, newsletter) so you can rotate a key without touching the others.
5. DMARC — what to do when both checks fail
DMARC does not authenticate anything itself. It tells receivers what to do when SPF and DKIM disagree with the From: domain, and where to send reports.
v=DMARC1; p=none; rua=mailto:dmarc@yourdomain.com
p=none means monitor only. Start there for two to four weeks, read the reports, and only then move to p=quarantine and later p=reject. Jumping straight to reject on a domain that still has an old CRM or contact form sending mail is how teams lose real business email.
Since early 2024, Gmail and Yahoo require SPF or DKIM plus a DMARC record from bulk senders, and apply stricter rules above roughly 5,000 messages a day — including one-click unsubscribe. Those rules favour servers that were configured properly from the start.
The defaults that quietly hurt you
- Port 25 outbound blocked. Many VPS providers block it by default to stop spam. Inbound mail works; outbound to other servers does not. Ask support to unblock it for your server.
- Sending everything at once. A new IP that sends 5,000 messages on day one looks exactly like a spam run. Send small volumes to engaged recipients first.
- A PTR that nobody checked. Ask your host what the PTR is, then confirm with
dig -xthat it is what you think it is. - DMARC at
rejecton day one. You will block legitimate mail from services you forgot about. - No DMARC reporting address. Without
rua, you are guessing.
A launch checklist that maps to what receivers check
- Confirm inbound ports 25, 587 and 993 are open, and that outbound 25 is unblocked.
- Set a PTR that matches the hostname your server announces.
- Publish MX, SPF (soft fail to start), DKIM and a DMARC record at
p=nonewith a reporting address. - Send a test message to a scoring service such as mail-tester.com and fix everything below 9/10.
- Warm up: low volume, engaged recipients, no purchased lists — ever.
- Watch bounces and deferrals. A queue that is growing usually means reputation, DNS or port 25, not a broken mail server.
- After a few weeks of clean DMARC reports, tighten to
quarantine, thenreject, and turn on strict alignment if your senders all line up.
Vexlyx scores each domain from 0 to 100 based on which authentication records are present and valid, generates SPF, DKIM, DMARC and MX when a domain gets its first mailbox, and publishes them for you if it hosts the zone — or shows you the exact records to add and checks them against public DNS if your DNS stays at your registrar. It also keeps a mail queue and delivery log, and starts you on soft-fail and monitor-only defaults deliberately, so nothing breaks before you understand the reports.
What it cannot do is the part that decides most outcomes: your server's PTR, whether your provider allows port 25, and how you send.
When it is still not enough
If your mail keeps landing in spam after the records check out, the problem is usually one of three things: an IP that was abused before you got it (ask your host for a clean one), a domain with a damaged history (send from a subdomain and let it earn reputation on its own), or sending behaviour that receivers read as bulk mail without consent.
What does help: fewer, better messages to people who want them, a plain-text alternative, and consistency — same server, same domain, same behaviour, for months.
FAQ
Do I need a separate IP for email?
No, but it helps at scale. On a small VPS the mail server shares the IP with your websites. If that IP has a poor history, ask your provider for a clean one before you start warming up.
Is DMARC required to deliver mail?
No receiving server requires it to accept a message, but Gmail and Yahoo require it from bulk senders, and many receivers weight it when deciding whether to spam-file you. Publish it at p=none from day one — it costs nothing and gives you reports.
Can I use my existing domain without moving DNS?
Yes. Keep the zone at your registrar and add the records the panel shows you. Vexlyx verifies them against public DNS once they propagate, and only writes the zone itself if you choose to host DNS there.
Why does my mail go to spam only on some providers?
Different receivers weight signals differently. Outlook is often stricter about IP reputation and PTR, Gmail leans on domain reputation and engagement. If only one provider files you as spam, your PTR and sending volume are the first things to check.
Does a panel fix deliverability?
No. A panel installs and wires the stack, generates the records and shows you their state. Reputation, PTR and sending behaviour stay with you. Any product promising otherwise is selling you a nicer dashboard, not deliverability.
Want the whole stack — Postfix, Dovecot, Roundcube webmail, SPF, DKIM and DMARC — on one server next to your sites and apps? See how Vexlyx handles self-hosted email, or compare it with cPanel if you are moving off a shared host.