Spam, Phishing and Email Security
Email: Security, Spam and Phishing
© Prof. Amir Herzberg Computer Science Department, Bar Ilan University http://AmirHerzberg.com
Introduction
Email: refresh, goals, threats, tools
Phishing and spam; tragedy, legal
Basic defenses
Port blocking
Content Filtering
White/Blacklists/Greylisting
Sending-domain Validation
SMTP-Sender Authorization (SPF, SIDF)
Signed Email (DKIM)
DMARC
Future? [Reputation, Accreditation, Penalties…]
And : phishing in social networks
Mail Servers
Why Study Email Security?
Email: vulnerable application
Spam, spoofed mail, phishing, scams…
Confidentiality, Integrity, Authenticaton, …
All easier on social networks… why email?
Answer: interoperability, use, openness, trust
Spear-phishing: still #1 way to attack enterprises!
Still… Least hot part of course
Toward end: phishing in social networks
User Agent submits (sends) via Mail Submit Agent (MSA)
User’s mailbox kept on Mail Delivery Agent (MDA)
Relaying by Mail Transfer Agents (MTA)
[RFC 821, 1821, 2821, 5321]
Over TCP Initiated by sending agent, called SMTP-client SMTP-server listens on port 25, receives email
MUA (Mail User Agent) to MTA (Mail Transfer Agent) Or: MTA to MTA
Three phases: handshake (greeting) transfer of (one or more) messages
Default: one message at a time (to 1 recipients) Pipelining: only supported by extension, see RFC 2920
1
But TCP still pipelines packets!
closure ASCII command/response interaction
MTA B
Internet MSA
MDA
Alice's MUA
Bob's MUA
Domain a.com
Domain b.com
Next (or skip): refresh on email
SMTP: Simple Mail Transfer Protocol
MTA A
SMTP Phases
Handshake (greeting)
TCP Connection Setup: SMTP-server rejects if overloaded Server hello: sends 220 B.com
Client hello: sends HELO A.com
Server may delay hello (until ready)
Or EHLO (extended hello: supports options)
Transfer of one or more messages:
Envelope:
MAIL FROM: sender email (for error report) RCPT TO: recipient that Receiver-SMTP should deliver to Multiple recipient by multiple RCPT TO commands
Message: RFC822 message (and headers), then terminator
SMTP closure: C: QUIT, S: 221 TCP connection closure (initiated by SMTP-client)
Closure
MAIL FROM:
[email protected]
RCPT TO:
[email protected]
MAIL FROM:
[email protected]
Repeat for more recipients
RCPT TO:
[email protected] m
DATA
Mail envelope: MAIL FROM:, RCPT TO: addresses SMTP uses these fields to route / reject messages
Accept only mail to known users
More on this later
Distinct from “similar” Email message headers (RFC 822, 2822, 5322)
Multiple recipients, BCC S: C: S: C: S: C: S: C: S: C: S: C: C: C: C: C: S: C: S:
220 sponge.com HELO A.com Notice also 250 OK multiple RCPT TO MAIL FROM:
(one per recipient, 250 OK all in same domain) RCPT TO: [email protected] 250 OK RCPT TO: [email protected] 250 OK DATA 354 Enter mail, end with "." on a line by itself from: [email protected] to: [email protected] I _still_ did not receive the goods you promised . 250 OK QUIT 221 b.com closing connection
Envelope
Message
SMTP delivers each message by the sequence:
Close
SMTP Envelope
Handshake
SMTP Phases and Envelope
– E.g.: Hex encoding (x’A3’”A3”); usually use base64 (~20% extra)
• Identify in header: MIME version, encoding, data type, subtype – Types: Text, Image, Audio, Video, Application, Multipart MIME version method used to encode data Data type/subtype encoded data
From: [email protected] To: [email protected] Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg base64 encoded data ..... ......................... ......base64 encoded data
Multipart: multiple parts (with separators defined in header)
2
[SYN] [SYN+ACK] [ACK, no data] 220 B.com Server (S:) B.com (Bob’s) HELO a.com Sender (client) MTA: a.com (Alice’s) 250 OK MAIL FROM: Inform [email protected] of 250 OK delivery error (`bounce`) RCPT TO: [email protected] 250 OK DATA 354 Enter mail, end with "." on a line by itself from: [email protected] to: [email protected], [email protected]
RFC822/2822/5322
Email Message Syntax what a wonderful message . 250 OK QUIT [+FIN] 221 b.com closing connection [+FIN] [ACK, no data]
Mail message format (RFC 822/2822/5322) • Header lines, e.g., – – – – –
To: From: Subject: MANY… unknown ignored. IDs may differ from envelope IDs, e.g. forwarding – And: more message IDs: sender, resent-from, …
Header
blank line
Body
• Blank line • Body – Message: printable ASCII chars only – Send non-ASCII chars by encoding (MIME – hidden)
Envelop IDs Message IDs
MIME: multimedia mail extension • How to send non-ASCII data, e.g. picture, program? • Encode: map data bytes to (more) printable-ASCII bytes
C: S: C: S: C: S: C: S: C: S: C: S: C: C: C: C: C: S: C: S: C:
(Alice sent via ISP A.net to [email protected], forward to [email protected])
S: C: S: C: S: C: S: C: S: C: C: C: C: C: S: C: S:
220 B.com Server (S:) B.com (Bob’s current company) HELO acm.org Sender (client) MTA: acm.org (forwarding srvc.) 250 OK MAIL FROM: Inform [email protected] of 250 OK delivery error (`bounce`) RCPT TO: [email protected] 250 OK DATA 354 Enter mail, end with "." on a line by itself from: [email protected] to: [email protected], [email protected]
RFC822/2822
Email Message Syntax what a wonderful message . 250 OK QUIT 221 b.com closing connection
Email Sender Identities
SMTP Error Handling
SMTP-Client, Helo name identify only relaying agent (MTA)
MUA displays only the From: header (usually)
Often unrelated to email sender identity
RFC822 From: asked for message to be sent
To allow transparent mobility But, spoofing email becomes trivial (so some prevent/authenticate) Partial alternate solution: use Reply-To: header
Reliability: Key goal of SMTP
If agent cannot deliver before sending OK:
Most MSAs allow use of any From: identity
Several other sender-related identifiers: Sender: “entered message into email stream” Resent-from, resent-sender:
RFC2822: identifies party (time, etc.) resending message Specifically not intended for use for forwarding (by user or by service) But used, e.g. by sendmail, mailing-lists…
Transient error (4xx): retransmit after reasonable delay
May try alternate route (use secondary mail server)
Send Delivery Status Notifications (DSN) to bounce (mail from) address
DSN has empty MAIL FROM field
Receiving Relay: recipient in the domain
Authorized-sender Relay: from authenticated domain user, or from authenticated MTA / MSA in domain
Or: Direct from sender domain to receiver domain
Assume: Authorized-sender or receiving relaying
Direct from sender domain to receiver domain
Or: Inside sender / receiver domain
Consider traceback (e.g. of spoofed email) No conflict using log files and traceback information, identify “real” source of email Traceback conflict:
From outgoing MTA in sender domain (e.g. a.com)
To incoming MTA in recipient’s domain (e.g. b.com)
If among servers within domain: detect corruption!
Traceable Relaying: Inside (sender/recipient) domain, or direct (sender to receiver domain)
If sender domain denies: may refuse future connections
Caution: Can also be corruption of “our” server or a spoofed connection
Allows responding to traceback conflict
SMTP Traceback
Email Relaying Services
Relaying within domain is fine
Open Relay MTA
MUA to MSA
MSA to MTA
MTA to MDA
Sender-Receiver relaying is fine
From sending MTA
To receiving MTA
Open Relay:
3
Permanent (5xx) or transient (4xx) error codes
Relaying and Traceback Conflict
Usually: Relaying is inside sender / receiver domain
Signal to Sender-SMTP by error code
If agent cannot deliver (after sending OK):
Relaying in SMTP
Recipient not known, sender not allowed, queue full, etc.
On error in transfer to next mail server:
Bounce address to report errors (next topic)
Not in sender and receiver domains
MTA A
MSA
What if bounce address is corrupted?
Solution: Traceback message route
Origin selects, puts unique Message-ID:
Each SMTP-receiver adds header to message
MTA B
MDA
Alice MUA
Bob MUA
Domain a.com
Domain b.com
Delivery Status Notifications (DSN) depend on bounce address (MAIL FROM / Return-Path:)
Possibly intentionally (to hide traces – e.g. spoofed email)
received: from by for ;
, : domain names and IP addresses
SMTP servers maintain logs to confirm / deny received: records
Traceback conflict one of the pair lies
Open Relay Considered Harmful
Open Relay: MTA that relays from and to outside its domain
Email Threats, Goals, Tools Exposure of: content, peers, location
Main tool: encryption (S/MIME, PGP,...)
Years Ago: No DNS Open relaying was critical
Receiver must be able to correctly decrypt
Today: Often abused to send spam
Other tools: unintentional leakage detection, …
Upon traceback, Open Relay claims it got email from xxx
Is xxx bad? Or is the Open Relay lying and faulty?
Tools: digital signatures/MAC (S/MIME, PGP, DKIM…)
Relay only inside domain or from sender’s domain
Many MTAs refuse email from Open Relays
Goals: integrity / authentication / `non-repudiation` (?)
Most MTAs are not Open Relays
Fake email
DKIM: invisible to non-supporting recipients
Spam/phishing: undesirable/malicious (email) msgs
Goals: block / limit / deter (punish)
Our focus
Key Distribution: still a challenge!
End-to-End E-mail Crypto
Encryption: public key, shared key, hybrid (envelop)
Digital signatures and message authentication
End-to-End: Sender & recipient must interoperate Backward-Compatible Crypto (`no harm’) ?
Encryption: absurd
Authentication / signatures ? It depends:
Ignored by non-compatible MUA
Spam: unsolicited, undesirable communication
Lots of it (`bulk')
Without `warning label` (AD: xxx...)
4
Getting certificate is not too difficult or expensive - but few do
In theory, user may edit (remove, add, etc.) – very few do
Security based on that of DNS (or DNSSEC)
PGP: User manages public keys of peers
Peer to peer – “web of trust” model
Each user acts as a certificate authority (CA)
User can accept public keys “certified” by a trusted peer
Messaging apps (e.g. What’s up):
Default: distributed by (trusted??) provider
Rarely used, problematic manual procedure
Why we hate Spam
Spam annoys and wastes time, resources
Spam motivates net-bots
Spam carries:
Mostly: (illegal) ads and phishing
Scams: Pyramid schemes, stock `tips`, …
Phishing: E.g., expose credentials to fake site
Enter (spoofed) website, run malware, etc.
Malware
Spear-phishing: same but targeted to user
Undesired content: e.g. “CP” (child porn)
Phishing: (spam) trying to fool the user:
Predefined list installed in MUA
Spam, Phishing, Spoofing
DKIM: Public key defined in DNS record
DKIM: “transparent” signatures & keys In new email message headers, e.g. DomainKeys-Signature:
S/MIME: List of trusted Certificate Authorities (CAs)
OpenpPGP, S/MIME: Signature as (unknown?) attachment
Identify/authenticate `sender’ [user? Server? Provider?]
May not be `bulk’
Spoofed site: unauthorized mimic of a site
If we all hate it, why providers send it?
The Spam Tragedy
Picture a common pasture, Ok for 105 sheeps
Ten farmers – each has ten sheep – all is fine
Often sent illegally, from Zombies (spambots)
If one farmer adds five sheeps...
Often contains scams, malware, porn, etc.
Other laws – privacy, copyright, minor-protection,...
He will have 11 sheep, and the pasture – 105 – great!
But if ALL farmers do this 150 sheeps!!
Similar: bandwidth, pollution, IP spoofing…
Marketing & commercial Email could be valuable
Spammers often violate many laws
Anti-Spam Legislation
Spam is an example of the tragedy of the commons
But: Costs of sending Email are negligible
Solutions: impose limits or costs
Technical and/or legal
Special anti-spam legislation
Opt-in – mandatory prior consent to Email
Opt-out
Benefits:
Reduced spam by law-abiding entities
Also: opt-out, `ad` labels easier to filter
Blocking Spam/Phishing: Goals
Spam, Phishing and Email Security
Introduction
Basic defenses
Port blocking
Content Filtering
White/Blacklists/Greylisting
SMTP-Sender Authorization (SPF, SIDF)
Signed Email (DKIM)
DMARC
No significant extra delay
Abuse-free, e.g. do not spam others
Esp.: avoid `framing' (Joe-Job)
Block most spam/phishing emails
Acceptable, `minimal' overhead
Applied by recipients or by mail servers
And : phishing in social networks
Avoid false-positives (lost mail)
Future? [Reputation, Accreditation, Penalties…]
Sending-domain Validation
Do no harm
False-negative phishing emails esp. harmful
Basic Spam Blocking Mechanisms
Content-Filtering
Blacklists [By IP/name]
No `open relays`
Send only via MSA, MTA
5
MTA A
Remote submit: port 587
Otherwise, a.com can be blamed (blacklisted) due to corrupt client
MSA
Alice's MUA Domain a.com
Carl's MUA
MDA MTA X
Bob's MUA Domain b.com
Validate senders
Users: auth, quota
Servers: rDNS, SPF,…
© Gila Kaplan (with permission)
Content Filtering: Inspect message contents to categorize, block spam
MTA B Internet
Block pkts to dst port 25
Content Filtering
Incoming and outgoing messages
By rules, statistics, machine learning [see hidden]
Actions:
Allow
Block entire message
Block/change part, e.g. suspect attachment, url
Label, e.g.: X-Spam-Status: No, score=1.7
Spam Content Filtering – Trends
Very effective against ‘simple’ spam
Identifies spam with removal instructions easily – benefit from anti-spam laws Detect “intentional typos”, etc.
Blacklists (DNSBL)
Viagra suspect spam, Viaggra surely spam
Ultra-short spam: “You must see ”
URL-blacklists become critical
Mimic legit mail (keywords,...)- esp. phishing
Learn to circumvent filtering
Efficient distribution – caching by ISPs
Spammers adapt
Most blacklists use DNS (hence DNSBL)
To query if IP address ww.xx.yy.zz is in list EBL.net
DNS “A” query for zz.yy.xx.ww.EBL.net
DNS addresses: right to left
IP addresses: left to right
Response ww.xx.yy.zz in EBL.net Can reply for addr block: xx.ww.EBL.net
Value can be “Type of Abuse”
Spam comment/blog: immediate feedback
Aggressive Blacklisting
May provide details, e.g. via DNS “TXT”
Criticism of Blacklists
Even when blocking entire `suspect IP prefixes’
Example: Spamhaus Block List (SBL): 12%
Imprecise, negligent, obsolete listings Unfair, hidden interests & agendas Collateral damages
Over-zealousness / censorship:
Not much spam is from known sources
Aggressive blacklisting: beyond “known sources”
XBL: Exploit Block List
Open proxies / relays, Open port 25,…
Block Email with URL to spam site
Example: Spamhaus “URL on SBL”
Source: Spamhaus
Some lists block entire countries!
Blocking open relay / proxy before abuse Blocking “spam supporters”
Fail for new spammer's IP
In particular, route-hijacking
Email connection from IP
Black/grey/white lists
Blacklists: known `bad` senders
Block entire `suspect' address blocks
Motivating ISP/companies to block port 25 (SMTP)
Fails if sending from valid acct at legit mail server
Add IP to greylist (after delay) Temporary Failure Invalid
Whitelist of (domain) names: allows flexibility, requires authentication (DKIM) or authorization of IP (SPF)
IP in blacklist
Is IP in one of the lists?
Validate sender Domain (SPF, DKIM,FCrDNS)
IP in whitelist Valid
N/A Domain in whitelist?
Most spammers are `new`, short lived, & do not resend
Whitelists: known `good` senders (by IP or name)
No
IP in greylist
Greylist: recently / already seen senders, blocked emails
Identify (mainly) by IP of sending server
No Opt: add IP to blacklist
Yes
Fail Content-based Classifier
Ok Ok Block
6
Block
Spam, Phishing and Email Security
Spam, Phishing and Email Security
Introduction
Introduction
Basic defenses
Basic defenses
Port blocking
Content Filtering
White/Blacklists/Greylisting
Sending-domain validation
SMTP-Sender Authorization (SPF, SIDF)
Signed Email (DKIM)
DMARC
Future? [Reputation, Accreditation, Penalties…]
Port blocking
White/Blacklists/Greylisting
Content Filtering
DNS-based Authorization
SMTP-Sender Authorization (SPF, SIDF)
Signed Email (DKIM)
Reputation, Accreditation, Penalties
And : phishing in social networks
SMTP-Sender Authorization
Authorization – Which Domain? Policy record: “Our domain users always send via this MTA(s)”
Email Outgoing server Authorization Policy
E.g.: “Our users send only via these mail servers”
Question: Which domain to query?
Receiver uses policy of sending domain
Naive choice: domain of From: address
Identify IP add of domain’s outgoing mail server
Found, valid sending-MTA's IP: Accept
Found, invalid sending-MTA's IP: reject or tag
No policy for sending domain: no indication
Which domain's policy?
From: address is shown to user
Agent (person / program) who wrote the message
But mailing lists keep “from:” address of Email
Mail-From? HELO? `From'???
Authorization – Which Domain?
Mailing list's MTA not of `From' domain
MTA A
Abe MUA Domain a.com
S: 220 ML.com C: HELO a.com S: 250 OK C: MAIL FROM: [email protected] S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter C: from: [email protected] C: to: [email protected] C: C: New to list
ListOut MTA
List Server
AList-In l MTA ListServ Domain ML.com
S: 220 B.com C: HELO ML.com S: 250 OK C: MAIL FROM: [email protected] S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter C: from: [email protected] C: to: [email protected] C: sender: [email protected] C: C: New …
MTA B
Policy record: “Our domain users always send via this MTA(s)”
Question: Which domain to query?
Naive choice: domain of From: address
Domain b.com
From: address is shown to user
Agent (person / program) who wrote the message
But mailing lists keep “from:” address of Email
SPF (Sender Policy Framework):
Sender-ID (later): Result of PRA algorithm
7
MAIL FROM address and (optionally) HELO address
DSN and MAIL FROM Address
Recall: if dest (B.com) cannot deliver after sending 250 OK:
DSN: Delivery Status Notification (`bounced mail’)
Mail-From also called bounce address
Use mainly when no MAIL-FROM exists (in a DSN) Details: soon
Persistent failure (DSN or direct): remove [email protected]
Sender Policy Framework (SPF)
Authorizes the Email envelope MAIL-FROM
SPF policy: a DNS record
SPF: who can send with given mail-from addr Mail-from: addr for bounce (DSN)
But: Mail-From is Not visible to recipients
A new SPF DNS record, or format for TXT DNS record
Record is for the domain name (no prefix), e.g. x.com
So, what’s the use??
Whitelist MTA of good sending domains
Other uses, e.g., monitoring sending MTAs
SPF Process
b.org's Incoming MTA
a.com's authoritative DNS server ... [email protected]...
b.org's DNS proxy server
SPF RR of a.com, e.g.: v=spf1 +IP:1.2.3.4 ~all Validate that sending MTA's IP (1.2.3.4) is authorized by policy
Yes: continue accepting email
No: reject (break SMTP connection)
Or wildcard: *.x.com
Plus: can define at top-level domain (not SPF.x.com)
Minus: conflict with other TXT records (try SPF first)
SPF policy records kept in DNS
In TXT and/or SPF records
Define which hosts can be Sender-SMTP for domain
Simple Format, e.g. v=spf1 +IP4:133.3.3.3 –all (version 1, 133.3.3.3 is outgoing MTA, none other)
SPFrecord =“v=spf”version terms
Terms= *( SP (([qualifier] mechanism) / modifier )
SPF RR of a.com
SPF of a.com doesn’t exist? … Ask for TXT instead!
Limit length (<450chars), so DNS response is sent w/ UDP
SPF Policy DNS Record
Request SPF RR of a.com
Request SPF RR of a.com
Or HELO address, when no MAIL-FROM exists (DSN)
Phishing: rfc822.from ≠ Mail-From
Sending MTA IP:1.2.3.4
8
SPF policy: a DNS record
Permanent failure: most lists do not send Delivery Status Notification (DSN) to sender [email protected]
Try alternate routes, retransmit on transient errors
Also the HELO address
DSN itself uses empty MAIL FROM address
The “bounce-to” address Used to send Delivery Status Notifications (DSN)
Do not bounce – can cause loop
Using SPF to…whitelist!
Authorizes the Email envelope MAIL-FROM
If list fails to deliver to b.com:
Most deployed outgoing MTA authorization system
Send DSN to MAIL FROM address: [email protected]
If bounce DSN fails to deliver:
Sender Policy Framework (SPF)
Version={1, 2.0}; version 2.0 is actually sender-ID [later]
qualifier: one of: +, -, ~, ?
Mechanism: method of identification, e.g. by IP address
mechanism = ( all / include / A / MX / PTR / IP4 / IP6 / exists )
SPF: Basic Mechanisms (Operators)
SPF Qualifiers
Basic SPF example: v=spf1 +IP4:133.3.3.3 ~all (version 1, 133.3.3.3 is outgoing MTA, none other)
Qualifiers:
+ [default] pass: allow use
– fail: forbid use
? don’t know (Neutral)
“all”
“ip4:” ip4addr [“/” length], “ip6”…(same)
“a” [“:” domain-spec] [“/” length]
“mx” [“:” domain-spec] [“/” length]
PTR = “ptr” [“:” domain-spec]
Domain-spec: a domain name, or macro…
E.g. this IP address allowed to send for this domain
Policy not enforced yet (e.g. experimentation phase)
~ probable fail (SoftFail)
Testing phase
SPF domain-spec & other macros
Many mechanism and modifiers can be macros
Complex syntax… simplified here; see ch. 8 of RFC 4408
domain-spec = *(macro / DomainString)
DomainString = *(alphanum / “.”alphanum)
macro = “%{“ macro-letter [*DIGIT] [‘r’] “}”
macro-letter =
“s” :
“l” : local-part of
“d” :
“i” : (in dot-format)
“h” : HELO/EHLO domain
“r” : domain name of host checking
“t” : current timestamp
`r`: reverse, e.g.: =1.2.3.4 %{ir}=4.3.2.1
Number of parts, e.g.: =1.2.3.4 %{2ir}=2.1
IN TXT “v=spf1 include:a.net include:a.org –all”
If result is…
9
Fail/Neutral: no match;
Error: abort with Error
address specified by MX record for domain-spec
Tests if rDNS for points to domain-spec
Tests for existence of domain-spec
Use for:
Check on blacklist: -exists:%{ir}.DNSBL.org
Generate log to identify use of domain in testing phase, e.g.: v=spf1 exists:%{h}.%{l}.%{i}.log.%{d} ?all
Create fine-grained restrictions, e.g. per-user, per-time,…
Limiting SPF's DNS Queries
SPF validation causes DNS queries:
For policy (TXT and SPF)
Due to some (A, MX, PTR, Include,..) mechanisms
Possible abuse for DoS, DNS-poisoning
Spec limits to 10 mech, 10 queries/mech
`pass` (+) - this is match;
address specified by A record for domain-spec
“exists:” domain-spec
Returns result of applying SPF policy of specified domain E.g., if a.com sends mail via a.net or a.org, it may have:
IPv4 / IPv6 address, e.g. +ip4:6.6.6.0/24
“exists:” Mechanism and its Use
include:domain Mechanism
matches everything (at end)
Still: can cause 100 queries by single email
Each query can cause ~10 retransmits (how?)
Other concern: forwarding
Forwarding with SPF
Forwarding retains Mail-From address
To inform sender (Abe) of problems
Hence: (typical) forwarding breaks SPF [hidden]
Don’t use SPF
include:b.edu or “skip SPF if sender-SMTP is b.edu”
Hence – should authorize sender-SMTP
May differ from originator (From:) identity
Function of message header fields, e.g. From:, Sender:
Criticism: modified use of SPF v=1 records
Idea: Authorize use of MUA-visible identity
Requires defining & displaying appropriate identity
Identify party responsible for sender-SMTP
Then look-up appropriate SPF record
PRA: Purported Responsible Address
Does PRA improve security – or degrade security? From [email protected] on behalf of [email protected]
What is secure now (for [email protected] - outlook user)
If SPF/PRA (Sender-ID) was validated by MDA, then we know “last sender” is really [email protected]
User can’t know whether validation was done!
But anyway…Recipient only cares about originator
Bob trusts [email protected]
Will not always be done – mail server decision
Can Bob trust that message was from [email protected] ?
“Hybrid identity” – to be displayed by mail clients
Spam, Phishing and Email Security
PRA: False Sense of Security? (2)
And/or (less common) other message (RFC 822) headers
PRA: False Sense of Security? (1)
Algorithm: in hidden foil
Use SPF records: existing (SPF v=1) or "spf2.0/pra", "spf2.0/mfrom", "spf2.0/mfrom,pra"
E.g. – mailing list identity (usually in “Sender”)
MAIL-FROM (bounce) – in envelope
How senders can get Delivery Status Notifications (DSN)?
Last “entity” responsible for message transfer
HELO/EHLO domain – in greeting
SIDF (Sender ID Framework – by Microsoft)
Change MAIL-FROM to forwarder (b.edu)
PRA: Purported Responsible Address
IP address – in IP header
Change forwarding
Almost all MUAs identify messages using From:
SPF-compatible forwarding (?)
SPF identifiers discussed so far:
Whitelist forwarders in SPF policy or by receiver
Solutions?
SPF vs. SIDF (Sender ID Framework)
Does PRA improve security – or degrade it?
From [email protected] on behalf of [email protected]
Introduction
Email: refresh, goals, threats, tools
Phishing and spam; tragedy, legal
Suppose SPF/PRA validation was done
And: Recipient (Bob) trusts [email protected]
Port blocking
Can he trust message was from [email protected] ?
Content Filtering
No! Would see this also if b.edu got message from:
White/Blacklists/Greylisting
Mailing list (not using SPF/PRA screening)
Another forwarder (not using SPF/PRA screening)
A spammer pretending to be such a list / forwarder
Misleading – false illusion of security?
Basic defenses
Sending-domain Validation
SMTP-Sender Authorization (SPF, SIDF)
Signed Email (DKIM)
DMARC
Future? [Reputation, Accreditation, Penalties…]
10
And : phishing in social networks
SMTP connection from IP
Email connection from IP No
Add IP to greylist (after delay) Temporary Failure
Validate sender Domain (SPF, DKIM,FCrDNS)
Temporary Failure
IP in whitelist
IP in greylist
Valid
No/bad signature (and sig required) Domain in whitelist? No
Fail Yes
N/A FCrDNS?
Valid
Check DKIM Signature
Valid
Opt: add IP to blacklist Ok
Fail
Content-based Classifier
Domain Keys Identified Mail (DKIM)
Transparent, compatible with existing MUA
Only adds new headers, e.g. DKIM-Signature:
Ignored by legacy clients & servers
No unknown MIME type no questions!
Email signed by:
Outgoing MTA (usually)
MUA (or intermediate MTA)
But could be any DNS entity
Simple key management
Domain in whitelist? Yes Ok
Example of DKIM Signature, Tags DKIM-Signature: a=rsa-sha1; d=example.net; s=brisbane; q=dns; [email protected]; t=1117574938; x=1118006938; h=from:to:subject:date; z=From:[email protected]|To:[email protected]| Subject:demo%20run b=dzdVyOfAKCdLXdJOc9G2q8LoXSlEniSbav+yuU4zGeeruD00lszZVoG
a= algorithm
d= domain
s= selector subdividing namespace for the "d=" (domain) tag
q= key query method(s) (default: dns)
i= `signing address`; email address of the user or agent accountable for message. Domain must be `under` d= domain.
t= Signature creation timestamp
x= Signature Expiration time (default: no expiration time)
h= Signed header fields, z= copied header fields
b= (encoded) signature value
DKIM Key Record
DKIM goal: simple key management
Simple textual, tag-value-list format
Default: fetch public key from DNS (q=dns)
v=; default (and only valid now) is v=DKIM1
Other methods to fetch key are optional (and not specified yet)
DKIM keys stored in “_domainkey” subdomain
DNS records, type: TXT or DKK
DKK: New binary DNS type
Binary – to allow large keys in single DNS UDP packet (<512B)
Try to get first – if fails, get TXT record
g=; local-part of the signing address (i=) specified in the signature
DKIM signatures identify PK by domain (d=) and selector (s=)
Given DKIM-Signature: d=example.net; s=brisbane…, the DNS query will be for brisbane._domainkey.example.com.
Block
Block
DKIM Key Management
11
No
Ok
Block
Block
Pass
N/A
Fail Content-based Classifier Ok
IP in blacklist
Is IP in one of the lists?
Unauthorized IP (SPF hard fail) Domain authorized Yes this IP? (SPF)
N/A
Opt: add IP to blacklist
No
Add IP to greylist (after delay)
IP in whitelist
IP in greylist
Invalid
IP in blacklist
Is IP in one of the lists?
Use * for wildcard, e.g. g=Alice+* (for [email protected]).
Hash and signature algorithms (h=, k=)
p=
(more: hidden foil)
DKIM: DNS Security Issues
DKIM retrieves public key using DNS (if q=dns)
Other methods to fetch key are optional (and not specified yet)
`Domain Policy Records` and Secure DNS
Problem 2: attacker blocks DNS response
How we know a domain uses DKIM?
Problem 1: DNS poisoning (fake response)
`Excuses` (in spec): complex attack, limited damage…
Solution: use Secure DNS (SEC-DNS)
Problem 2: attacker blocks DNS response
How we know a domain uses DKIM?
DKIM ADSP Process a.com's authoritative DNS server
b.org's Incoming MTA
... [email protected]...(no DKIM-Signature)
What if attacker blocks ADSP?? Secure DNS server signs response range: We get proof of no record for Alice, Bob!
DKIM and Email Mangling (1) b.org's DNS proxy server
Request ADSP RR of a.com
Request ADSP RR of a.com ADSP RR of a.com: all my email has DKIM-Signature !
ADSP Reject: email from a.com must have DKIM-Signature
Mail agents may modify (“mangle”) email:
Add header
Insert “white space”, e.g.
Add prefix and/or suffix text (“scanned by…”)
Cryptographic authentication:
Sensitive to any change in content
Validatev(m’, Signs(m)) = False
For any m’m, even m’mangle(m)
Solutions in DKIM [hidden]
Scope limitations
Canonicalization
Spam, Phishing and Email Security
Email: refresh, goals, threats, tools
Phishing and spam; tragedy, legal
Basic defenses
DMARC: Domain-based Message Authentication, Reporting & Conformance Addresses problems with DKIM+SPF:
Port blocking
Content Filtering
SPF fails w/ forwarding, DKIM with mangling…
White/Blacklists/Greylisting
DMARC: do both!
Report authentication-failures
Sender policy on handling failures
Reduces false positives usable!!
Sending-domain Validation
SMTP-Sender Authorization (SPF, SIDF)
Signed Email (DKIM)
DMARC
Future? [Reputation, Accreditation, Penalties…]
12
DMARC
Introduction
ADSP says: This domain always uses DKIM (cf. SPF)
Signac.il.s(from=“abe”,fkey=a8us…,to=“carl”,tkey=acjk7 …)
Validation fails… false positive / negative?
Sending MTA IP:1.2.3.4
To ignore unsigned (fake) messages from it ADSP (author domain signing practices) in DNS
Validation fails… false positive / negative?
And : phishing in social networks
Identify attacks & problems, resolve problems
DMARC Policy Yet-another
DNS record: TXT _dmarc.xxx.com
DKIM adoption
Receivers should:
Verify alignment (DKIM signature Ok)
Verify authorized sender (SPF):
From header domain = SMTP MAIL FROM domain
Senderʼs IP address is validated by SPF.
If either true: email Ok by DMARC (`pass’)
If both fail: follow DMARC failure policy
Block, quarantine, report
SPF adoption DMARC adoption
©Lars Eggert Thanks Lars!
Phish-or-Real Classification
Phish-or-Real Classification Content classifiers work well for most ad-spam
Classification to three types:
Keywords, URLs, lots of learning samples,...
Challenge: classify to `Phish' vs. `non-phish`
Phishing messages, sites mimic legit messages, sites
Very different phishing (& legit) styles
Often few samples (esp. `spear phishing')
Identified spam/phishing emails
`PhishOrReal` (of foo.com)
Other emails
Phish/spam
Idea: combine classifier with auth (SPF,DKIM,DMARC...)
classify as `Phish-or-Real' of foo.com, `other'
Phishing: `Phish-or-Real`of foo.com
Block
If `real' foo.com always passes SPF/DKIM/IP
Email: refresh [skip], goals, threats, tools
Phishing and spam
Basic defenses
Port blocking
Content Filtering
White/Blacklists/Greylisting
SMTP-Sender Authorization (SPF, SIDF)
Signed Email (DKIM)
DMARC
Future? [Reputation, Accreditation, Penalties…]
And : phishing in social networks
Add IP to blacklist with prob. p
Yes
foo.com always identified by IP, DKIM or SPF?
No
Sender authentication (DKIM etc.) allows white-list
What about unknown & un-trusted senders?
Reputation and Accreditation:
Authorization, Signing, Policies
Other
Content-based classifier
Display
Beyond Authentication
Introduction
Email not from bank (e.g.: bank always signs)
PhishOrReal from foo.com
Spam, Phishing and Email Security
13
Senders provide certificates (“recommendations”) from trusted authority
Reputation: Based on history of sender
Accreditation: Sender pays or deposits at authority
Cost-based:
Postage: Pay for each message
Penalty: Pay only for spam
Several services, limited deployment / success [hidden foils]
Who is Guarding the Guards?
Can we trust reputation & accreditation services?
Accreditation services charge the senders
Incentive to be permissive
Financial Penalties for Spam
Also some reputation services Allow (some) customers to spam
False positives also possible (if not signed, or by Zombie)
“Denial of reputation” – bad-mouthing attack “I got spam from xxx” - maybe a lie? Or due to spoofing?
Sender/server digitally signs mail and label
Recipient may select content based on labels
Penalize if label is incorrect
In following foils: label {‘OK’, ‘spam’}
Penalties may be:
Interoperable, `open` accreditation services?
Loss of trust / reputation
Solutions:
Financial compensation / fine (money)
Proofs of spamming, not just complaints
Use well-defined spam: Email without “warning” label Compensation to victim
Automated / secure resolution process
Penalty to spammers
Bootstrapping, “do no harm”…
SocNet Spam & Phishing
SocNet: known users, accountabtiliy… but…
~20% of malicious `friend-req’ approved
Most req’ to honey-pot accts from real users!
Great info for targeted spam/phishing
~45% of links posted by `friends’ clicked
Much better than email spam/phishing
Huge vulnerability!!
Conclusions
Challenges:
Spam and phishing are major problems Combine authentication, reputation and classification to filter spam and phishing Appropriate combination can make phishing unprofitable Still very important area
14
Spam mail with incorrect label
With research challenges, too
Proposals [e.g., SeARP – hidden], none deployed
Social Network Malware: KoobFace
