[LINK] More than you want to know about headers (was: Header overkill)

[Glen Turner]

Hi Roger,

The headers are best through of in the spirit of writing
on an envelope:

  - Some you write.
      Snail-mail: Address, From.
      E-mail: To, Subject.

  - Some are added by the post office but are of use
    to the recipient.
      Snail-mail: Date and place on postmark.
      E-mail: Date, X-Spam-Score.

  - Some are added by the post office but may as well
    be heiroglyphic to most recipients
      Snail-mail: barcodes.
      E-mail: Received.

In a silly design error, the header information in a received
message actually comes from two sources:

  1) The To and From are transmitted in the SMTP protocol
     (MAIL FROM, RCPT TO).  In another silly error, we
     called this the "envelope".  In a third silly error,
     the protocol wasn't expandable to other envelope
     items.

  2) The remaining information about the e-mail is sent
     as the very first items in the e-mail message,
     ending in a blank line.  We named these the
     message "headers".  The receiving e-mail server
     adds the "envelope" items to the "headers".

As a worked example, I'll send you a e-mail.  Firstly,
we'll find your mail server by asking DNS for a "Mail
eXchanger" record for your domain:

    $ dig mx xamax.com.au
    0 mail.apex.net.au
    20 mail2.apex.net.au

We try the lowest-numbered MX first, the others are backups.
SMTP is assigned TCP port 25, so we start our SMTP transaction
(my machine is yoyo):

    $ telnet mail.apex.net.au 25
    HELO yoyo.aarnet.edu.au

We'll now send the "envelope":

    MAIL FROM: glen.turner at aarnet.edu.au
    RCPT TO: roger.clarke at xamax.com.au

We'll now start the e-mail message, with the "headers"
as the first lines of the message:

    DATA
    Subject: SMTP is simple

We'll press Enter again for a blank line to mark the
end of the headers and send the text of our e-mail:

       [blank line]
    Hello Roger,
    SMTP is a purely textual protocol, so telnet is the most basic
    e-mail client.
    Regards, Glen.

A period on a line by itself marks the end of the e-mail
body.  We then close the connection

    .
    QUIT

 > (1)  why are headers headers, and not footers?  Is it because email
 >      standards only allow for headers and not footers?

E-mail doesn't have footers.  Although increasingly e-mail
clients are treating text following a line with "-- " as
a signature block.  Given massive length of some disclaimers
some e-mail clients will render this text in shaded tones.

Some e-mail clients also interpret textual markup such
as *bold*, _underline_ and :-).

> If so, they are *record*-headers, but does that mean that they
 > have to be rendered in the first few lines of the display?
 > That's a decision by the designers of email-clients presumably;

Yes.  Most clients supress headers.  Some mailers annoyingly
quote all headers when forwarding a message, often becuase those
mailer client authors don't believe in MIME-formatted messages.

   [An old argument dating back to the days when sending a MIME
    message to a user with a plain text mail client produced the
    sort of responses that e-mailing HTML to some people now
    produces.]

> (2)  why can't they be suppressed in the default display, and only
>      visible when the user selects the display of full header information?

Most clients do.  Like most user interface design, things
are more subtle then they at first appear.  For example,
when saving a e-mail to a file do you save the displayed
headers or all of the headers?  The most correct answer
is probably to save all the headers, but listing the
displayed headers first (thus making it easy to delete the
trash headers).

-- 
  Glen Turner                (08) 8303 3936 or +61 8 8303 3936
  Australian Academic and Research Network   www.aarnet.edu.au