gecko-dev/grendel/storage/selftestdata/Inbox

From - Mon Jun 10 09:11:27 1996
Delivery-Date: Fri, 07 Jun 1996 19:41:36 -0700
Return-Path: list@glacier.mcom.com
Received: from glacier.mcom.com (glacier.mcom.com [205.217.233.39]) by dome.mcom.com (8.6.10/8.6.9) with ESMTP id TAA26191 for <terry@dome.mcom.com>; Fri, 7 Jun 1996 19:41:33 -0700
Received: (from list@localhost) by glacier.mcom.com (8.7.3/8.7.3) id TAA01443; Fri, 7 Jun 1996 19:38:52 -0700 (PDT)
Resent-Date: Fri, 7 Jun 1996 19:38:52 -0700 (PDT)
To: clienteng@netscape.com
Path: usenet
X-UIDL: 834201705.000
From: Jamie Zawinski <jwz@netscape.com>
Newsgroups: mcom.dev.clienteng
Subject: Re: Email bug was [Fwd: Apologies and complaints]
Date: Fri, 07 Jun 1996 18:38:41 -0700
Organization: Netscape Communications Corporation, Mozilla Division
Lines: 280
Message-ID: <31B8E7B1.41C6@netscape.com>
References: <31B8AEBF.4A7B@netscape.com> <ML-2.3.834195020.8606.max@jimi-hendrix.mcom.com>
NNTP-Posting-Host: gruntle.mcom.com
Mime-Version: 1.0
Content-Type: multipart/digest; boundary="------------167E2781446B"
X-Mailer: Mozilla 3.0b5 (X11; U; IRIX 5.3 IP22)
To: Mike Macgirvin <MAX@netscape.com>
Content-Disposition: inline
Resent-Message-ID: <"lghyh3.0.VM.wUEkn"@glacier>
Resent-From: clienteng@netscape.com
X-Mailing-List: <clienteng@netscape.com> archive/latest/1926
X-Loop: clienteng@netscape.com
Precedence: list
Resent-Sender: clienteng-request@netscape.com
Status: U
X-Mozilla-Status: 8010
Content-Length: 12874

This is a multi-part message in MIME format.

--------------167E2781446B
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Mike Macgirvin wrote:
> 
>         I guess I need to find out why this is an issue (for us). We are doing
> "proper" delivery using native methods.

So that means that on Unix, you use sendmail (wearing its MDA hat
instead of its MTA hat) to move messages into /var/spool/mail/$USER ?

I think that's a totally reasonable thing to do.

> Seems to me that movemail needs fixing to deal with Solaris'
> Content-Length, or replaced with something that can handle it. Yes,
> we can hack something in on this side too, but
> 
> Mlocal,       P=/bin/mail, F=EflsSDFMmnP, S=10, R=20, A=mail -d $u
> 
> isn't gonna' fly with our customers. A check box that turns on "From
> stuffing" "...IF you use Unix delivery and IF you are using a Solaris
> 2.x OS and IF you have Netscape Navigator users accessing the local
> spool..." isn't very clean either.

(Well first off, fixing that is really hard, which is why I didn't just
do it.)

But I think, actually, that telling them that they need to configure
sendmail in that was *does* fly, because *most* people on Solaris do
that anyway -- the simple fact of life is that almost all Unix MUAs
expect /var/spool/mail/$USER to be in BSD Mailbox format, not in
Content-Length format.  It's not just Netscape.  If any Solaris user
wants to use a mailer other than one that Sun ships, they have to
reconfigure sendmail in this way.

> Fixing movemail is a much better solution.

That would be a nice solution, but you can't look at /var/mail/ and know
which of the two formats its in, so you need a new user option in the
client.  And its one that no end-user is going to know how to set.

And it only fixes the "Get Mail" case.  The thing that sucks most about
this, actually, is not the "Get Mail" case, but the fact that both Sun's
MailTool and Eudora write folder files using the Content-Length format;
they don't do From mangling.  So that means that importing folders is
dicey -- every now and then, a message will get split.

> 'Nuther Q... (Jamie? and I really don't know the answer to this...)
>         Does Nav itself parse From lines? Does it actually rely on
> stuffing in its parsing algorithms? Or is this strictly a movemail
> issue?

Movemail depends on From-mangling, because it assumes BSD format.
Mozilla has movemail code built-in, and also can use an external
program to do this.  

But, Mozilla *also* uses BSD format for its saved folders.  So if it's
pointed at a folder written by another program, and that other program
uses the Content-Length format, messages might get split.

> Remember, we don't need stuffing on POP/IMAP delivery. Do we
> need to have it to make Nav work right? 

No; this isn't an issue for Get-Mail-using-POP3, only for
Get-Mail-using-movemail (and for folders already on disk.)

> What are you going to do when you get a message from a Microsoft
> Exchange server which doesn't need From stuffing and doesn't do it?

Responses from POP3 servers need not be mangled; those are terminated by
`CRLF . CRLF' so there's no ambiguity (and then when we write that
message to disk, we write it in our native folder format, which is to
say, we mangle the internal From lines.)

> Tell them their server is broken? This isn't strong ground to be
> standing on...because their server may in fact not be. Nothing says
> that messages have to be stuffed for transport (except in the case of
> \r\n.\r\n via SMTP). It's purely a Unix storage issue. 

Yes, I agree.  We don't have to tell them their server is broken,
because Get-Mail-via-POP3 isn't a problem for us.

In fact, in the Get Mail case, it's only an issue on Solaris, and only
when using movemail.  All other Unix platforms on which we ship come
with MDAs which use the BSD format, not the Content-Length format. 
Solaris can do either, but comes configured for Content-Length.

Why is supporting Content-Length hard?

  = You can't look at a file and tell whether it's in BSD format or
    Content-Length format (the presence of the CL headers doesn't 
    actually tell you anything.)

  = If you're in a world where you can't really make assumptions about
    how many bytes a line terminator is (as all of our code is, since
    so far we've managed to support input with CR, LF, and CRLF on all
    platforms, regardless of which is native) you will often find
    yourself faced with Content-Length lines that no longer match, so
    your parser has to do sophisticated error recovery.

  = We are currently writing out-of-spec BSD files anyway -- we are
    treating "\nFrom " as a delimiter, when really we should be 
    treating "\n\nFrom " as a delimiter.  This messes with the lengths,
    so that would have to be fixed first (there's a bug about it.)

More background:

--------------167E2781446B

Message-ID: <319CEA7A.7A79@netscape.com>
Date: Fri, 17 May 1996 14:07:06 -0700
From: Jamie Zawinski <jwz@netscape.com>
Organization: Netscape Communications Corporation, Mozilla Division
X-Mailer: Mozilla 3.0b4 (X11; U; IRIX 5.3 IP22)
MIME-Version: 1.0
Newsgroups: comp.mail.headers
Subject: Re: "From_" specification
References: <1996May16.171931.26752@jarvis.cs.toronto.edu> <4nhcr1$7sf@lyra.csx.cam.ac.uk>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Philip Hazel wrote:
> 
> In article <1996May16.171931.26752@jarvis.cs.toronto.edu>, edwin@cs.toronto.edu (Edwin Allum) writes:
> |>
> |> Is there a standard specification (rfc) or format for the "From_" line
> |> used to delimit messages in the mail spool file?
> 
> There is some discussion of From_ lines in RFC 976 ("UUCP Mail
> Interchange Format Standard").

RFCs specify internet protocols, that is, on-the-wire formats.  The
thing that the original poster is looking for is a description of the
BSD Mailbox file format (which is not something an RFC would cover.)

But, here's the good news, there *is* no true specification of this file
format, just a collection of word-of-mouth behaviors of the various
programs over the last few decades which have used that format.

Essentially the only safe way to parse that file format is to consider
all lines which begin with the characters "From " (From-space), which
are preceeded by a blank line or beginning-of-file, to be the division
between messages.  That is, the delimiter is "\n\nFrom .*\n" except for
the very first message in the file, where it is "^From .*\n".

Some people will tell you that you should do stricter parsing on those
lines: check for user names and dates and so on.  They are wrong.  The
random crap that has traditionally been dumped into that line is without
bound; comparing the first five characters is the only safe and portable
thing to do.  *Usually*, but not always, the next token on the line
after "From " will be a user-id, or email address, or UUCP path, and
*usually* the next thing on the line will be a date specification, in
*some* format, and *usually* there's nothing after that.  But you can't
rely on any of this.

In the BSD format, the only safe way to add a message to a file is to
mangle occurences of the "From " delimiter in the body of messages to
some other string, usually ">From ".  This is mangling, not quoting,
because it's not a reversible process (since ">From " is not also
quoted.)

Now, there are actually two very similar-looking file formats.  One is
the BSD format, which I've described.  The other, which one might as
well call the "content-length" format, is used by some SYSV-derived
systems, notably Solaris.  It's very similar, but subtly incompatible.
This format does not quote "From " lines, but instead relies on a
Content-Length header in the message proper to indicate the exact
byte-position of the end of each message.  

This latter format is non-portable, easily-corruptible, and overall,
brain-damaged (that's a technical term.)  But I'll refrain from ranting
about it again right now...

-- 
Jamie Zawinski    jwz@netscape.com   http://www.netscape.com/people/jwz/
``A signature isn't a return address, it is the ASCII equivalent of a
  black velvet clown painting; it's a rectangle of carets surrounding
  a quote from a literary giant of weeniedom like Heinlein or Dr. Who.''
                                                         -- Chris Maeda

--------------167E2781446B

Message-ID: <319D3B7A.6201@netscape.com>
Date: Fri, 17 May 1996 19:52:42 -0700
From: Jamie Zawinski <jwz@netscape.com>
Organization: Netscape Communications Corporation, Mozilla Division
X-Mailer: Mozilla 3.0b4 (X11; U; IRIX 5.3 IP22)
MIME-Version: 1.0
Newsgroups: comp.mail.headers
Subject: Re: "From_" specification
References: <1996May16.171931.26752@jarvis.cs.toronto.edu>
		<4nhcr1$7sf@lyra.csx.cam.ac.uk> <319CEA7A.7A79@netscape.com> <xcdvihuyjsq.fsf@woodlawn.uchicago.edu>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Soren Dayton wrote:
> 
> Our friend, Jamie Zawinski <jwz@netscape.com>, wrote:
> 
> > Now, there are actually two very similar-looking file formats.
> 
>  <snip>
> 
> > This latter format is non-portable, easily-corruptible, and overall,
> > brain-damaged (that's a technical term.)  But I'll refrain from ranting
> > about it again right now...
> 
> So this is to claim that the one style is portable easily-corruptible,
> brain-damage while the other is non-portable?
> 
> letter to a file, very hard to forget to quote the '^From_'s

I'm not sure exactly what you're trying to say, but I'll clarify what I
meant: I'm not saying that the BSD Mailbox format is *good*.  Just that
the Content-Length variant of that format is *worse*.

Ok, so someone took the From_ format, and extended it to not require
mangling by adding a length indicator to the format.  At first glance,
this may sound simple and elegant, but it breaks the world, and one
shouldn't encourage its use to spread.

The thing that breaks is taking an existing, widely-implemented format,
and *adding* a requirement that it have a length indicator.  This means
that any existing software that already *thinks* it knows how to
manipulate that format is going to damage the file (any change to the
data will cause the length indicator to be wrong w.r.t.  the *new*
specification but not w.r.t. the *old* specification.)

If the content-length-based format was not otherwise-indistinguishable
from the "From " format, there wouldn't be a problem; the old software
would simply fail to work with this new file format, instead of 
"corrupting" the documents (in quotes, because it's really just a 
matter of which spec you're following.)

Also, mailboxes are by their nature a textual format; but, the
content-length header measures in *bytes* rather than lines.  This
means that if you move the file to a system which has a different
end-of-line representation (Windows <=> Mac, or Windows <=> Unix) then
the content-lengths will suddenly be wrong, because the linebreaks now
take two bytes instead of one, or vice versa.

It's impossible for a mail client to look at a file, and tell which of
the two formats (From_ or Content-Length) it is in; they are
programatically indistinguishable.  The presence of a Content-Length
header is not enough, because suppose you were on a system which knew
nothing at all about that header, and some *incoming* message just
happened to have that header in it.  Then that header would end up in
your mailbox (because nobody would have known to remove or recalculate
it), and it would possibly be incorrect.  (Presume further that the
header was not just incorrect, but intentionally malicious...)

Stricter parsing of the "From " seperator line doesn't help either,
because there are many, many variations on what goes in that line
(since it was never standardized either); and also, some mail readers
include that line verbatim when forwarding messages (Sun's MailTool,
for example) so a stricter parser wouldn't help that case at all,
because message bodies tend to contain "valid" matches.

Some mail readers attempt to cope with this by recognising the case
where the Content-Length is not obviously spot-on-target, and then
searching forward and backward for the nearest message delimiter; but
this is obviously not foolproof, and makes one's parser much more
inefficient (requiring arbitrary lookahead and backtracking.)

Conventional wisdom is, ``if you believe the Content-Length header,
I've got a bridge to sell you.''

-- 
Jamie Zawinski    jwz@netscape.com   http://www.netscape.com/people/jwz/
``A signature isn't a return address, it is the ASCII equivalent of a
  black velvet clown painting; it's a rectangle of carets surrounding
  a quote from a literary giant of weeniedom like Heinlein or Dr. Who.''
                                                         -- Chris Maeda


--------------167E2781446B--


From - Wed Dec 03 07:58:41 1997
Path: pixie.mcom.com!news
From: Eric Mader <emader@netscape.com>
Newsgroups: mcom.humour
Subject: [Fwd: Luggage]
Date: Wed, 19 Nov 1997 10:32:22 -0800
Organization: Netscape Communications Corporation
Lines: 60
Message-ID: <347330B6.42D3CDD8@netscape.com>
NNTP-Posting-Host: 208.12.37.141
Mime-Version: 1.0
Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms691B6036709EBB7DE811477F"
X-Mailer: Mozilla 4.01 [en] (WinNT; U)
X-Priority: 3 (Normal)
X-Mozilla-Status: 8009

This is a cryptographically signed message in MIME format.

--------------ms691B6036709EBB7DE811477F
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

<forwards have been stowed in the overhead compartment>
-----------------------------------------------------------------------
As migration approached, two elderly vultures doubted they could make
the trip south, so they decided to go by airplane.

When they checked their baggage, the attendant noticed that they were
carrying two dead raccoons.  "Do you wish to check the raccoons
through as luggage?" she asked.

"No, thanks," replied the vultures.  "They're carrion."
--------------ms691B6036709EBB7DE811477F
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIIHewYJKoZIhvcNAQcCoIIHbDCCB2gCAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3DQEHAaCC
BWwwggKrMIICFKADAgECAgIIOjANBgkqhkiG9w0BAQQFADB3MQswCQYDVQQGEwJVUzEsMCoG
A1UEChMjTmV0c2NhcGUgQ29tbXVuaWNhdGlvbnMgQ29ycG9yYXRpb24xHDAaBgNVBAsTE0lu
Zm9ybWF0aW9uIFN5c3RlbXMxHDAaBgNVBAMTE3Jvb3RjYS5uZXRzY2FwZS5jb20wHhcNOTcx
MTE0MDEwMDE0WhcNOTgwNTEzMDEwMDE0WjCBhjELMAkGA1UEBhMCVVMxJjAkBgNVBAoTHU5l
dHNjYXBlIENvbW11bmljYXRpb25zIENvcnAuMRMwEQYDVQQDEwpFcmljIE1hZGVyMSIwIAYJ
KoZIhvcNAQkBFhNlbWFkZXJAbmV0c2NhcGUuY29tMRYwFAYKCZImiZPyLGQBARMGZW1hZGVy
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDMFuyovDlaYG44cW2wAEV66eUqhC4RAKOC
76xkYy7c0ZvV2B2RDYeTGqw6ZBJotuY/h6tztgOX61lz2O6ViYl2NRUq+Fl2bbY2rq+QXsLX
/04TveXoxpQMnY0bYe8nOfAUxj7pRNXG2CfWc/QVFti+yg6XwgCmTo1xmQjq5lhf9wIDAQAB
ozYwNDARBglghkgBhvhCAQEEBAMCAKAwHwYDVR0jBBgwFoAU/OBU6Afxld4695nGrvoVDG7E
LpIwDQYJKoZIhvcNAQEEBQADgYEAP/kXpuPBjKW/pypwPfwJYxfmTgaKLcZAjZ8YpbIJEeiU
rY3/uSofuVWhjx6TpMigY8v3o5ufQi1ge/1ERIG3GjbjHhcUUINRU2Z/+a7+80nadciAQV+n
jsJm47w9jDA8vQxazDIz6M720malCHU8xRxkWxjBxX4jxhdkK48iVdMwggK5MIICIqADAgEC
AgEBMA0GCSqGSIb3DQEBBAUAMHcxCzAJBgNVBAYTAlVTMSwwKgYDVQQKEyNOZXRzY2FwZSBD
b21tdW5pY2F0aW9ucyBDb3Jwb3JhdGlvbjEcMBoGA1UECxMTSW5mb3JtYXRpb24gU3lzdGVt
czEcMBoGA1UEAxMTcm9vdGNhLm5ldHNjYXBlLmNvbTAeFw05NzAzMjYwMTQ0MzhaFw05OTAz
MjYwMTQ0MzhaMHcxCzAJBgNVBAYTAlVTMSwwKgYDVQQKEyNOZXRzY2FwZSBDb21tdW5pY2F0
aW9ucyBDb3Jwb3JhdGlvbjEcMBoGA1UECxMTSW5mb3JtYXRpb24gU3lzdGVtczEcMBoGA1UE
AxMTcm9vdGNhLm5ldHNjYXBlLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAwao+
/i0/pYfDR9/72m3YGKBFPfnHm0POJ7RaE5wRfb/S8ohex7+yi3m6p+UoC0CmjpkxVcX4zpYG
XiKEdr8BImLDqZknuwhoERTHCn7csof4x+AkMAG8LZaF5xnDLqGTdyw0GC/736JIs+egr3oD
5IuMdaQtkyCMIDlUp0W6QGUCAwEAAaNVMFMwEQYJYIZIAYb4QgEBBAQDAgAEMB0GA1UdDgQW
BBT84FToB/GV3jr3mcau+hUMbsQukjAfBgNVHSMEGDAWgBT84FToB/GV3jr3mcau+hUMbsQu
kjANBgkqhkiG9w0BAQQFAAOBgQBZ99sbXHoGxObFmGGEGM76BksgsSTK/Fl+Pxjx5L6sENlK
0mmPbvyRyvUEHAquufrKOexNABmmZ5TM5UBbWYQkkvABLBnkCy87HPYPG4VF7MOX8eC6QMvd
V3GJ4ItJcEkf3bbLNG9vzy8h5FPRGWaPZ2Lw3e4dSCrwR3uDdId5yDGCAdcwggHTAgEBMH0w
dzELMAkGA1UEBhMCVVMxLDAqBgNVBAoTI05ldHNjYXBlIENvbW11bmljYXRpb25zIENvcnBv
cmF0aW9uMRwwGgYDVQQLExNJbmZvcm1hdGlvbiBTeXN0ZW1zMRwwGgYDVQQDExNyb290Y2Eu
bmV0c2NhcGUuY29tAgIIOjAJBgUrDgMCGgUAoIGxMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0B
BwEwIwYJKoZIhvcNAQkEMRYEFFagObyiR2QAb/2AgG/uvXOz13ByMBwGCSqGSIb3DQEJBTEP
Fw05NzExMTkxODMyMjJaMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcN
AwICAgCAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgFAMA0GCCqGSIb3DQMCAgEoMA0GCSqGSIb3
DQEBAQUABIGAD7qsSJAuy1NeHQJOQphJ3Zeo0rvtieSNPDK2mf1MzIDTjD1wk1zyd9wImzFw
Kt99mxZgvmA6Ynqh32zbl2eB0MnZ7CHV2cYAN8Dto+DwKopQvKhTLgpFIWHtqK3jHFiew9JI
EZxq6VIuj+09gq6XaSBOYWLwGcdSr6ZegqXfM2I=
--------------ms691B6036709EBB7DE811477F--

From - Wed Dec 03 08:00:37 1997
Return-Path: <terry@dome.mcom.com>
Received: from dome.mcom.com ([208.12.42.33]) by terryw.mcom.com
          (Netscape Messaging Server 3.01)  with ESMTP id 400
          for <terry@terryw.mcom.com>; Tue, 2 Dec 1997 11:29:00 -0800
Received: from dome.mcom.com ([127.0.0.1]) by dome.mcom.com
          (Netscape Mail Server v2.02) with SMTP id AAA18756
          for <terry@terryw>; Tue, 2 Dec 1997 11:29:00 -0800
Return-Path: list@glacier.mcom.com
Return-Path: <list@glacier.mcom.com>
Received: from glacier.mcom.com ([205.217.233.39]) by dome.mcom.com
          (Netscape Mail Server v2.02) with ESMTP id AAA18742
          for <terry@dome.mcom.com>; Tue, 2 Dec 1997 11:28:58 -0800
Received: (from list@localhost) by glacier.mcom.com (8.7.3/8.7.3) id LAA09662; Tue, 2 Dec 1997 11:28:46 -0800 (PST)
Resent-Date: Tue, 2 Dec 1997 11:28:46 -0800 (PST)
Message-ID: <3484604F.AED7FEED@netscape.com>
Date: Tue, 02 Dec 1997 11:23:59 -0800
From: dcasados@netscape.com (Debbie Casados)
Reply-To: dcasados@netscape.com
Organization: Netscape
X-Mailer: Mozilla 4.02 [en]C-NSCP  (Win95; U)
MIME-Version: 1.0
To: stp-all@netscape.com, cpd-all@netscape.com, spd-all@netscape.com
CC: pdhr@netscape.com, beer@netscape.com, campsey@netscape.com
Subject: SITN REGISTRATION INFO
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"H4WE_2.0.CM2.W56Xq"@glacier>
Resent-From: cpd-all@netscape.com
X-Mailing-List: <cpd-all@netscape.com> archive/latest/63
X-Loop: cpd-all@netscape.com
Precedence: list
Resent-Sender: cpd-all-request@netscape.com
X-Mozilla-Status: 8001

<HTML>
I wanted to update you on another developmental opportunity that Netscape
is sponsoring for extended education.&nbsp; SITN, Stanford Instructional
Television Network, will be broadcast to the Netscape campus beginning
in the winter quarter.&nbsp; This is an opportunity for individuals to
enroll in Stanford's School of Engineering to study for a technical, graduate
degree.

<P><B>Registration for non-degree classes and credit ends on <U>December
5</U> while registration for auditing courses ends on <U>Dec. 19th.</U></B>

<P>We will be having an information session tomorrow, <B><FONT COLOR="#CC0000">December
3, from 11-12pm in building 14 Grab-n-Go area.</FONT></B>&nbsp; If you
are interested in learning more about SITN please drop by the session.&nbsp;
If you would like to register for SITN please check out Stanford's SITN
website.&nbsp;<A HREF="www.SCPD.stanford.edu"> SCPD.stanford.edu</A>

<P>--
<BR>Debbie Casados
<BR>HR Director, Product Development
<BR>415-937-3883
<BR>Fax 415-938-9039
<BR>&nbsp;</HTML>