Replace mbox format driver with mboxrb

* NEWS: update

* configure.ac: Revert 9763fd4a.
* include/mailutils/registrar.h: Likewise.
* libproto/Makefile.am: Likewise.

* libmailutils/stream/stream.c (mu_stream_size): Fix size calculation
in case of line and full buffering scheme.
* libmailutils/stream/streamcpy.c (mu_stream_copy_nl): Change semantics.
Ensure that the copied data ends with two newline characters.  Append
them if necessary.

* testsuite/mbop.c: New commands: message_lines and message_size.

* libproto/mbox/mboxrb.c (mboxrb_mailbox_init_stream): Always add
MU_STREAM_READ permission.
Use the same flags for mu_mapfile_stream_create and
mu_file_stream_create.
(mboxrb_rescan_unlocked): Count terminating newline as part of the
message.
(mailbox_append_message): Make sure each message ends with an empty
line.
(mboxrb_tracker_sync): Update mailbox mesg_count.
(mboxrb_flush_temp): Truncate the temporary stream to the new size.
(mboxrb_flush_unlocked): Clear all modifications only in FLUSH_UIDVALIDITY
mode.
(mboxrb_message_copy_with_uid): Fix message_end computation.  Use
mu_stream_copy_nl to copy message body.
(mu_mboxrb_message_reconstruct): Update ref on return.
* po/POTFILES.in: Update.

* mail/copy.c (append_to_file): Use mu_stream_stat_buffer to compute
statistics.
* mail/mail.c: Allow for opening read-only mailboxes.

* libproto/mbox/tests/body.at: Fix the expected byte count.
* libproto/mbox/tests/qget.at: Likewise.
* libproto/mbox/tests/delete.at: Add more tests.

* mail/tests/align.at: Fix the expected byte and line count.
* mail/tests/hold.at: Likewise.
* mail/testsuite/mail/read.exp: Likewise.
* mail/testsuite/mail/tag.exp: Likewise.
* mail/testsuite/mail/z.exp: Likewise.
* mail/tests/copy04.at: Don't use line count and size in comparison.
* mail/testsuite/mail/write.exp: Use regexps instead of exact strings
for the same reason.

* pop3d/testsuite/pop3d/read.exp: Fix the expected byte count.

* readmsg/tests/all.at: Expect extra newline in output.
* readmsg/tests/hdr.at: Likewise.
* readmsg/tests/nohdr.at: Likewise.
* readmsg/tests/twomsg.at: Likewise.
* readmsg/tests/weed.at: Likewise.

* sieve/tests/redirect.at: Fix the expected byte count.
* sieve/tests/reject.at: Expect extra newline in output.

* mh/inc.c (incmbx): Parse truncate and nomoveto URL parameters prior
to opening the mailbox.  If notruncate is requested, open the mailbox
read-only.
* mh/mh_format.c (str_compress_ws): Fix copying between overlapping
memory regions.

* libtests/Makefile.am: Remove lstuid tests.
* libtests/testsuite.at: Likewise.
* libtests/lstuid.c: Remove.
* libtests/lstuid00.at: Remove.
* libtests/lstuid01.at: Remove.
* libtests/lstuid02.at: Remove.
* libtests/mime.at: Fix the expected byte count.

* imap4d/tests/IDEF0956.at: Fix the expected byte count.
* map4d/tests/fetch.at: Likewise.
* imap4d/tests/append00.at: Remove X-* headers and Status from the mbox
prior to comparison.
* imap4d/tests/append01.at: Likewise.

* libproto/mbox: Remove old code.  Replace it with mboxrb.
* mail/tests/nohome.at: Update lines/sizes.
* mail/testsuite/mail/folder.exp: Likewise.
* mail/testsuite/mail/tag.exp: Likewise.
* mail/testsuite/mail/z.exp: Likewise.

* mda/mda/tests/mda.at: Fix shell quoting.
* mda/putmail/tests/putmail.at: Remove extra newline from the expectation.
Eliminate X-IMAPbase and X-UID from the resulting mailbox.
* mda/tests/mda.sh: Eliminate X-IMAPbase and X-UID from the output.

* pop3d/testsuite/pop3d/read.exp: Fix expected lines/sizes
* readmsg/tests/hdr.at: Fix extra whitespace.

* sieve/tests/action.at: Eliminate X-IMAPbase and X-UID. Fix extra
whitespace in the expected From_ line.
* sieve/tests/addheader.at: Likewise.
* sieve/tests/pipeact.at: Likewise.

5 files changed, 20 insertions, 28 deletions

diff --git a/sieve/tests/action.at b/sieve/tests/action.at
index 84259b1e5..a2da81388 100644
--- a/sieve/tests/action.at
+++ b/sieve/tests/action.at
@@ -44,7 +44,7 @@ MUT_TESTCASE([action fileinto],[],
 [require "fileinto";
 fileinto "+file";
 ],
-[cat file
+[sed -e '/^X-IMAPbase:/d' -e '/^X-UID:/d' file
 ],
 [0],
 [From coyote@desert.example.org Sun May  6 22:16:47 2001
@@ -79,7 +79,7 @@ $20,000 IN LESS THAN TWO MONTHS!  AND IT'S LEGAL!!!!!!!!!
 !!!!!!!!!!!!!!!!!!111111111!!!!!!!11111111111!!1  JUST
 SEND $5 IN SMALL, UNMARKED BILLS TO THE ADDRESSES BELOW!
 
-From bar@dontmailme.org  Fri Dec 28 23:28:09 2001
+From bar@dontmailme.org Fri Dec 28 23:28:09 2001
 Received: (from bar@dontmailme.org)
 	by dontmailme.org id fERKR9N16790
 	for foobar@nonexistent.net; Fri, 28 Dec 2001 22:18:08 +0200
diff --git a/sieve/tests/addheader.at b/sieve/tests/addheader.at
index 814d0f64a..3a2afd46d 100644
--- a/sieve/tests/addheader.at
+++ b/sieve/tests/addheader.at
@@ -23,7 +23,7 @@ addheader "X-Sieve-Filtered" "<kim@job.example.com>";
 ],
 [MUT_MBCOPY($abs_top_srcdir/testsuite/spool/sieve.mbox,mailbox)
 sieve MUT_SIEVE_CMDLINE MUT_SIEVE_OPTIONS -f ./mailbox prog
-sed '/X-IMAPbase:/d;s/ *$//' mailbox
+sed -e '/^X-IMAPbase:/d' -e '/^X-UID:/d' mailbox
 ],
 [From coyote@desert.example.org Sun May  6 22:16:47 2001
 X-Sieve-Filtered: <kim@job.example.com>
@@ -31,8 +31,6 @@ From: coyote@desert.example.org
 To: roadrunner@acme.example.com
 Subject: I have a present for you
 X-Caffeine: C8H10N4O2
-Status: 
-X-UID: 1
 
 Look, I'm sorry about the whole anvil thing, and I really
 didn't mean to try and drop it on you from the top of the
@@ -52,8 +50,6 @@ To: rube@landru.example.edu
 Subject: $$$ YOU, TOO, CAN BE A MILLIONAIRE! $$$
 Date: TBD
 X-Number: 0015
-Status: 
-X-UID: 2
 
 YOU MAY HAVE ALREADY WON TEN MILLION DOLLARS, BUT I DOUBT
 IT!  SO JUST POST THIS TO SIX HUNDRED NEWSGROUPS!  IT WILL
@@ -63,7 +59,7 @@ $20,000 IN LESS THAN TWO MONTHS!  AND IT'S LEGAL!!!!!!!!!
 !!!!!!!!!!!!!!!!!!111111111!!!!!!!11111111111!!1  JUST
 SEND $5 IN SMALL, UNMARKED BILLS TO THE ADDRESSES BELOW!
 
-From bar@dontmailme.org  Fri Dec 28 23:28:09 2001
+From bar@dontmailme.org Fri Dec 28 23:28:09 2001
 X-Sieve-Filtered: <kim@job.example.com>
 Received: (from bar@dontmailme.org)
 	by dontmailme.org id fERKR9N16790
@@ -73,8 +69,6 @@ From: Bar <bar@dontmailme.org>
 To: Foo Bar <foobar@nonexistent.net>
 Message-Id: <200112232808.fERKR9N16790@dontmailme.org>
 Subject: Coffee
-Status: 
-X-UID: 3
 
 How about some coffee?
 
@@ -90,7 +84,7 @@ addheader :last "X-Sieve-Filtered" "<kim@job.example.com>";
 ],
 [MUT_MBCOPY($abs_top_srcdir/testsuite/spool/sieve.mbox,mailbox)
 sieve MUT_SIEVE_CMDLINE MUT_SIEVE_OPTIONS -f ./mailbox prog
-sed '/X-IMAPbase:/d;s/ *$//' mailbox
+sed -e '/^X-IMAPbase:/d' -e '/^X-UID:/d' mailbox
 ],
 [From coyote@desert.example.org Sun May  6 22:16:47 2001
 From: coyote@desert.example.org
@@ -98,8 +92,6 @@ To: roadrunner@acme.example.com
 Subject: I have a present for you
 X-Caffeine: C8H10N4O2
 X-Sieve-Filtered: <kim@job.example.com>
-Status: 
-X-UID: 1
 
 Look, I'm sorry about the whole anvil thing, and I really
 didn't mean to try and drop it on you from the top of the
@@ -119,8 +111,6 @@ Subject: $$$ YOU, TOO, CAN BE A MILLIONAIRE! $$$
 Date: TBD
 X-Number: 0015
 X-Sieve-Filtered: <kim@job.example.com>
-Status: 
-X-UID: 2
 
 YOU MAY HAVE ALREADY WON TEN MILLION DOLLARS, BUT I DOUBT
 IT!  SO JUST POST THIS TO SIX HUNDRED NEWSGROUPS!  IT WILL
@@ -130,7 +120,7 @@ $20,000 IN LESS THAN TWO MONTHS!  AND IT'S LEGAL!!!!!!!!!
 !!!!!!!!!!!!!!!!!!111111111!!!!!!!11111111111!!1  JUST
 SEND $5 IN SMALL, UNMARKED BILLS TO THE ADDRESSES BELOW!
 
-From bar@dontmailme.org  Fri Dec 28 23:28:09 2001
+From bar@dontmailme.org Fri Dec 28 23:28:09 2001
 Received: (from bar@dontmailme.org)
 	by dontmailme.org id fERKR9N16790
 	for foobar@nonexistent.net; Fri, 28 Dec 2001 22:18:08 +0200
@@ -140,8 +130,6 @@ To: Foo Bar <foobar@nonexistent.net>
 Message-Id: <200112232808.fERKR9N16790@dontmailme.org>
 Subject: Coffee
 X-Sieve-Filtered: <kim@job.example.com>
-Status: 
-X-UID: 3
 
 How about some coffee?
 
diff --git a/sieve/tests/pipeact.at b/sieve/tests/pipeact.at
index 6ec309393..22e05048f 100644
--- a/sieve/tests/pipeact.at
+++ b/sieve/tests/pipeact.at
@@ -35,10 +35,10 @@ if header :is "Subject" "Coffee"
 [MUT_MBCOPY($abs_top_srcdir/testsuite/spool/sieve.mbox,mailbox)
 make_savemsg
 sieve MUT_SIEVE_CMDLINE MUT_SIEVE_OPTIONS -f ./mailbox prog
-cat sieve.out
+sed -e '/^X-IMAPbase:/d' -e '/^X-UID:/d' sieve.out
 ],
 [Message begin
-From bar@dontmailme.org  Fri Dec 28 23:28:09 2001
+From bar@dontmailme.org Fri Dec 28 23:28:09 2001
 Received: (from bar@dontmailme.org)
 	by dontmailme.org id fERKR9N16790
 	for foobar@nonexistent.net; Fri, 28 Dec 2001 22:18:08 +0200
@@ -66,10 +66,10 @@ if header :is "Subject" "Coffee"
 [MUT_MBCOPY($abs_top_srcdir/testsuite/spool/sieve.mbox,mailbox)
 make_savemsg
 sieve MUT_SIEVE_CMDLINE MUT_SIEVE_OPTIONS -f ./mailbox prog
-cat sieve.out
+sed -e '/^X-IMAPbase:/d' -e '/^X-UID:/d' sieve.out
 ],
 [Message begin
-From bar@dontmailme.org  Fri Dec 28 23:28:09 2001
+From bar@dontmailme.org Fri Dec 28 23:28:09 2001
 Message end
 ],
 [IMPLICIT KEEP on msg uid 1
@@ -87,7 +87,7 @@ if header :is "Subject" "Coffee"
 [MUT_MBCOPY($abs_top_srcdir/testsuite/spool/sieve.mbox,mailbox)
 make_savemsg
 sieve MUT_SIEVE_CMDLINE MUT_SIEVE_OPTIONS -f ./mailbox prog
-cat sieve.out
+sed -e '/^X-IMAPbase:/d' -e '/^X-UID:/d' sieve.out
 ],
 [Message begin
 Received: (from bar@dontmailme.org)
@@ -116,7 +116,7 @@ if header :is "Subject" "Coffee"
 [MUT_MBCOPY($abs_top_srcdir/testsuite/spool/sieve.mbox,mailbox)
 make_savemsg
 sieve MUT_SIEVE_CMDLINE MUT_SIEVE_OPTIONS -f ./mailbox prog
-cat sieve.out
+sed -e '/^X-IMAPbase:/d' -e '/^X-UID:/d' sieve.out
 ],
 [Message begin
 How about some coffee?
@@ -137,10 +137,10 @@ if header :is "Subject" "Coffee"
 [MUT_MBCOPY($abs_top_srcdir/testsuite/spool/sieve.mbox,mailbox)
 make_savemsg
 sieve MUT_SIEVE_CMDLINE MUT_SIEVE_OPTIONS -f ./mailbox prog
-cat sieve.out
+sed -e '/^X-IMAPbase:/d' -e '/^X-UID:/d' sieve.out
 ],
 [Message begin
-From bar@dontmailme.org  Fri Dec 28 23:28:09 2001
+From bar@dontmailme.org Fri Dec 28 23:28:09 2001
 How about some coffee?
 Message end
 ],
diff --git a/sieve/tests/redirect.at b/sieve/tests/redirect.at
index 779922e9e..ded583705 100644
--- a/sieve/tests/redirect.at
+++ b/sieve/tests/redirect.at
@@ -38,7 +38,7 @@ cat $MAIL_DUMP
 SENDER: coyote@desert.example.org
 NRCPT: 1
 RCPT[0]: gray@gnu.org
-LENGTH: 628
+LENGTH: 629
 From: coyote@desert.example.org
 To: roadrunner@acme.example.com
 Subject: I have a present for you
@@ -56,11 +56,12 @@ for you over the years, but I know we can work this out.
 --
 Wile E. Coyote   "Super Genius"   coyote@desert.example.org
 
+
 MSGID: 0001
 SENDER: b1ff@de.res.example.com
 NRCPT: 1
 RCPT[0]: gray@gnu.org
-LENGTH: 595
+LENGTH: 596
 From: youcouldberich!@reply-by-postal-mail.invalid
 To: rube@landru.example.edu
 Subject: $$$ YOU, TOO, CAN BE A MILLIONAIRE! $$$
@@ -76,6 +77,7 @@ $20,000 IN LESS THAN TWO MONTHS!  AND IT'S LEGAL!!!!!!!!!
 !!!!!!!!!!!!!!!!!!111111111!!!!!!!11111111111!!1  JUST
 SEND $5 IN SMALL, UNMARKED BILLS TO THE ADDRESSES BELOW!
 
+
 MSGID: 0001
 SENDER: bar@dontmailme.org
 NRCPT: 1
diff --git a/sieve/tests/reject.at b/sieve/tests/reject.at
index 5739c598d..832c0d3b4 100644
--- a/sieve/tests/reject.at
+++ b/sieve/tests/reject.at
@@ -104,6 +104,7 @@ for you over the years, but I know we can work this out.
 --
 Wile E. Coyote   "Super Genius"   coyote@desert.example.org
 
+
 --(boundary)
 
 MSGID: 0001
@@ -154,6 +155,7 @@ $20,000 IN LESS THAN TWO MONTHS!  AND IT'S LEGAL!!!!!!!!!
 !!!!!!!!!!!!!!!!!!111111111!!!!!!!11111111111!!1  JUST
 SEND $5 IN SMALL, UNMARKED BILLS TO THE ADDRESSES BELOW!
 
+
 --(boundary)
 
 MSGID: 0001