Replace mbox format driver with mboxrb

* NEWS: update

* configure.ac: Revert 9763fd4a.
* include/mailutils/registrar.h: Likewise.
* libproto/Makefile.am: Likewise.

* libmailutils/stream/stream.c (mu_stream_size): Fix size calculation
in case of line and full buffering scheme.
* libmailutils/stream/streamcpy.c (mu_stream_copy_nl): Change semantics.
Ensure that the copied data ends with two newline characters.  Append
them if necessary.

* testsuite/mbop.c: New commands: message_lines and message_size.

* libproto/mbox/mboxrb.c (mboxrb_mailbox_init_stream): Always add
MU_STREAM_READ permission.
Use the same flags for mu_mapfile_stream_create and
mu_file_stream_create.
(mboxrb_rescan_unlocked): Count terminating newline as part of the
message.
(mailbox_append_message): Make sure each message ends with an empty
line.
(mboxrb_tracker_sync): Update mailbox mesg_count.
(mboxrb_flush_temp): Truncate the temporary stream to the new size.
(mboxrb_flush_unlocked): Clear all modifications only in FLUSH_UIDVALIDITY
mode.
(mboxrb_message_copy_with_uid): Fix message_end computation.  Use
mu_stream_copy_nl to copy message body.
(mu_mboxrb_message_reconstruct): Update ref on return.
* po/POTFILES.in: Update.

* mail/copy.c (append_to_file): Use mu_stream_stat_buffer to compute
statistics.
* mail/mail.c: Allow for opening read-only mailboxes.

* libproto/mbox/tests/body.at: Fix the expected byte count.
* libproto/mbox/tests/qget.at: Likewise.
* libproto/mbox/tests/delete.at: Add more tests.

* mail/tests/align.at: Fix the expected byte and line count.
* mail/tests/hold.at: Likewise.
* mail/testsuite/mail/read.exp: Likewise.
* mail/testsuite/mail/tag.exp: Likewise.
* mail/testsuite/mail/z.exp: Likewise.
* mail/tests/copy04.at: Don't use line count and size in comparison.
* mail/testsuite/mail/write.exp: Use regexps instead of exact strings
for the same reason.

* pop3d/testsuite/pop3d/read.exp: Fix the expected byte count.

* readmsg/tests/all.at: Expect extra newline in output.
* readmsg/tests/hdr.at: Likewise.
* readmsg/tests/nohdr.at: Likewise.
* readmsg/tests/twomsg.at: Likewise.
* readmsg/tests/weed.at: Likewise.

* sieve/tests/redirect.at: Fix the expected byte count.
* sieve/tests/reject.at: Expect extra newline in output.

* mh/inc.c (incmbx): Parse truncate and nomoveto URL parameters prior
to opening the mailbox.  If notruncate is requested, open the mailbox
read-only.
* mh/mh_format.c (str_compress_ws): Fix copying between overlapping
memory regions.

* libtests/Makefile.am: Remove lstuid tests.
* libtests/testsuite.at: Likewise.
* libtests/lstuid.c: Remove.
* libtests/lstuid00.at: Remove.
* libtests/lstuid01.at: Remove.
* libtests/lstuid02.at: Remove.
* libtests/mime.at: Fix the expected byte count.

* imap4d/tests/IDEF0956.at: Fix the expected byte count.
* map4d/tests/fetch.at: Likewise.
* imap4d/tests/append00.at: Remove X-* headers and Status from the mbox
prior to comparison.
* imap4d/tests/append01.at: Likewise.

* libproto/mbox: Remove old code.  Replace it with mboxrb.
* mail/tests/nohome.at: Update lines/sizes.
* mail/testsuite/mail/folder.exp: Likewise.
* mail/testsuite/mail/tag.exp: Likewise.
* mail/testsuite/mail/z.exp: Likewise.

* mda/mda/tests/mda.at: Fix shell quoting.
* mda/putmail/tests/putmail.at: Remove extra newline from the expectation.
Eliminate X-IMAPbase and X-UID from the resulting mailbox.
* mda/tests/mda.sh: Eliminate X-IMAPbase and X-UID from the output.

* pop3d/testsuite/pop3d/read.exp: Fix expected lines/sizes
* readmsg/tests/hdr.at: Fix extra whitespace.

* sieve/tests/action.at: Eliminate X-IMAPbase and X-UID. Fix extra
whitespace in the expected From_ line.
* sieve/tests/addheader.at: Likewise.
* sieve/tests/pipeact.at: Likewise.

2 files changed, 24 insertions, 7 deletions

diff --git a/libmailutils/stream/stream.c b/libmailutils/stream/stream.c
index 8ff0b808d..df651a335 100644
--- a/libmailutils/stream/stream.c
+++ b/libmailutils/stream/stream.c
@@ -1137,8 +1137,12 @@ mu_stream_size (mu_stream_t stream, mu_off_t *psize)
   rc = stream->size (stream, &size);
   if (rc == 0)
     {
-      if (stream->buftype != mu_buffer_none && stream->offset == size)
-	size += stream->level;
+      if (stream->buftype != mu_buffer_none)
+	{
+	  size_t n = stream->offset + stream->level;
+	  if (n > size)
+	    size = n;
+	}
       *psize = size;
     }
   return mu_stream_seterr (stream, rc, rc != 0);
diff --git a/libmailutils/stream/streamcpy.c b/libmailutils/stream/streamcpy.c
index 884b997d2..52bcbb649 100644
--- a/libmailutils/stream/streamcpy.c
+++ b/libmailutils/stream/streamcpy.c
@@ -154,21 +154,34 @@ mu_stream_copy (mu_stream_t dst, mu_stream_t src, mu_off_t size,
 static void
 capture_last_char (char *buf, size_t size, void *data)
 {
-  *(char*)data = buf[size-1];
+  int *n = data;
+  if (buf[size-1] == '\n')
+    {
+      if (size == 1)
+	{
+	  ++*n;
+	}
+      else if (buf[size-2] == '\n')
+	*n = 2;
+      else
+	*n = 1;
+    }
+  else
+    *n = 0;
 }
 
 /* Same as mu_stream_copy, but also ensures that the copied data end with
-   a '\n' character. */
+   two '\n' characters. */
 int
 mu_stream_copy_nl (mu_stream_t dst, mu_stream_t src, mu_off_t size,
 		   mu_off_t *pcsz)
 {
-  char lc;
+  int lc = 0;
   int status = mu_stream_copy_wcb (dst, src, size, capture_last_char, &lc,
 				   pcsz);
-  if (status == 0 && lc != '\n')
+  if (status == 0 && lc < 2)
     {
-      status = mu_stream_write (dst, "\n", 1, NULL);
+      status = mu_stream_write (dst, "\n\n", 2 - lc, NULL);
       if (status == 0 && pcsz)
 	++ *pcsz;
     }