diff options
author | Sergey Poznyakoff <gray@gnu.org> | 2020-11-24 21:41:27 +0200 |
---|---|---|
committer | Sergey Poznyakoff <gray@gnu.org> | 2020-11-25 14:25:09 +0200 |
commit | a3bd653315227a7cfb0a855acfa5bbe13c8f2645 (patch) | |
tree | d8dd520c39db21c1cbd6cbcb50166844657a1bb6 | |
parent | c004169d5bd1a93270b5fb195d0c737b1c76af0e (diff) | |
download | mailutils-a3bd653315227a7cfb0a855acfa5bbe13c8f2645.tar.gz mailutils-a3bd653315227a7cfb0a855acfa5bbe13c8f2645.tar.bz2 |
Replace mbox format driver with mboxrb
* NEWS: update
* configure.ac: Revert 9763fd4a.
* include/mailutils/registrar.h: Likewise.
* libproto/Makefile.am: Likewise.
* libmailutils/stream/stream.c (mu_stream_size): Fix size calculation
in case of line and full buffering scheme.
* libmailutils/stream/streamcpy.c (mu_stream_copy_nl): Change semantics.
Ensure that the copied data ends with two newline characters. Append
them if necessary.
* testsuite/mbop.c: New commands: message_lines and message_size.
* libproto/mbox/mboxrb.c (mboxrb_mailbox_init_stream): Always add
MU_STREAM_READ permission.
Use the same flags for mu_mapfile_stream_create and
mu_file_stream_create.
(mboxrb_rescan_unlocked): Count terminating newline as part of the
message.
(mailbox_append_message): Make sure each message ends with an empty
line.
(mboxrb_tracker_sync): Update mailbox mesg_count.
(mboxrb_flush_temp): Truncate the temporary stream to the new size.
(mboxrb_flush_unlocked): Clear all modifications only in FLUSH_UIDVALIDITY
mode.
(mboxrb_message_copy_with_uid): Fix message_end computation. Use
mu_stream_copy_nl to copy message body.
(mu_mboxrb_message_reconstruct): Update ref on return.
* po/POTFILES.in: Update.
* mail/copy.c (append_to_file): Use mu_stream_stat_buffer to compute
statistics.
* mail/mail.c: Allow for opening read-only mailboxes.
* libproto/mbox/tests/body.at: Fix the expected byte count.
* libproto/mbox/tests/qget.at: Likewise.
* libproto/mbox/tests/delete.at: Add more tests.
* mail/tests/align.at: Fix the expected byte and line count.
* mail/tests/hold.at: Likewise.
* mail/testsuite/mail/read.exp: Likewise.
* mail/testsuite/mail/tag.exp: Likewise.
* mail/testsuite/mail/z.exp: Likewise.
* mail/tests/copy04.at: Don't use line count and size in comparison.
* mail/testsuite/mail/write.exp: Use regexps instead of exact strings
for the same reason.
* pop3d/testsuite/pop3d/read.exp: Fix the expected byte count.
* readmsg/tests/all.at: Expect extra newline in output.
* readmsg/tests/hdr.at: Likewise.
* readmsg/tests/nohdr.at: Likewise.
* readmsg/tests/twomsg.at: Likewise.
* readmsg/tests/weed.at: Likewise.
* sieve/tests/redirect.at: Fix the expected byte count.
* sieve/tests/reject.at: Expect extra newline in output.
* mh/inc.c (incmbx): Parse truncate and nomoveto URL parameters prior
to opening the mailbox. If notruncate is requested, open the mailbox
read-only.
* mh/mh_format.c (str_compress_ws): Fix copying between overlapping
memory regions.
* libtests/Makefile.am: Remove lstuid tests.
* libtests/testsuite.at: Likewise.
* libtests/lstuid.c: Remove.
* libtests/lstuid00.at: Remove.
* libtests/lstuid01.at: Remove.
* libtests/lstuid02.at: Remove.
* libtests/mime.at: Fix the expected byte count.
* imap4d/tests/IDEF0956.at: Fix the expected byte count.
* map4d/tests/fetch.at: Likewise.
* imap4d/tests/append00.at: Remove X-* headers and Status from the mbox
prior to comparison.
* imap4d/tests/append01.at: Likewise.
* libproto/mbox: Remove old code. Replace it with mboxrb.
* mail/tests/nohome.at: Update lines/sizes.
* mail/testsuite/mail/folder.exp: Likewise.
* mail/testsuite/mail/tag.exp: Likewise.
* mail/testsuite/mail/z.exp: Likewise.
* mda/mda/tests/mda.at: Fix shell quoting.
* mda/putmail/tests/putmail.at: Remove extra newline from the expectation.
Eliminate X-IMAPbase and X-UID from the resulting mailbox.
* mda/tests/mda.sh: Eliminate X-IMAPbase and X-UID from the output.
* pop3d/testsuite/pop3d/read.exp: Fix expected lines/sizes
* readmsg/tests/hdr.at: Fix extra whitespace.
* sieve/tests/action.at: Eliminate X-IMAPbase and X-UID. Fix extra
whitespace in the expected From_ line.
* sieve/tests/addheader.at: Likewise.
* sieve/tests/pipeact.at: Likewise.
81 files changed, 1543 insertions, 3549 deletions
@@ -1,10 +1,15 @@ -GNU mailutils NEWS -- history of user-visible changes. 2020-11-17 +GNU mailutils NEWS -- history of user-visible changes. 2020-11-25 See the end of file for copying conditions. Please send mailutils bug reports to <bug-mailutils@gnu.org>. Version 3.10.90 (git) +* Support for the traditional mbox format rewritten from scratch + +The new format library handles all flavors of the "mbox" format +family. The created mailboxes adhere to the "mboxrb" format. + * Fix incorrect message flags in maildir Message flags (the "info" filename part) used in maildir file names diff --git a/configure.ac b/configure.ac index 6c80ace4f..ae80fd9c2 100644 --- a/configure.ac +++ b/configure.ac @@ -989,7 +989,6 @@ MU_ENABLE_REMOTE_MAILBOX_FORMAT([pop]) MU_ENABLE_LOCAL_MAILBOX_FORMAT([mh]) MU_ENABLE_LOCAL_MAILBOX_FORMAT([maildir]) MU_ENABLE_LOCAL_MAILBOX_FORMAT([dotmail]) -MU_ENABLE_LOCAL_MAILBOX_FORMAT([mboxrb]) AC_SUBST(MU_SMTP_PROGRAMS_BUILD) AC_SUBST(MU_SMTP_DEJATOOL) diff --git a/imap4d/tests/IDEF0956.at b/imap4d/tests/IDEF0956.at index f2761fc32..9470d9582 100644 --- a/imap4d/tests/IDEF0956.at +++ b/imap4d/tests/IDEF0956.at @@ -38,10 +38,11 @@ X LOGOUT * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [[PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft)]] Permanent flags 1 OK [[READ-WRITE]] SELECT Completed -* 1 FETCH (BODY[[TEXT]]<0> {161} +* 1 FETCH (BODY[[TEXT]]<0> {163} ABASEMENT, n. A decent and customary mental attitude in the presence of wealth of power. Peculiarly appropriate in an employee when addressing an employer. + ) 2 OK FETCH Completed * BYE Session terminating. diff --git a/imap4d/tests/append00.at b/imap4d/tests/append00.at index c362e8aa7..2c3de5f77 100644 --- a/imap4d/tests/append00.at +++ b/imap4d/tests/append00.at @@ -33,7 +33,7 @@ Hello Joe, do you think we can meet at 3:30 tomorrow X LOGOUT ]) echo "==" -awk 'NR==1 {print "1:",$1,$2; next} NF==0 {print NR":"; next} {print NR":",$0}' mbox +sed -e '/^X-/d' -e /^Status:/d mbox | awk 'NR==1 {print "1:",$1,$2; next} NF==0 {print NR":"; next} {print NR":",$0}' ], [0], [* PREAUTH IMAP4rev1 Test mode diff --git a/imap4d/tests/append01.at b/imap4d/tests/append01.at index 71114c9bb..569a415ec 100644 --- a/imap4d/tests/append01.at +++ b/imap4d/tests/append01.at @@ -33,7 +33,7 @@ Better yet at 04:00? X LOGOUT ]) echo "==" -awk 'NF==0 {print NR":"; next} {print NR":",$0}' mbox +sed -e '/^X-/d' -e /^Status:/d mbox | awk 'NF==0 {print NR":"; next} {print NR":",$0}' ], [0], [* PREAUTH IMAP4rev1 Test mode diff --git a/imap4d/tests/fetch.at b/imap4d/tests/fetch.at index 4575a5508..245f56a65 100644 --- a/imap4d/tests/fetch.at +++ b/imap4d/tests/fetch.at @@ -93,10 +93,10 @@ FETCH_CHECK([envelope],[fetch-envelope fetch04], FETCH_CHECK([ALL],[fetch-all fetch05], [1:* ALL], -[* 1 FETCH (FLAGS (\Recent) INTERNALDATE "28-Dec-2001 22:18:09 +0000" RFC822.SIZE 1298 ENVELOPE ("Fri, 28 Dec 2001 22:18:08 +0200" "Jabberwocky" (("Foo Bar" NIL "foobar" "nonexistent.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) (("Bar" NIL "bar" "dontmailme.org")) NIL NIL NIL "<200112282018.fBSKI8N04906@nonexistent.net>")) -* 2 FETCH (FLAGS (\Recent) INTERNALDATE "28-Dec-2001 23:28:09 +0000" RFC822.SIZE 547 ENVELOPE ("Fri, 28 Dec 2001 23:28:08 +0200" "Re: Jabberwocky" (("Bar" NIL "bar" "dontmailme.org")) (("Bar" NIL "bar" "dontmailme.org")) (("Bar" NIL "bar" "dontmailme.org")) (("Foo Bar" NIL "foobar" "nonexistent.net")) NIL NIL NIL "<200112232808.fERKR9N16790@dontmailme.org>")) -* 3 FETCH (FLAGS (\Recent) INTERNALDATE "13-Jul-2002 00:43:18 +0000" RFC822.SIZE 1611 ENVELOPE ("Sat, 13 Jul 2002 00:43:18 +0300" "Simple MIME" (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) NIL NIL NIL "<200207122143.g6CLhIb05086@example.net>")) -* 4 FETCH (FLAGS (\Recent) INTERNALDATE "13-Jul-2002 00:50:58 +0000" RFC822.SIZE 3483 ENVELOPE ("Sat, 13 Jul 2002 00:50:58 +0300" "Nested MIME" (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) NIL NIL NIL "<200207122150.g6CLowb05126@example.net>")) +[* 1 FETCH (FLAGS (\Recent) INTERNALDATE "28-Dec-2001 22:18:09 +0000" RFC822.SIZE 1300 ENVELOPE ("Fri, 28 Dec 2001 22:18:08 +0200" "Jabberwocky" (("Foo Bar" NIL "foobar" "nonexistent.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) (("Bar" NIL "bar" "dontmailme.org")) NIL NIL NIL "<200112282018.fBSKI8N04906@nonexistent.net>")) +* 2 FETCH (FLAGS (\Recent) INTERNALDATE "28-Dec-2001 23:28:09 +0000" RFC822.SIZE 549 ENVELOPE ("Fri, 28 Dec 2001 23:28:08 +0200" "Re: Jabberwocky" (("Bar" NIL "bar" "dontmailme.org")) (("Bar" NIL "bar" "dontmailme.org")) (("Bar" NIL "bar" "dontmailme.org")) (("Foo Bar" NIL "foobar" "nonexistent.net")) NIL NIL NIL "<200112232808.fERKR9N16790@dontmailme.org>")) +* 3 FETCH (FLAGS (\Recent) INTERNALDATE "13-Jul-2002 00:43:18 +0000" RFC822.SIZE 1613 ENVELOPE ("Sat, 13 Jul 2002 00:43:18 +0300" "Simple MIME" (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) NIL NIL NIL "<200207122143.g6CLhIb05086@example.net>")) +* 4 FETCH (FLAGS (\Recent) INTERNALDATE "13-Jul-2002 00:50:58 +0000" RFC822.SIZE 3485 ENVELOPE ("Sat, 13 Jul 2002 00:50:58 +0300" "Nested MIME" (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) NIL NIL NIL "<200207122150.g6CLowb05126@example.net>")) * 5 FETCH (FLAGS (\Recent) INTERNALDATE "13-Jul-2002 00:43:18 +0000" RFC822.SIZE 884 ENVELOPE ("Sat, 13 Jul 2002 00:43:18 +0300" "Empty MIME Parts" (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) NIL NIL NIL "<200207122143.g6CLhIb05086@example.net>"))], [], [fixup_tz]) @@ -107,8 +107,8 @@ FETCH_CHECK([ALL],[fetch-all fetch05], # [MIME-IMB] headers. FETCH_CHECK([BODYSTRUCTURE],[fetch-bodystructure fetch06], [1:* BODYSTRUCTURE], -[* 1 FETCH (BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 972 35 NIL NIL NIL)) -* 2 FETCH (BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 219 4 NIL NIL NIL)) +[* 1 FETCH (BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 974 36 NIL NIL NIL)) +* 2 FETCH (BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 221 5 NIL NIL NIL)) * 3 FETCH (BODYSTRUCTURE (("text" "plain" ("name" "msg.1" "charset" "us-ascii") "<5082.1026510189.1@example.net>" "How doth" "7BIT" 239 9 NIL NIL NIL)("application" "octet-stream" ("name" "msg.21") "<5082.1026510189.2@example.net>" "Father William Part I" "base64" 468 NIL NIL NIL) "mixed" ("boundary" "----- =_aaaaaaaaaa0") NIL NIL)) * 4 FETCH (BODYSTRUCTURE (("text" "plain" ("name" "msg.21" "charset" "us-ascii") "<5122.1026510654.2@example.net>" "Father William Part I" "7BIT" 351 10 NIL NIL NIL)(("application" "octet-stream" ("name" "msg.22") "<5122.1026510654.4@example.net>" "Father William Part II" "base64" 486 NIL NIL NIL)(("application" "octet-stream" ("name" "msg.23") "<5122.1026510654.6@example.net>" "Father William Part III" "base64" 490 NIL NIL NIL)("application" "octet-stream" ("name" "msg.24") "<5122.1026510654.7@example.net>" "Father William Part IV" "base64" 502 NIL NIL NIL) "mixed" ("boundary" "----- =_aaaaaaaaaa2") NIL NIL) "mixed" ("boundary" "----- =_aaaaaaaaaa1") NIL NIL) "mixed" ("boundary" "----- =_aaaaaaaaaa0") NIL NIL)) * 5 FETCH (BODYSTRUCTURE (("text" "plain" ("name" "empty" "charset" "us-ascii") "<5082.1026510189.1@example.net>" "Empty part" "7BIT" 0 0 NIL NIL NIL)("text" "plain" ("name" "single.line" "charset" "us-ascii") "<5082.1026510189.2@example.net>" "Single line part" "7BIT" 2 1 NIL NIL NIL) "mixed" ("boundary" "----- =_aaaaaaaaaa0") NIL NIL))]) @@ -117,8 +117,8 @@ FETCH_CHECK([BODYSTRUCTURE],[fetch-bodystructure fetch06], # Non-extensible form of BODYSTRUCTURE. FETCH_CHECK([BODY],[fetch-body fetch07], [1:* BODY], -[* 1 FETCH (BODY ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 972 35)) -* 2 FETCH (BODY ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 219 4)) +[* 1 FETCH (BODY ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 974 36)) +* 2 FETCH (BODY ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 221 5)) * 3 FETCH (BODY (("text" "plain" ("name" "msg.1" "charset" "us-ascii") "<5082.1026510189.1@example.net>" "How doth" "7BIT" 239 9)("application" "octet-stream" ("name" "msg.21") "<5082.1026510189.2@example.net>" "Father William Part I" "base64" 468) "mixed" NIL NIL NIL)) * 4 FETCH (BODY (("text" "plain" ("name" "msg.21" "charset" "us-ascii") "<5122.1026510654.2@example.net>" "Father William Part I" "7BIT" 351 10)(("application" "octet-stream" ("name" "msg.22") "<5122.1026510654.4@example.net>" "Father William Part II" "base64" 486)(("application" "octet-stream" ("name" "msg.23") "<5122.1026510654.6@example.net>" "Father William Part III" "base64" 490)("application" "octet-stream" ("name" "msg.24") "<5122.1026510654.7@example.net>" "Father William Part IV" "base64" 502) "mixed" NIL NIL NIL) "mixed" NIL NIL NIL) "mixed" NIL NIL NIL)) * 5 FETCH (BODY (("text" "plain" ("name" "empty" "charset" "us-ascii") "<5082.1026510189.1@example.net>" "Empty part" "7BIT" 0 0)("text" "plain" ("name" "single.line" "charset" "us-ascii") "<5082.1026510189.2@example.net>" "Single line part" "7BIT" 2 1) "mixed" NIL NIL NIL))]) @@ -185,7 +185,7 @@ Message-Id: <200112282018.fBSKI8N04906@nonexistent.net> # the message, omitting the [RFC-822] header. FETCH_CHECK([BODY[[TEXT]]],[fetch-body-text fetch11], [1 BODY[[TEXT]]], -[* 1 FETCH (FLAGS (\Seen) BODY[[TEXT]] {972} +[* 1 FETCH (FLAGS (\Seen) BODY[[TEXT]] {974} `Twas brillig, and the slithy toves Did gyre and gimble in the wabe; All mimsy were the borogoves, @@ -221,6 +221,7 @@ Did gyre and gimble in the wabe; All mimsy were the borogoves, And the mome raths outgrabe. + )]) # It is possible to fetch a substring of the @@ -260,7 +261,7 @@ FETCH_CHECK([BODY[[TEXT]]<X-too-big>],[fetch-body-text-off2big fetch14], FETCH_CHECK([BODY[[TEXT]] (truncated)],[fetch-body-text-truncated fetch15], [1 BODY[[TEXT]]<0.4000>], -[* 1 FETCH (FLAGS (\Seen) BODY[[TEXT]]<0> {972} +[* 1 FETCH (FLAGS (\Seen) BODY[[TEXT]]<0> {974} `Twas brillig, and the slithy toves Did gyre and gimble in the wabe; All mimsy were the borogoves, @@ -296,6 +297,7 @@ Did gyre and gimble in the wabe; All mimsy were the borogoves, And the mome raths outgrabe. + )]) # The MIME part specifier refers to the [MIME-IMB] @@ -416,7 +418,7 @@ Subject: Jabberwocky FETCH_CHECK([RFC822],[fetch-rfc822 fetch24], [1 RFC822], -[* 1 FETCH (FLAGS (\Seen) RFC822 {1298} +[* 1 FETCH (FLAGS (\Seen) RFC822 {1300} Received: (from foobar@nonexistent.net) by nonexistent.net id fBSKI8N04906 for bar@dontmailme.org; Fri, 28 Dec 2001 22:18:08 +0200 @@ -461,6 +463,7 @@ Did gyre and gimble in the wabe; All mimsy were the borogoves, And the mome raths outgrabe. + )]) # RFC822.HEADER Functionally equivalent to BODY.PEEK[HEADER], @@ -485,18 +488,19 @@ Subject: Re: Jabberwocky FETCH_CHECK([RFC822.SIZE],[fetch-rfc822-size fetch26], [3 RFC822.SIZE], -[* 3 FETCH (RFC822.SIZE 1611)]) +[* 3 FETCH (RFC822.SIZE 1613)]) # RFC822.TEXT Functionally equivalent to BODY[TEXT], differing in # the syntax of the resulting untagged FETCH data # (RFC822.TEXT is returned). FETCH_CHECK([RFC822.TEXT],[fetch-rfc822-text fetch27], [2 RFC822.TEXT], -[* 2 FETCH (FLAGS (\Seen) RFC822.TEXT {219} +[* 2 FETCH (FLAGS (\Seen) RFC822.TEXT {221} It seems very pretty, but it's *rather* hard to understand!' Somehow it seems to fill my head with ideas -- only I don't exactly know what they are! However, SOMEBODY killed SOMETHING: that's clear, at any rate... + )]) # FAST Macro equivalent to: (FLAGS INTERNALDATE @@ -504,7 +508,7 @@ that's clear, at any rate... FETCH_CHECK([FAST],[fetch-fast fetch28], [1 FAST], -[* 1 FETCH (FLAGS (\Recent) INTERNALDATE "28-Dec-2001 22:18:09 +0000" RFC822.SIZE 1298)], +[* 1 FETCH (FLAGS (\Recent) INTERNALDATE "28-Dec-2001 22:18:09 +0000" RFC822.SIZE 1300)], [], [fixup_tz]) @@ -513,7 +517,7 @@ FETCH_CHECK([FAST],[fetch-fast fetch28], FETCH_CHECK([FULL],[fetch-full fetch29], [4 FULL], -[* 4 FETCH (FLAGS (\Recent) INTERNALDATE "13-Jul-2002 00:50:58 +0000" RFC822.SIZE 3483 ENVELOPE ("Sat, 13 Jul 2002 00:50:58 +0300" "Nested MIME" (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Sergey Poznyakoff" NIL "gray" "example.net")) (("Foo Bar" NIL "foobar" "nonexistent.net")) NIL NIL NIL "<200207122150.g6CLowb05126@example.net>") BODY (("text" "plain" ("name" "msg.21" "charset" "us-ascii") "<5122.1026510654.2@example.net>" "Father William Part I" "7BIT" 351 10)(("application" "octet-stream" ("name" "msg.22") "<5122.1026510654.4@example.net>" "Father William Part II" "base64" 486)(("application" "octet-stream" ("name" "msg.23") "<5122.1026510654.6@example.net>" "Father William Part III" "base64" 490)("application" "octet-stream" ("name" "msg.24") "<5122.1026510654.7@example.net>" "Father William Part IV" "base64" 502) "mixed" NIL |