Redo the support for transport-based streams.

If a stream takes another stream(s) as a transport, it
always increases its reference count.  This means that
when that stream is destroyed the underlying stream is
not destroyed by default (its refcount is decreased, that's
all). To force destruction of the underlying stream,
the caller must explicitly unreference it after creating
the stream that uses it (or give its creation function
the MU_STREAM_AUTOCLOSE flag, if it supports flags).

Similarly, if a stream uses a non-stream object (e.g. a file
descriptor) as the transport, it will not destroy it, unless
it has been created with the MU_STREAM_AUTOCLOSE flag. This
differs from the previous behavior.

The MU_STREAM_NO_CHECK and MU_STREAM_NO_CLOSE flags are removed.

* examples/base64.c (main): Call mu_filter_create with the
MU_STREAM_AUTOCLOSE flag.
* examples/mimetest.c (message_display_parts): Likewise.
* examples/murun.c (main): Unref the input stream after
passing it to mu_filter_prog_stream_create.
* imap4d/fetch.c (fetch_io): Update arguments to mu_filter_create
and mu_memory_stream_create.
* imap4d/preauth.c (decode64_buf)
(do_preauth_ident): Update arguments to mu_memory_stream_create and
mu_tcp_stream_create.
* imap4d/util.c (util_setio): Update arguments to mu_filter_create and
mu_stdio_stream_create.
* include/mailutils/stream.h (MU_STREAM_NO_CLOSE)
(MU_STREAM_NO_CHECK): Remove.
(MU_STREAM_AUTOCLOSE): New flag.
(mu_iostream_create): Remove the `flags' argument.
* libmu_argp/muinit.c (mu_app_init): Update arguments to
mu_stdio_stream_create.
* libmu_auth/ldap.c: Update arguments to
mu_memory_stream_create and mu_filter_create.
* libmu_auth/tls.c (_tls_io_close, _tls_close): Always try to close the
transport stream. Rely on refcount to protect it.
(_tls_io_done, _tls_done): Likewise, always unreference it.
(_mu_tls_io_stream_create): Increase reference counter on the
transport stream, unless MU_STREAM_AUTOCLOSE is requested.
(_mu_tls_stream_create): Rewrite using the new MU_STREAM_AUTOCLOSE
logic.
* libmu_sieve/extensions/spamd.c (spamd_connect_tcp): Update arguments to
mu_tcp_stream_create.
(spamd_connect_tcp): Update arguments to mu_socket_stream_create and
mu_filter_create.
* libmu_sieve/extensions/vacation.c (build_mime): Update arguments to
mu_filter_create.
* mail/decode.c (display_submessage): Update arguments to
mu_filter_create.
* mailbox/attachment.c (mu_message_save_attachment): Update arguments to
mu_filter_create.
* mailbox/cfg_driver.c (mu_cfg_tree_reduce): Update arguments to
mu_stdio_stream_create.
* mailbox/dbgstream.c (_dbg_done): Use MU_STREAM_AUTOCLOSE bit to
decide whether to destroy the debug object.
(mu_dbgstream_create): Use MU_STREAM_AUTOCLOSE instead of
MU_STREAM_NO_CLOSE.
* mailbox/file_stream.c (fd_close): Use MU_STREAM_AUTOCLOSE bit to
decide whether to close the descriptor.
(mu_file_stream_create): Force MU_STREAM_AUTOCLOSE bit.
* mailbox/filter.c (filter_create_rd, filter_create_wr): Change
substream creation logic.
* mailbox/filter_iconv.c (_icvt_close): Always try to close the
transport stream. Rely on refcount to protect it.
(_icvt_done): Ditto for destroying it.
(mu_filter_iconv_create): Increase refcount on the transport stream,
unless MU_STREAM_AUTOCLOSE is requested.
* mailbox/fltstream.c (filter_done): Always dereference the
transport stream. Rely on refcount to protect it.
(filter_close): Ditto for closing it.
(mu_filter_stream_create): Increase refcount on the transport stream,
unless MU_STREAM_AUTOCLOSE is requested.
* mailbox/iostream.c (_iostream_close) : Always try to close the
transport stream. Rely on refcount to protect it.
(_iostream_done): Ditto for closing it.
(mu_iostream_create): Remove the use of MU_STREAM_NO_CLOSE.
* mailbox/mimehdr.c (mu_mimehdr_decode_param): Remove the use of
MU_STREAM_NO_CLOSE.
* mailbox/mutil.c (mu_decode_filter) Change substream creation logic.
* mailbox/prog_stream.c (_prog_open): Use MU_STREAM_AUTOCLOSE bit
in arguments to mu_stdio_stream_create.
(mu_filter_prog_stream_create): Increase refcount on the transport
(input) stream.
* mailbox/rfc2047.c (mu_rfc2047_decode): Dereference in_stream after
passing it to mu_decode_filter.
Pass MU_STREAM_AUTOCLOSE in flags to mu_filter_create.
* mailbox/socket_stream.c (mu_socket_stream_create): Force
MU_STREAM_AUTOCLOSE bit.
* mailbox/streamref.c (_streamref_close): Always close the
transport stream (refcount will protect it, if necessary).
(mu_streamref_create_abridged): Mask out the MU_STREAM_AUTOCLOSE bit.
* mailbox/temp_file_stream.c (mu_temp_file_stream_create: Force
MU_STREAM_AUTOCLOSE bit.
* mailbox/xscript-stream.c (_xscript_close): Always close the
transport stream (refcount will protect it, if necessary).
(mu_xscript_stream_create): Increase refcounts on both underlying streams,
unless MU_STREAM_AUTOCLOSE is set.
* pop3d/extra.c (pop3d_setio): Remove uses of MU_STREAM_NO_CLOSE.

* examples/nntpclient.c: Remove uses of MU_STREAM_NO_CHECK/MU_STREAM_NO_CLOSE.
* examples/pop3client.c: Likewise.
* libmu_auth/gsasl.c: Likewise.
* libproto/nntp/nntp_stream.c: Likewise.
* libproto/pop/pop3_stream.c: Likewise.
* mailbox/tcp.c: Likewise.
* mailbox/vartab.c: Likewise.
* mh/mh_list.c: Likewise.
* mimeview/mimeview.c: Likewise.
* mh/mhn.c: Likewise.
(edit_mime): Use MU_STREAM_AUTOCLOSE.

Bugfixes:

* mailbox/fltstream.c (init_iobuf): Fix input initialization.
(filter_write_internal): Bugfix.
* mailbox/stream.c (_stream_buffer_freespace): New macro.
(_stream_buffer_is_full): New macro.
(BUFFER_FULL_P): Remove, use _stream_buffer_is_full instead. All callers
updated.
(_stream_flush_buffer): Operation for full buffered streams does not
depend on the `all' flag.
(mu_stream_write): Fix calculation of the bytes available in the
buffer.

5 files changed, 9 insertions, 8 deletions

diff --git a/examples/base64.c b/examples/base64.c
index cc0a7e8b3..226cde96b 100644
--- a/examples/base64.c
+++ b/examples/base64.c
@@ -166,7 +166,8 @@ main (int argc, char * argv [])
   if (flags == MU_STREAM_READ)
     {
       MU_ASSERT (mu_filter_create (&flt, in, encoding, mode,
-				   MU_STREAM_READ|MU_STREAM_SEEK));
+				   MU_STREAM_READ|MU_STREAM_SEEK|
+				   MU_STREAM_AUTOCLOSE));
       if (shift)
 	MU_ASSERT (mu_stream_seek (flt, shift, MU_SEEK_SET, NULL));
       c_copy (out, flt);
@@ -174,7 +175,7 @@ main (int argc, char * argv [])
   else
     {
       MU_ASSERT (mu_filter_create (&flt, out, encoding, mode,
-				   MU_STREAM_WRITE));
+				   MU_STREAM_WRITE|MU_STREAM_AUTOCLOSE));
       if (shift)
 	MU_ASSERT (mu_stream_seek (in, shift, MU_SEEK_SET, NULL));
       c_copy (flt, in);
diff --git a/examples/mimetest.c b/examples/mimetest.c
index 257f133b0..0f4f1194a 100644
--- a/examples/mimetest.c
+++ b/examples/mimetest.c
@@ -253,7 +253,7 @@ message_display_parts (mu_message_t msg, int indent)
           /* Make sure the original body stream is not closed when
              str gets destroyed */
           mu_filter_create (&str, str, encoding, MU_FILTER_DECODE,
-			    MU_STREAM_READ | MU_STREAM_NO_CLOSE);
+			    MU_STREAM_READ);
 	  
 	  while (mu_stream_readline (str, buf, sizeof (buf), &nbytes) == 0
 		 && nbytes)
diff --git a/examples/murun.c b/examples/murun.c
index 59e4a3097..51b471082 100644
--- a/examples/murun.c
+++ b/examples/murun.c
@@ -71,6 +71,8 @@ main (int argc, char *argv[])
       MU_ASSERT (mu_stdio_stream_create (&in, MU_STDIN_FD, 0));
       MU_ASSERT (mu_stream_open (in));
       rc = mu_filter_prog_stream_create (&stream, cmdline, in);
+      /* Make sure closing/destroying stream will close/destroy in */
+      mu_stream_unref (in);
     }
   else
     rc = mu_prog_stream_create (&stream, cmdline, flags);
@@ -80,7 +82,7 @@ main (int argc, char *argv[])
 	       argv[0], mu_strerror (rc));
       exit (1);
     }
-
+  
   rc = mu_stream_open (stream);
   if (rc)
     {
diff --git a/examples/nntpclient.c b/examples/nntpclient.c
index 4fac95309..4c7722f3a 100644
--- a/examples/nntpclient.c
+++ b/examples/nntpclient.c
@@ -1004,8 +1004,7 @@ com_connect (char *arg)
       if (verbose)
 	com_verbose ("on");
       status =
-	mu_tcp_stream_create (&tcp, host, port,
-			   MU_STREAM_READ | MU_STREAM_NO_CHECK);
+	mu_tcp_stream_create (&tcp, host, port, MU_STREAM_READ);
       if (status == 0)
 	{
 	  mu_nntp_set_carrier (nntp, tcp);
diff --git a/examples/pop3client.c b/examples/pop3client.c
index 86acaca02..b8f3db2b2 100644
--- a/examples/pop3client.c
+++ b/examples/pop3client.c
@@ -800,8 +800,7 @@ com_connect (char *arg)
       if (verbose)
 	com_verbose ("on");
       status =
-	mu_tcp_stream_create (&tcp, argv[0], n,
-			      MU_STREAM_READ | MU_STREAM_NO_CHECK);
+	mu_tcp_stream_create (&tcp, argv[0], n, MU_STREAM_READ);
       if (status == 0)
 	{
 	  mu_pop3_set_carrier (pop3, tcp);