summaryrefslogtreecommitdiffabout
authorSergey Poznyakoff <gray@gnu.org>2019-02-13 12:42:26 (GMT)
committer Sergey Poznyakoff <gray@gnu.org>2019-02-13 12:49:44 (GMT)
commit49ab7b5fa7843a77605b91adf8a3689794fc091a (patch) (side-by-side diff)
treea66112caaaf9fe883ce449c788df1a8480594db9
parent7eb61c964c74af1bafb948584c0404208c9467b8 (diff)
downloadvmod-dbrw-49ab7b5fa7843a77605b91adf8a3689794fc091a.tar.gz
vmod-dbrw-49ab7b5fa7843a77605b91adf8a3689794fc091a.tar.bz2
Implement two new flags and a special req.http header to indicate errors
* NEWS: Document changes. * doc/vmod-dbrw.3: Likewise. * doc/vmod-dbrw.texi: Likewise. * configure.ac: Version 2.4.90 * src/dbrw.h (dbrw_config): New member: match_type * src/vmod_dbrw.c (parse_flags): Handle "eq" and "regex" flags. (findmatch): Handle the "eq" flag. (do_rewrite): Return error code. (vmod_rewrite): Set the X-VMOD-DBRW-Error header on errors. * tests/atlocal.in (at_vcl_backend): Special handling for X-VMOD-DBRW-Error. * tests/initdb.at: Test the 'eq' flag. * tests/rewrite07.at: New testcase. * tests/Makefile.am: Add new testcase. * tests/testsuite.at: Likewise.
Diffstat (more/less context) (ignore whitespace changes)
-rw-r--r--NEWS19
-rw-r--r--configure.ac4
-rw-r--r--doc/vmod-dbrw.324
-rw-r--r--doc/vmod-dbrw.texi38
-rw-r--r--src/dbrw.h1
-rw-r--r--src/vmod_dbrw.c110
-rw-r--r--tests/Makefile.am1
-rw-r--r--tests/atlocal.in9
-rw-r--r--tests/initdb.at4
-rw-r--r--tests/rewrite07.at40
-rw-r--r--tests/testsuite.at1
11 files changed, 198 insertions, 53 deletions
diff --git a/NEWS b/NEWS
index 0300ec3..516ca69 100644
--- a/NEWS
+++ b/NEWS
@@ -1,8 +1,23 @@
-vmod-dbrw -- history of user-visible changes. 2018-12-10
+vmod-dbrw -- history of user-visible changes. 2019-02-13
See the end of file for copying conditions.
Please send vmod-dbrw bug reports to <gray@gnu.org>
+Version 2.4.90 (git)
+
+* req.http.X-VMOD-DBRW-Error
+
+This header is set to 1 by dbrw.rewrite to indicate that an error
+occurred during the rewrite.
+
+* New flags: regex and eq
+
+One of this flags can appear in the fourth column of the returned data
+set. The 'eq' flag instructs dbrw.rewrite to use exact string match,
+instead of regular expressions. The 'regex' flag instructs it to use
+regular expression matching. It is the default.
+
+
Version 2.4, 2018-12-10
* Support for Varnish version 6.0.2
@@ -78,7 +93,7 @@ Initial release
=========================================================================
Copyright information:
-Copyright (C) 2013-2018 Sergey Poznyakoff
+Copyright (C) 2013-2019 Sergey Poznyakoff
Permission is granted to anyone to make or distribute verbatim copies
of this document as received, in any medium, provided that the
diff --git a/configure.ac b/configure.ac
index 62c3afe..fd618d3 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,5 +1,5 @@
# This file is part of vmod-dbrw -*- autoconf -*-
-# Copyright (C) 2013-2018 Sergey Poznyakoff
+# Copyright (C) 2013-2019 Sergey Poznyakoff
#
# Vmod-dbrw is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -14,7 +14,7 @@
# You should have received a copy of the GNU General Public License
# along with vmod-dbrw. If not, see <http://www.gnu.org/licenses/>.
AC_PREREQ(2.69)
-AC_INIT([vmod-dbrw], 2.4, [gray@gnu.org])
+AC_INIT([vmod-dbrw], 2.4.90, [gray@gnu.org])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_SRCDIR(src/vmod_dbrw.vcc)
diff --git a/doc/vmod-dbrw.3 b/doc/vmod-dbrw.3
index 2abef65..9bec008 100644
--- a/doc/vmod-dbrw.3
+++ b/doc/vmod-dbrw.3
@@ -13,7 +13,7 @@
.\"
.\" You should have received a copy of the GNU General Public License
.\" along with vmod-dbrw. If not, see <http://www.gnu.org/licenses/>.
-.TH VMOD-DBRW 3 "December 8, 2018" "VMOD-DBRW" "User Reference"
+.TH VMOD-DBRW 3 "February 13, 2019" "VMOD-DBRW" "User Reference"
.SH NAME
vmod-dbrw \- Database-driven rewrite rules for Varnish Cache
.SH SYNOPSIS
@@ -190,6 +190,9 @@ Treat \fBregexp\fR as case-insensitive.
.B case
Treat \fBregexp\fR as case-sensitive (default).
.TP
+.B eq
+Use exact string matching.
+.TP
.BR QSA " or " qsappend
Treat the resulting value as URL; append any query string from the
original \fBvalue\fR to it.
@@ -200,13 +203,30 @@ the original \fBvalue\fR.
.TP
\fBredirect=\fICODE\fR or \fBR=\fICODE\fR
On success, set the \fBX\-VMOD\-DBRW\-Status\fR header to \fICODE\fR,
-which must be a valid HTTP status code.
+which must be a valid HTTP status code.
+.TP
+.B regex
+Use regular expression matching. This is the default. This flag is
+provided for completeness sake, as a counterpart of
+.BR eq .
.PP
If \fBregexp\fR or \fBvalue\fR is NULL, the tuple is handled as
described in
.BR "Strict matches" .
.PP
If \fBflags\fR is NULL, it is ignored.
+.SH HTTP HEADERS
+Upon return,
+.B dbrw.return
+may set one of the following headers in
+.BR resp.http :
+.TP
+.B X\-VMOD\-DBRW\-Status
+If the \fBredirect\fR flag was used, this header contains the HTTP
+response code to be used instead of the default.
+.TP
+.B X\-VMOD\-DBRW\-Error
+This header is set to \fB1\fR if an error occurred during the rewrite.
.SH EXAMPLES
The examples in this section assume \fBMySQL\fR databases. Any
details not related to \fBvmod-dbrw\fR are omitted.
diff --git a/doc/vmod-dbrw.texi b/doc/vmod-dbrw.texi
index 734dadb..6a987ed 100644
--- a/doc/vmod-dbrw.texi
+++ b/doc/vmod-dbrw.texi
@@ -132,7 +132,14 @@ for substituting default values and invoking built-in functions during
the expansion of the query. @xref{Expansions}, for a detailed
description of these.
+@anchor{X-VMOD-DBRW-Error}
+@vindex X-VMOD-DBRW-Error
Having undergone expansions, the query is sent to the database server.
+If the query returns no records or if an error occured, @code{rewrite}
+returns empty string. In case of error, it also sets the HTTP header
+@samp{X-VMOD-DBRW-Error: 1}. It can be used in VLC code to provide a
+special handling for such failures.
+
The returned set of records (if non-empty) is processed depending on the
number of fields it contains.
@@ -166,7 +173,7 @@ resulting value is then returned to the caller.
@cindex flags
@anchor{flags}
Optional @var{flags} column is a comma-separated list of flags that
-modify regular expression handling:
+control the matching algorithm.
@table @samp
@kindex NC
@@ -183,6 +190,11 @@ Treat @var{regexp} as case-insensitive regular expression.
@item case
Treat @var{regexp} as case-sensitive (default).
+@kindex eq
+@cindex exact matching
+@item eq
+Use exact string matching.
+
@kindex QSA
@kindex qsappend
@cindex query string handling
@@ -209,6 +221,12 @@ the original @var{value}.
@item R=@var{code}
On success, set the @samp{X-VMOD-DBRW-Status} header to @var{code},
which must be a valid HTTP status code.
+
+@kindex regex
+@cindex regular expression matching
+@item regex
+Use regular expression matching. This is the default. This flag is
+provided for completeness sake, as a counterpart of @samp{eq}.
@end table
If @var{regexp} or @var{value} is NULL, strict matching is assumed
@@ -701,6 +719,24 @@ The @code{X-VMOD-DBRW-Status} header, if set, contains the status code to be
returned to the client (@pxref{X-VMOD-DBRW-Status}). Notice the use
of the @command{vmod_std} module to cast it to integer.
+If an error occured during the rewrite, it is recommended to not
+cache the response. This way the next request will call rewrite again
+and eventually complete the rewriting. This can be achieved using the
+following @code{vcl_backend_response} fragment:
+
+@example
+@group
+sub vcl_backend_response
+@{
+ if (bereq.http.X-VMOD-DBRW-Error == "1") @{
+ set beresp.uncacheable = true;
+ return (deliver);
+ @}
+@}
+@end group
+@end example
+
+
@node Reporting Bugs
@chapter How to Report a Bug
diff --git a/src/dbrw.h b/src/dbrw.h
index bd0aa2a..9a8adf6 100644
--- a/src/dbrw.h
+++ b/src/dbrw.h
@@ -71,6 +71,7 @@ struct dbrw_config {
char *query;
int qdisp;
int regflags;
+ int match_type;
char status[HTTP_STATUS_LEN+1];
int idle_timeout;
VTAILQ_ENTRY(dbrw_config) list;
diff --git a/src/vmod_dbrw.c b/src/vmod_dbrw.c
index 63d4ea1..beeb9af 100644
--- a/src/vmod_dbrw.c
+++ b/src/vmod_dbrw.c
@@ -1,5 +1,5 @@
/* This file is part of vmod-dbrw
- Copyright (C) 2013-2018 Sergey Poznyakoff
+ Copyright (C) 2013-2019 Sergey Poznyakoff
Vmod-dbrw is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -25,6 +25,11 @@ enum {
QDISP_DISCARD
};
+enum {
+ MATCH_REGEX,
+ MATCH_EQ
+};
+
void
dbrw_debug(const char *fmt, ...)
{
@@ -154,7 +159,7 @@ is_http_status(const char *arg)
}
static int
-parse_flags(const char *arg, int *qdisp, int *flags, char status[])
+parse_flags(const char *arg, int *qdisp, int *flags, int *match, char status[])
{
struct wordsplit ws;
int rc = 0;
@@ -204,6 +209,10 @@ parse_flags(const char *arg, int *qdisp, int *flags, char status[])
HTTP_STATUS_LEN);
status[HTTP_STATUS_LEN] = 0;
}
+ } else if (strcmp(ws.ws_wordv[i], "regex") == 0) {
+ *match = MATCH_REGEX;
+ } else if (strcmp(ws.ws_wordv[i], "eq") == 0) {
+ *match = MATCH_EQ;
} else {
dbrw_error("unrecognized flag: %s", ws.ws_wordv[i]);
rc = 1;
@@ -277,6 +286,7 @@ vmod_config(VRT_CTX, struct vmod_priv *priv,
conf->qdisp = QDISP_NONE;
conf->regflags = REG_EXTENDED;
+ conf->match_type = MATCH_REGEX;
conf->status[0] = 0;
s = findparam(conf->param, "flags");
@@ -284,6 +294,7 @@ vmod_config(VRT_CTX, struct vmod_priv *priv,
parse_flags(s,
&conf->qdisp,
&conf->regflags,
+ &conf->match_type,
conf->status);
AZ(pthread_mutex_lock(&config_pool_mtx));
@@ -413,6 +424,7 @@ findmatch(VRT_CTX, struct dbrw_connection *conn, char **param)
int rflags = conn->conf->regflags;
char status[HTTP_STATUS_LEN+1];
int qdisp = conn->conf->qdisp;
+ int match_type = conn->conf->match_type;
char *qry = NULL;
if (ISEMPTY(pat)) {
@@ -435,7 +447,7 @@ findmatch(VRT_CTX, struct dbrw_connection *conn, char **param)
strcpy(status, conn->conf->status);
if (nf == 4) {
if (parse_flags(sql_get_column(conn, i, 3),
- &qdisp, &rflags, status))
+ &qdisp, &rflags, &match_type, status))
continue;
}
@@ -456,38 +468,45 @@ findmatch(VRT_CTX, struct dbrw_connection *conn, char **param)
qry = NULL;
}
}
-
- rc = regcomp(&re, pat, rflags);
- if (rc) {
- char errbuf[512];
-
- regerror(rc, &re, errbuf, sizeof(errbuf));
- dbrw_error("cannot compile regexp \"%s\": %s",
- pat, errbuf);
- continue;
- }
- matchcount = re.re_nsub + 1;
- if (conn->matchsize < matchcount) {
- void *p = realloc(conn->matches,
- sizeof(conn->matches[0]) * matchcount);
- if (!p) {
- dbrw_error("not enough memory");
- regfree(&re);
+ if (match_type == MATCH_EQ) {
+ rc = ((rflags & REG_ICASE)
+ ? strcasecmp : strcmp) (pat, val);
+ } else {
+ rc = regcomp(&re, pat, rflags);
+ if (rc) {
+ char errbuf[512];
+
+ regerror(rc, &re, errbuf, sizeof(errbuf));
+ dbrw_error("cannot compile regexp \"%s\": %s",
+ pat, errbuf);
continue;
}
- conn->matches = p;
- conn->matchsize = matchcount;
- }
- rc = regexec(&re, val, matchcount, conn->matches, 0);
+ matchcount = re.re_nsub + 1;
+ if (conn->matchsize < matchcount) {
+ void *p = realloc(conn->matches,
+ sizeof(conn->matches[0]) * matchcount);
+ if (!p) {
+ dbrw_error("not enough memory");
+ regfree(&re);
+ continue;
+ }
+ conn->matches = p;
+ conn->matchsize = matchcount;
+ }
+
+ rc = regexec(&re, val, matchcount, conn->matches, 0);
- if (rc == 0)
- res = expand_backref(ctx, val,
- sql_get_column(conn, i, 0),
- matchcount, conn->matches, qry);
+ if (rc == 0)
+ res = expand_backref(ctx, val,
+ sql_get_column(conn, i, 0),
+ matchcount,
+ conn->matches, qry);
- regfree(&re);
+ regfree(&re);
+ }
+
if (rc == 0) {
debug(conn->conf, 1, ("match"));
if (status[0]) {
@@ -606,23 +625,24 @@ query_command_expand(char **ret, const char *cmd, size_t len, char **argv,
return expand_error(ret, argv[0], "unknown command");
}
-static char *
-do_rewrite(VRT_CTX, struct dbrw_connection *cp, VCL_STRING arg)
+static int
+do_rewrite(VRT_CTX, struct dbrw_connection *cp, VCL_STRING arg, char **result)
{
struct wordsplit ws, wsenv;
int i, rc;
- char *res;
int wsflags;
+
+ *result = NULL;
if (sql_connect(cp) || cp->state != state_connected)
- return NULL;
+ return -1;
debug(cp->conf, 2, ("vmod_rewrite: splitting arg"));
wsenv.ws_delim = ";";
if (wordsplit(arg, &wsenv, WRDSF_NOVAR|WRDSF_NOCMD|WRDSF_DELIM)) {
dbrw_error("cannot split string `%s': %s",
arg, wordsplit_strerror(&wsenv));
- return NULL;
+ return -1;
}
if (cp->conf->backend->sql_escape) {
@@ -632,7 +652,7 @@ do_rewrite(VRT_CTX, struct dbrw_connection *cp, VCL_STRING arg)
if (!p) {
dbrw_error("cannot expand argument");
wordsplit_free(&wsenv);
- return NULL;
+ return -1;
}
free(wsenv.ws_wordv[i]);
wsenv.ws_wordv[i] = p;
@@ -657,24 +677,24 @@ do_rewrite(VRT_CTX, struct dbrw_connection *cp, VCL_STRING arg)
dbrw_error("cannot expand query `%s': %s",
cp->conf->query, wordsplit_strerror(&ws));
wordsplit_free(&wsenv);
- return NULL;
+ return -1;
}
rc = sql_query(cp, ws.ws_wordv[0]);
wordsplit_free(&ws);
if (rc) {
- debug(cp->conf, 1, ("vmod_rewrite: query failed"));
+ dbrw_error("vmod_rewrite: query failed");
wordsplit_free(&wsenv);
- return NULL;
+ return -1;
}
- res = findmatch(ctx, cp, wsenv.ws_wordv);
+ *result = findmatch(ctx, cp, wsenv.ws_wordv);
wordsplit_free(&wsenv);
sql_free_result(cp);
- return res;
+ return 0;
}
VCL_STRING
@@ -683,13 +703,17 @@ vmod_rewrite(VRT_CTX, struct vmod_priv *priv, VCL_STRING arg)
struct dbrw_config *conf = priv->priv;
struct dbrw_connection *cp;
char *res = NULL;
-
+ int rc;
+
AN(conf);
debug(conf, 2, ("vmod_rewrite(%s) begin", arg));
cp = dbrw_connection_get(conf);
AN(cp);
- res = do_rewrite(ctx, cp, arg);
+ rc = do_rewrite(ctx, cp, arg, &res);
dbrw_connection_release(cp);
- debug(conf, 1, ("vmod_rewrite: res=%s", res ? res : "(NULL)"));
+ debug(conf, 1, ("vmod_rewrite: %d, res=%s", rc, res ? res : "(NULL)"));
+ if (rc) {
+ dbrw_sethdr(ctx, HDR_REQ, "\022X-VMOD-DBRW-Error:", "1");
+ }
return res;
}
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 792ce24..28b795f 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -49,6 +49,7 @@ TESTSUITE_AT = \
rewrite04.at\
rewrite05.at\
rewrite06.at\
+ rewrite07.at\
testsuite.at
TESTSUITE = $(srcdir)/testsuite
diff --git a/tests/atlocal.in b/tests/atlocal.in
index 18a0b69..14f3fd4 100644
--- a/tests/atlocal.in
+++ b/tests/atlocal.in
@@ -51,10 +51,17 @@ varnish v1 -vcl+backend {
set req.http.X-Redirect-To =
dbrw.rewrite("host=" + req.http.Host + ";" +
"url=" + req.url);
- return(synth(301, "Redirect"));
+ if (req.http.X-Redirect-To != "") {
+ return(synth(301, "Redirect"));
+ } else if (req.http.X-VMOD-DBRW-Error == "1") {
+ return(synth(500, "DBRW Error"));
+ }
}
sub vcl_synth {
+ if (req.http.X-VMOD-DBRW-Error == "1") {
+ set resp.http.X-VMOD-DBRW-Error = "1";
+ }
if (resp.status == 301) {
if (req.http.X-VMOD-DBRW-Status) {
set resp.status = std.integer(req.http.X-VMOD-DBRW-Status, 301);
diff --git a/tests/initdb.at b/tests/initdb.at
index 0355e3b..f7bfe0b 100644
--- a/tests/initdb.at
+++ b/tests/initdb.at
@@ -44,7 +44,7 @@ CREATE TABLE rewrite (
CREATE INDEX source ON rewrite(host,url);
INSERT INTO rewrite VALUES
-('en.example.net','/local','http://uno.example.com/remote',NULL,NULL,NULL),
+('en.example.net','/local','http://uno.example.com/remote',NULL,NULL,'eq'),
('en.example.net','/local','http://dos.example.com/$[]1','$url','/local/(.*)',NULL),
('en.example.net','/local2','http://to.example.net/$[]1$[]2','$url','/local2/([[^\\?]]*)(\\?.*)?',NULL),
('to.example.net','/local','http://dos.example.net/$[]1','$url','/local/(.*)','QSA'),
@@ -57,4 +57,4 @@ EOT
[],
[mv err $FAILFILE],
[echo "OK" > $INITFILE])
-]) \ No newline at end of file
+])
diff --git a/tests/rewrite07.at b/tests/rewrite07.at
new file mode 100644
index 0000000..6554fc7
--- a/dev/null
+++ b/tests/rewrite07.at
@@ -0,0 +1,40 @@
+# This file is part of vmod-dbrw -*- autotest -*-
+# Copyright (C) 2013-2018 Sergey Poznyakoff
+#
+# Vmod-dbrw is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# Vmod-dbrw is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with vmod-dbrw. If not, see <http://www.gnu.org/licenses/>.
+
+AT_SETUP(Rewrite 7 - SQL error handling)
+AT_KEYWORDS(rewrite rewrite07)
+
+AT_DBRW_INIT
+
+AT_CHECK([
+AT_DBINIT_PREREQ
+# This test simulates rewrite failure in order to check whether
+# the X-VMOD-DBRW-Error header is correctly set on return.
+AT_VCL([SELECT dest,pattern,value,flags
+ FROM rewrite
+ WHERE host='$BADHOST' AND url IN ($(urlprefixes $url))
+ ORDER BY LENGTH(dest),value DESC],
+ [txreq -url /local/foo/bar?x&y&z -hdr "Host:tre.example.net"
+ rxresp
+ expect resp.status == 500
+ expect resp.http.X-VMOD-DBRW-Error == 1
+])
+AT_VARNISHTEST
+],
+[0],
+[OK
+])
+AT_CLEANUP
diff --git a/tests/testsuite.at b/tests/testsuite.at
index 5858b6a..203b667 100644
--- a/tests/testsuite.at
+++ b/tests/testsuite.at
@@ -60,6 +60,7 @@ m4_include([rewrite03.at])
m4_include([rewrite04.at])
m4_include([rewrite05.at])
m4_include([rewrite06.at])
+m4_include([rewrite07.at])
# End of testsuite.at

Return to:

Send suggestions and report system problems to the System administrator.