summaryrefslogtreecommitdiffabout
path: root/doc/vmod-dbrw.3
blob: 9c35772a96587a1fb951cb3074ab16f70d217f94 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
.\" This file is part of Vmod-dbrw -*- nroff -*-
.\" Copyright (C) 2013 Sergey Poznyakoff
.\"
.\" Vmod-dbrw is free software; you can redistribute it and/or modify
.\" it under the terms of the GNU General Public License as published by
.\" the Free Software Foundation; either version 3, or (at your option)
.\" any later version.
.\"
.\" Vmod-dbrw is distributed in the hope that it will be useful,
.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
.\" GNU General Public License for more details.
.\"
.\" You should have received a copy of the GNU General Public License
.\" along with vmod-dbrw.  If not, see <http://www.gnu.org/licenses/>.
.TH VMOD_DBRW 1 "July 19, 2013" "VMOD_DBRW" "User Reference"
.SH NAME
vmod-dbrw \- Database-driven rewrite rules for Varnish Cache
.SH SYNOPSIS
.sp
.nf
import dbrw;

sub vcl_init {
    dbrw.config("\fIDBTYPE\fR", "\fIPARAMS\fR", "\fIQUERY\fR");
}

sub vcl_recv {
    set req.http.\fIXHEADER\fR = dbrw.rewrite(\fIARGS\fR);
    if (req.http.\fIXHEADER\fR) {
        error(750, "Redirect");
    }
}

sub vcl_error {
    if (obj.status == 750) {
        set obj.http.Location = req.http.\fIXHEADER\fR;
        set obj.status = 301;
        return (deliver);
    }
}
.fi
.SH DESCRIPTION
.B Vmod-dbrw
is a Varnish Cache module implementing database-driven rewrite procedures.
Its intended use is for web sites that need an exceedingly big number
of redirect and/or rewrite rules.
.PP
Upon startup, the \fBconfig\fR function should be called.  This call
configures the module and supplies it the information necessary for
accessing the database and interpreting the data obtained from it.
.PP
Two database types are supported: \fBMySQL\fR or \fBPostgreSQL\fR.
The first argument \fBconfig\fR instructs the module which type to
use.
.PP
.B Vmod-dbrw
does not impose any requirements on the structure of the database,
therefore the connection arguments and access credentials may vary
depending on the flavor of the database used.  This information is
supplied in the \fIPARAMS\fR argument, which is a list of parameter
assignments separated with semicolons.
.PP
Once configured, the \fBrewrite\fR function can be called in the
appropriate place of the Varnish configuration file.  Its argument
\fIARGS\fR is a list of variable assignments separated with
semicolons, similarly to the \fIPARAMS\fR argument described above.
Each assignment has the form \fINAME\fB=\fIVALUE\fR.  When called
the function expands the \fBSQL\fR query, supplied with the \fIQUERY\fR
argument to the \fBconfig\fR call, by replacing each \fB$\fINAME\fR
construct (a \fBvariable reference\fR) with the corresponding
\fIVALUE\fR from its argument.  Similarly to the shell syntax, the
variable reference can be written as \fB${\fINAME\fB}\fR.  This latter
form can be used in contexts where the variable name is immediately
followed by another letter, to prevent it from being counted as a part
of the name.
.PP
The expanded query is then sent to the database server.  The handling
of the return value depends on the number of fields it contains.
.SS Strict matches
If the returned set consists of one or two columns, only the
first tuple is used and the value of its first column is returned.
The second column (if present) is ignored.
.SS Regexp matches
Otherwise, if the returned set consists of three or four columns, the
columns are interpreted as follows: \fBresult\fR, \fBregexp\fR,
\fBvalue\fR and optional \fBflags\fR.  For each returned tuple, the
\fBvalue\fR column undergoes variable expansion, using the same
algorithm as when preparing the query, and the resulting string is
matched with the \fBregexp\fR column, which is treated as an extended
\fBPOSIX\fR regular expression.  If the match occurs, the \fBresult\fR
column is expanded by replacing \fBbackreferences\fR.  Each occurrence
of \fB$\fIDIGIT\fR (where \fIDIGIT\fR stands for a decimal digit from
\fB0\fR through \fB9\fR) is replaced with the contents of the
\fIDIGIT\fR's parenthesized subexpression in \fBregexp\fR.  For
compatibility with the traditional usage, the \fB\\\fIDIGIT\fR
notation is also allowed.  The resulting value is returned.
.PP
Optional \fBflags\fR column is a comma-separated list of flags that
modify regular expression handling.  The following flags are defined:
.TP
.BR NC " or " nocase
Treat \fBregexp\fR as case-insensitive.
.TP
.B case
Treat \fBregexp\fR as case-sensitive (default).
.TP
.BR QSA " or " qsappend
Treat the resulting value as URL; append any query string from the
original \fBvalue\fR to it.
.TP
.BR QSD " or " qsdiscard
Treat the resulting value as URL; discard any query string attached to
the original \fBvalue\fR.
.TP
\fBredirect=\fICODE\fR or \fBR=\fICODE\fR
On success, set the \fBX\-VMOD\-DBRW\-Status\fR header to \fICODE\fR,
which must be a valid HTTP status code. 
.PP
If \fBregexp\fR or \fBvalue\fR is NULL, the tuple is handled as
described in
.BR "Strict matches" .
.PP
If \fBflags\fR is NULL, it is ignored.
.SH FUNCTIONS
.SS config
.TP
.B Prototype
config(STRING \fIDBTYPE\fR, STRING \fIPARAMS\fR, STRING \fIQUERY\fR)
.TP
.B Return value
VOID
.TP
.B Description
This function configures the module and supplies the data necessary to
connect and use the database.
.TP
.B Arguments
.RS 4
.TP
.I DBTYPE
Type of the database.  Valid values are \fBmysql\fR (for \fBMySQL\fR)
and \fBpgsql\fR (for \fBPostgreSQL\fR).
.TP
.I PARAMS
Database connection parameters.  This is a list of
\fINAME\fB=\fIVALUE\fR assignments separated with semicolons.  The
\fIVALUE\fR can be any sequence of characters, excepting white space
and semicolon.  If any of these have to appear in it, they must either
be escaped by prepending them with a backslash, or entire \fIVALUE\fR
enclosed in a pair of (single or double) quotes.  The following
\fBescape sequences\fR are allowed for use in \fIVALUE\fR:
.sp
.nf
.ta 8n 18n 42n
.ul
	Sequence	Expansion	ASCII
	\fB\\\\\fR	\fB\\\fR	134
	\fB\\"\fR	\fB"\fR	042
	\fB\\a\fR	audible bell	007	
	\fB\\b\fR	backspace	010
	\fB\\f\fR	form-feed	014
	\fB\\n\fR	new line	012
	\fB\\r\fR	charriage return	015
	\fB\\t\fR	horizontal tabulation	011
	\fB\\v\fR	vertical tabulation	013
.fi
.sp
Any other character following a backslash is output verbatim.
.sp
The valid parameters are:
.RS 8
.TP
\fBdebug\fR=\fIN\fR
Set debugging level.  \fIN\fR is a decimal number.
.TP
\fBserver\fR=\fIHOST\fR
Name or IP address of the database server to connect to.  If not
defined, localhost (\fB127.0.0.1\fR) is assumed.  For \fBMySQL\fR
databases, if \fIHOST\fR begins with a slash (\fB/\fR), its value is
taken to be the full pathname of the local UNIX socket to connect to.
.TP
\fBport\fR=\fINUM\fR
Port number on the \fBserver\fR to connect to.  Default is \fB3306\fR
for \fBMySQL\fR and \fB5432\fR for \fBPostgres\fR.
.TP
\fBdatabase=\fINAME\fR
The database name.
.TP
\fBconfig\fR=\fIFILE\fR
(\fBMySQL\fR-specific) Read credentials from the \fBMySQL\fR options
file \fIFILE\fR.
.TP
\fBgroup\fR=\fINAME\fR
(\fBMySQL\fR-specific) Read credentials from section \fINAME\fR of the
options file supplied with the \fBconfig\fR parameter.  Default
section name is \fBclient\fR.
.TP
\fBcacert\fR=\fIFILE\fR
Use secure connection to the database server via SSL.  The \fIFILE\fR
is a full pathname to the certificate authority file.
.TP
\fBoptions\fR=\fISTRING\fR
(\fBPostgres\fR-specific) Connection options.
.TP
\fBuser\fR=\fINAME\fR
Database user name.
.TP
\fBpassword\fR=\fISTRING\fR
Password to access the database.
.RE
.TP
.I QUERY
\fBSQL\fR query to use.  It can contain variable references in the
form \fB$\fINAME\fR or \fB${\fINAME\fB|\fR, which will be replaced
prior to the execution with the actual value of the \fINAME\fR
argument to the function \fBrewrite\fR.
.RE
.SS rewrite
.TP
.B Prototype
rewrite(STRING \fIARGS\fR)
.TP
.B Return value
STRING
.TP
.B Description
Rewrites the argument according to the database.  See the section
\fBDESCRIPTION\fR for a detailed discussion.
.SH EXAMPLES
The examples in this section assume \fBMySQL\fR databases.  Any
details not related to \fBvmod-dbrw\fR are omitted.  The \fBsub
vcl_error\fR is responsible for actual returning the 301 or 302
error.  If not shown it is the same as in section \fBSYNOPSIS\fR.
.SS Redirects
.PP
This example shows how to implement apache-style permanent redirects
in Varnish.
.PP
The following table is used to keep the redirection data:
.sp
.nf
.in +2
CREATE TABLE redirects (
    host varchar(255) NOT NULL DEFAULT '',
    url varchar(255) NOT NULL DEFAULT '',
    dest varchar(255) DEFAULT NULL,
    PRIMARY KEY (host,url)
);
.in
.fi
.PP
VCL code:
.sp
.nf
.in +2
sub vcl_init {
    dbrw.config("mysql", "database=dbname;user=varnish",
                {"SELECT dest
                    FROM redirects
                   WHERE host='$host'
                     AND url='$url'"});
}

sub vcl_recv {
    set req.http.X-Redirect-To =
        dbrw.rewrite("host=" + req.http.Host + ";" +
                     "url=" + req.url);
    if (req.http.X-Redirect-To != "") {
        error(750, "Redirect");
    }
}
.in
.fi
.PP
Notice the use of concatenation to build the argument to
\fBdbrw.rewrite\fR.
.SS Rewrites.
.PP
The database structure is as follows:
.sp
.nf
.in +2
CREATE TABLE rewrite (
    host varchar(255) NOT NULL DEFAULT '',
    url varchar(255) NOT NULL DEFAULT '',
    dest varchar(255) DEFAULT NULL,
    value varchar(255) DEFAULT NULL,
    pattern varchar(255) DEFAULT NULL,
    flags char(64) DEFAULT NULL,
    KEY source (host,url)
)
.in
.fi
.PP
VCL code:
.sp
.nf
.in +2
import std;   # The std module is used by \fBvcl_error\fR below
import dbrw;

sub vcl_init {
    # It is supposed that the url column contains an SQL-style
    # wildcard pattern.
    dbrw.config("mysql", "database=varnish;user=varnish;debug=10",
                {"SELECT dest,pattern,value,flags FROM rewrite
                  WHERE host='$host' and '$url' like url"});
}

sub vcl_recv {
    set req.http.X-Redirect-To =
        dbrw.rewrite("host=" + req.http.Host + ";" +
                     "url=" + req.url);
    if (req.http.X-Redirect-To != "") {
        error(750, "Redirect");
    }
}
.in
.fi
.PP
The \fBvcl_error\fR function uses the \fBX\-VMOD\-DBRW\-Status\fR
header, which may be set by \fBredirect=\fR flag in the
\fBflags\fR column.  Its value is converted to integer using the
\fBinteger\fR function from the \fBstd\fR module.  If the header is
not set, permanent redirect is assumed:
.sp
.nf
.in +2
sub vcl_error {
    if (obj.status == 750) {
        set obj.http.Location = req.http.X-Redirect-To;
        set obj.status = std.integer(req.http.X-VMOD-DBRW-Status, 301);
        return (deliver);
    }
}
.in
.fi
.SS Use of vmod-dbrw and vmod-redirect
.PP
Using the \fBvmod-redirect\fR module
.BR ( https://www.varnish-cache.org/vmod/redirect )
you can get rid of the \fBvcl_error\fR subroutine.
.sp
.nf
.in +2
import std;
import dbrw;
import redirect;

sub vcl_recv {
    set req.http.X-Redirect-To =
        dbrw.rewrite("host=" + req.http.Host + ";" +
                     "url=" + req.url);
    if (req.http.X-Redirect-To != "") {
        error(redirect.location(
	        std.integer(req.http.X-VMOD-DBRW-Status, 301),
	        req.http.X-Redirect-To), "Redirection");
    }
}    
.in
.fi
.SH "SEE ALSO"
.BR vcl (7),
.BR varnishd (1).
.PP
The full documentation for
.B vmod-dbrw
is maintained as a Texinfo
manual.  If the
.B info
program and
.B vmod-dbrw
module are properly installed at your site, the  command
.sp
.nf
.in +4
.B info vmod-dbrw
.in
.fi
.PP
should give you access to the complete manual.
.SH AUTHORS
Sergey Poznyakoff
.SH "BUG REPORTS"
Report bugs to <gray@gnu.org>.
.SH COPYRIGHT
Copyright \(co 2013 Sergey Poznyakoff
.br
.na
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
.br
.ad
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
.\" Local variables:
.\" eval: (add-hook 'write-file-hooks 'time-stamp)
.\" time-stamp-start: ".TH [A-Z_][A-Z0-9_.-]* [0-9] \""
.\" time-stamp-format: "%:B %:d, %:y"
.\" time-stamp-end: "\""
.\" time-stamp-line-limit: 20
.\" end:

Return to:

Send suggestions and report system problems to the System administrator.