Age | Commit message (Collapse) | Author | Files |
|
|
|
* src/grecs-lex.l: Fix preprocessor mode.
Fix regex in the STR state.
* tests/gcfpp.c: New option -a: enable string location adjustment.
* tests/ipp.at: Add two new tests.
* tests/locus01.at: Fix locations (string location adjustment works
now for here-documents as well).
|
|
|
|
|
|
|
|
When external preprocessing is required, each source file is preprocessed
separately. This means that if you use #include or #include_once
statements, a separate copy of external preprocessor will be started
for each included file. In other words, when both internal and external
preprocessing is used, the order is: external first, internal next.
Previous versions of grecs implemented opposite order, when one external
preprocessor was started, which obtained input from the internal
preprocessor.
One concequence of this is that any macros defined in a configuration file
are no longer visible for files included from it.
* include/grecs/lex.h (grecs_parse_line_directive)
(grecs_parse_line_directive_cpp): Change signature.
* include/grecs/preproc.h (grecs_preproc_init)
(grecs_preproc_done)
(grecs_preproc_run): Remove.
(grecs_preprocess): New proto.
* src/grecs-lex.l: Rewrite.
* src/Make.am: Remove preproc.c
* src/preproc.c: Remove.
* src/bind-lex.l: Update calls to grecs_parse_line_directive*
functions.
* src/dhcpd-lex.l: Likewise.
* tests/Makefile.am: Add new tests.
* tests/testsuite.at: Inlcude new tests.
* tests/epp.at: New file.
* tests/ipp.at: New file.
* tests/fpp.c: Rename to tests/failpp.c
* tests/gcfpp.c: New file.
* tests/incl01.at: Change expected strings.
* tests/ppexit.at: Change expected strings.
|
|
* src/grecs-gram.y (yyerror): Suppress error diagnostics if
grecs_yyerror_suppress is set.
* src/grecs-lex.l (yywrap): Suppress error diagnostics on
external preprocessor failure and on #abend.
* src/preproc.c (grecs_preproc_run): Return exit status of
the preprocessor or -1 on errors.
(grecs_preproc_extrn_start): Likewise.
* tests/fpp.c: New file.
* tests/gcffmt.c: New option: -preproc
* tests/abend.at: Test error suppression after #abend.
* tests/ppexit.at: New file.
* tests/.gitignore: Add fpp.
* tests/Makefile.am: Add new files.
* tests/testsuite.at: Include ppexit.at
|
|
* doc/grecs-syntax.texi: Document new preprocessor statements.
* include/grecs/preproc.h (grecs_preproc_extrn_shutdown): Return int.
* src/grecs-lex.l: Handle new preprocessor statements.
* src/preproc.c (grecs_preproc_extrn_shutdown): Return error code.
Bail out unless the preprocessor command terminates with code 0.
* tests/Makefile.am: Add new tests.
* tests/testsuite.at: Include new tests.
* tests/abend.at: New test.
* tests/error.at: Likewise.
* tests/warning.at: Likewise.
|
|
* tests/cfhelp.at: Reflect changes in wording.
|
|
|
|
|
|
* am/grecs.m4: Quote the use of $DEFAULT_PREPROCESSOR as argument to
expr.
* tests/Makefile.am: Fix LDADD
|
|
* src/.gitignore: Remove grecs.h
* tests/Makefile.am: Fix AM_CPPFLAGS
|
|
* .gitmodules: New module: wordsplit
* Makefile.am: Distribute wordsplit files.
* am/grecs.m4 (GRECS_INCLUDES): Add wordsplit directory
* doc/Makefile.am (dist_man_MANS): Distribute wordsplit/wordsplit.3
* doc/wordsplit.3: Remove.
* include/Makefile.am (GRECS_HDR): Add wordsplit/wordsplit.
* include/wordsplit.h: Remove.
* src/Make-inst.am: Define nodist_libgrecs_la_SOURCES.
* src/Make-shared.am: Likewise.
* src/Make-static.am: Define nodist_libgrecs_a_SOURCES.
* src/Make.am (GRECS_SRC): Remove wordsplit.c
(NODIST_GRECS_SRC): Define to wordsplit.c
(VPATH): Add the wordsplit directory.
* src/wordsplit.c: Remove.
* tests/wordsplit.at: Remove.
* tests/wsp.c: Remove.
* tests/Makefile.am: Build separate testsuite for wordsplit. Use VPATH
to build wsp.
* tests/.gitignore: Update.
|
|
Up to now the parser incorrectly accepted the use of compound statement
keyword in scalar context, unless the keyword description had non NULL
value in the callback field. This change fixes it.
It also defines several assertion functions for use inm callbacks.
* src/tree.c (grecs_value_type_string): New function.
(find_keyword): Take into account node type.
(nodeproc,reduceproc): Leave error reporting to find_keyword.
* tests/set.at: Change expected error string.
* include/grecs/tree.h (grecs_assert_value_type)
(grecs_assert_scalar_stmt)
(grecs_assert_node_value_type): New protos.
* include/grecs/value.h (grecs_value_type_string): New proto.
* src/Make.am: Add assert.c
* src/assert.c: New file.
* src/format.c (grecs_data_type_string): Add i18n markers to the
returned strings.
|
|
* include/wordsplit.h (wordsplit) <ws_errctx>: New field.
* src/wordsplit.c (wordsplit_init): Initialize ws_errctx.
(expvar,wordsplit_pathexpand): Save error context.
(wordsplit_clearerr): Free error context.
(wordsplit_perror): Use error context if available.
* tests/wordsplit.at: Reflect changes.
|
|
|
|
|
|
This improves 3e07e3ad
* include/wordsplit.h (ws_paramv,ws_paramc)
(ws_parambuf,ws_paramidx,ws_paramsiz): New fields.
(WRDSO_PARAMV,WRDSO_PARAM_NEGIDX): New options.
(WRDSE_BADPARAM): New error code.
(wordsplit_free_parambuf): New proto.
* src/wordsplit.c (wordsplit_init): Initialize new fields.
(wsplt_assign_var): Fix double-free and memory leak.
(expvar): Expand positional parameters.
(begin_var_p): Add '#'
(wordsplit_free_envbuf): Fix condition.
(wordsplit_free_parambuf): New function.
(wordsplit_free): Call wordsplit_free_parambuf.
(_wordsplit_errstr): New error description.
* tests/wordsplit.at: Update wsp invocations.
Test positional parameters.
* tests/wsp.c: Rewrite.
|
|
refs
* include/wordsplit.h (WRDSO_NOVARSPLIT)
(WRDSO_NOCMDSPLIT): New options.
* src/wordsplit.c (scan_word): Treat any variable reference, even
containing whitespace, as a single word if WRDSO_NOVARSPLIT is set.
Ditto for commands and WRDSO_NOCMDSPLIT.
* tests/wordsplit.at: Add new tests.
* tests/wsp.c: Recognize novarsplit and nocmdsplit
options.
For future use: recognize bskeep_words, bskeep_quote, bskeep.
|
|
* src/wordsplit.c (wordsplit_finish): Make sure at one non-null
argument is always returned in WRDSF_NOSPLIT mode.
* tests/wordsplit.at: Add new test.
|
|
New variable GRECS_HOST_PROJECT_LDADD complements
GRECS_HOST_PROJECT_INCLUDES. It provides a way to link test binaries
with host project libraries. This may be useful if config.h redefines
missing or defective library functions to their replacements, which are
provided in third-party libraries, such as e.g. gnulib,
* am/grecs.m4 (GRECS_HOST_PROJECT_LDADD): New substitution variable.
* tests/Makefile.am (LDADD): Add GRECS_HOST_PROJECT_LDADD.
|
|
* include/wordsplit.h (WRDSO_ARGV): Remove.
* src/wordsplit.c (expcmd): Always split command line into arguments.
This fixes https://savannah.gnu.org/bugs/?54830
* tests/wsp.c: Implement internal commands, instead of
calling shell ones. This fixes https://savannah.gnu.org/bugs/?54829.
* tests/wordsplit.at: Rewrite command expansion tests.
|
|
* tests/wordsplit.at (append + dooffs + env): Disable environment.
|
|
This feature allows the user to limit the number of words returned
by a call to wordsplit. When the number of words in expansion reaches
the predefined limit, the rest of input line will be expanded and returned
as a single last word.
For example, to parse a /etc/passwd line:
struct wordsplit ws;
ws.ws_delim = ":";
ws.ws_maxwords = 7;
ws.ws_options = WRDSO_MAXWORDS;
wordsplit(str, &ws, WRDSF_NOVAR | WRDSF_NOCMD | WRDSF_DELIM | WRDSF_OPTIONS);
* doc/wordsplit.3: Document the maxwords feature.
* include/wordsplit.h (wordsplit) <ws_maxwords>
<ws_wordi>: New members.
(WRDSO_MAXWORDS): New option.
* src/wordsplit.c (WSP_RETURN_DELIMS): New macro.
(_wsplt_subsplit): Rewrite.
(wordsplit_init0): Don't reset node list.
(wordsplit_init): Initialize ws_wordi and the node list.
(wsnode_insert): Correctly insert lists.
(coalesce_segment): Additional safety check.
(wsnode_tail_coalesce): New static function.
(wordsplit_finish): Postprocess delimiters.
(expvar,expcmd): Use new _wsplt_subsplit.
(wordsplit_varexp): Don't try to expand delimiter nodes.
(skip_delim): Remove delimiter processing. It is now done
in wordsplit_finish.
(scan_word): New argument 'consume_all' instructs it to
consume the rest of input as one token.
(wordsplit_process_list): Handle wsp->ws_maxwords setting.
This also fixed a long-standing bug: quotes weren't processed
in WRDSF_NOSPLIT mode. See the testcase 59 (incremental nosplit).
(wordsplit_run): Rewrite.
(wordsplit_free): Free node list.
* tests/wordsplit.at: Update for the new wsp output format.
(incremental nosplit): Expect correct output.
Add tests for the maxwords feature.
* tests/wsp.c (maxwords): New flag.
Print the ws_wordi value as "TOTALS" at the end of each run.
|
|
This includes the following commits pushed between
2015-09-19 and 2017-10-10:
090c7b9a Allow ws_getvar to set value to NULL and
return MU_WRDSE_OK.
The value is processed as if it were "", i.e.
MU_WRDSE_UNDEF is returned.
64313fdf Fix MU_WRDSF_INCREMENTAL | MU_WRDSF_NOSPLIT
46d7640f Add wordsplit_append function
151eb4b9 Fix nested expansions and command expansions
occurring after variable expansions.
ad3cc340 Replace void wordsplit_getwords with
int wordsplit_get_words.
* include/wordsplit.h (wordsplit_get_words): New function.
(wordsplit_getwords): Mark as deprecated.
(wordsplit_append): New function.
* src/wordsplit.c (wordsplit_append): New function. MU 46d7640f.
(expvar): Treat NULL value as "". MU 090c7b9a.
(expcmd): Allow command and variable expansions in subsplit.
(exptab): Change ordering of expansions so that command expansion
occurs first. This fixes nested expansions and command expansions
occurring after variable expansions. MU 151eb4b9.
(wordsplit_process_list): Update wsp->ws_endp in nosplit mode. This
fixes wordsplit MU_WRDSF_INCREMENTAL | MU_WRDSF_NOSPLIT. MU 64313fdf.
(wordsplit_get_words): New function. MU ad3cc340.
* tests/wordsplit.at: Test the above changes.
* tests/wsp.c: Accept extra arguments to append using wordsplit_append.
|
|
* tests/gcfpeek.c: New options: -strcat -stradj
* tests/locus02.at: Use these.
|
|
The adjacent string concatenation feature doesn't work well with
multiple-argument statements. The common example (found in GNU
pies) is the "env" statement:
env "PATH=/sbin:$PATH" "PRELOAD=true"
The intent was to pass it two arguments, but in fact they get
concatenated into one.
To control this, the grecs_parser_options variable is introduced.
If it has the GRECS_OPTION_QUOTED_STRING_CONCAT bit set, adjacent
string concatenation is enabled. By default it is disabled.
The GRECS_OPTION_ADJUST_STRING_LOCATIONS bit controls the way the
quoted sring locations are computed. If it is set, the beginning of
the string is counted at the first character after the opening double
quote, and its end at the character immediately preceding the closing
double quote. Otherwise, both double-quote characters are included in
the location (the default).
The change is backward incompatible.
* doc/grecs-syntax.texi: Update.
* include/grecs/parser.h (grecs_adjust_string_locations): Remove.
(grecs_parser_options): New extern.
* src/grecs-lex.l: QSTRING is returned only if
GRECS_OPTION_QUOTED_STRING_CONCAT option is set.
The GRECS_OPTION_ADJUST_STRING_LOCATIONS option controls string locus
adjustment.
* src/parser.c (grecs_adjust_string_locations): Remove.
(grecs_parser_options): New variable.
* tests/Makefile.am: Add strcat.at, stradj.at
* tests/testsuite.at: Likewise.
* tests/gcffmt.c: New options -strcat and -stradj
* tests/stradj.at: New test case.
* tests/strcat.at: New test case.
|
|
* Makefile.am (SUBDIRS): Add include
* am/grecs.m4 (GRECS_INCLUDES): Update
(GRECS_COND_BUILD_INSTALL)
(GRECS_COND_BUILD_SHARED)
(GRECS_COND_BUILD_STATIC): New conditions.
Build include/Makefile and include/grecs/Makefile.
* configure.ac: Use src/grecs-gram.y as a source
validity marker
* include/Makefile.a: New file.
* src/grecs.hin: Remove. Split into include/*. See below.
* include/grecs.h: New file.
* include/grecs/.gitignore: New file.
* include/grecs/Makefile.am: New file.
* include/grecs/doc.h: New file.
* include/grecs/error.h: New file.
* include/grecs/format.h: New file.
* include/grecs/lex.h: New file.
* include/grecs/list.h: New file.
* include/grecs/mem.h: New file.
* include/grecs/node.h: New file.
* include/grecs/parser.h: New file.
* include/grecs/preproc.h: New file.
* include/grecs/sockaddr.h: New file.
* include/grecs/symtab.h: New file.
* include/grecs/table.h: New file.
* include/grecs/tree.h: New file.
* include/grecs/txtacc.h: New file.
* include/grecs/types.h.in: New file.
* include/grecs/util.h: New file.
* include/grecs/value.h: New file.
* include/grecs/version.h: New file.
* src/grecs-locus.h: Move to ...
* include/grecs/locus.h: ... here
* src/grecsopt.h: Move to ...
* include/grecs/opt.h: ... here
* src/json.h: Move to ...
* include/grecs/json.h: ... here
* src/wordsplit.h: Move to ...
* include/wordsplit.h: ... here
* src/Make-inst.am (include_HEADERS): Remove.
* src/Make-shared.am (grecsinclude_HEADERS)
(noinst_HEADERS): Remove.
* src/Make-static.am (noinst_HEADERS): Remove.
* src/Make.am: Remove header-related stuff
* src/bind-gram.y: Fix includes.
* src/bind-lex.l: Likewise.
* src/cidr.c: Likewise.
* src/dhcpd-gram.y: Likewise.
* src/dhcpd-lex.l: Likewise.
* src/diag.c: Likewise.
* src/format.c: Likewise.
* src/grecs-gram.y: Likewise.
* src/grecs-lex.l: Likewise.
* src/json-gram.y: Likewise.
* src/json-lex.l: Likewise.
* src/jsonfmt.c: Likewise.
* src/lookup.c: Likewise.
* src/meta1-gram.y: Likewise.
* src/meta1-lex.l: Likewise.
* src/opthelp.c: Likewise.
* src/preproc.c: Likewise.
* src/sockaddr.c: Likewise.
* src/txtacc.c: Likewise.
* src/version.c: Likewise.
* tests/gcfenum.c: Likewise.
* tests/gcffmt.c: Likewise.
* tests/gcfpeek.c: Likewise.
* tests/gcfset.c: Likewise.
* tests/gcfver.c: Likewise.
* tests/json.c: Likewise.
|
|
|
|
* src/dhcpd-gram.y: Support assignments and conditional
expressions.
* src/dhcpd-lex.l: New exclusive states BOOL and EXPR.
* src/parsertab.c (parser_tab) [ENABLE_DHCPD_PARSER]: Define
dhcpd parser.
* tests/parser-dhcpd.at: New file.
* tests/Makefile.am: Add parser-dhcpd.at
* tests/testsuite.at: Likewise.
|
|
* src/Make.am: Add jsonfmt.c
* src/json.h (json_format): New struct.
(json_format_value): New proto.
* src/jsonfmt.c: New file.
* tests/json.c: Use json_format_value.
|
|
* tests/wsp.c (make_env_kv): Fix array size
|
|
* src/Make-inst.am: Add json.h
* src/Make-shared.am: Likewise.
* src/Make-static.am: Likewise.
* src/Make.am: Likewise.
* src/json-gram.y: New file.
* src/json-lex.l: New file.
* src/json.h: New file.
* src/yytrans: Translate json prefixes.
* src/.gitignore: Update.
* tests/json.c: New file.
* tests/json00.at: New file.
* tests/json01.at: New file.
* tests/Makefile.am: Add new tests; build json
* tests/testsuite.at: Add new tests.
* tests/.gitignore: Update.
* am/grecs.m4: New flag "json"
* configure.ac (GRECS_SETUP): Require json
* src/Make-inst.am (include_HEADERS): Assign GRECS_HDR value.
* src/Make-shared.a [GRECS_COND_INSTALLHEADERS] (grecsinclude_HEADERS)
[!GRECS_COND_INSTALLHEADERS] (noinst_HEADERS): Likewise.
* src/Make-static.am (noinst_HEADERS): Likewise.
* src/Make.am [GRECS_COND_JSON]: Define GRECS_JSON and GRECS_EXTRA_JSON.
|
|
|
|
* src/wordsplit.c (_WSNF_DELIM): New node flag
(wsnode_flagstr): Print 'd' for that flag.
(expvar): Don't set WRDSF_WS when expanding variable
value. This fixes, e.g., expansion of a$x if x begins with
a whitespace.
(wsnode_nullelim): A _WSNF_DELIM node breaks join chain.
(wordsplit_trimws): Don't affect nodes within join chains.
(scan_word): Set _WSNF_DELIM for returned delimiters.
* tests/wordsplit.at (TESTWSP): 8th argument - prologue
shell code. Use it to unset environment variables in
env/getenv tests.
Test ws elimination with return_delims set.
|
|
|
|
|
|
* src/wordsplit.c: Implement default assignment, word
expansion in variable defaults, distinction between
${variable:-word} and ${variable-word}.
* doc/wordsplit.3: New file.
* src/wordsplit.h (wordsplit)<ws_envbuf,ws_envidx>
<ws_envsiz>: New members.
(WRDSF_ARGV): Remove.
(WRDSF_OPTIONS): New flag.
(WRDSO_ARGV): New option bit.
* tests/wordsplit.at: Add new tests.
* tests/wsp.c: Set WRDSF_OPTIONS flag if one of the options is requested.
|
|
|
|
* src/wordsplit.c (wordsplit_tildexpand)
(wordsplit_pathexpand): New functions.
(wordsplit_process_list): Run tilde and pathname expansion
if WRDSF_PATHEXPAND flag is set.
* src/wordsplit.h (wordsplit)<ws_options>: New member.
(WRDSF_PATHEXPAND): New flag.
(WRDSO_NULLGLOB,WRDSO_FAILGLOB)
(WRDSO_DOTGLOB): New defines.
* tests/wsp.c: New options pathexpand, nullglob, failglob,
dotglob. Fix help output.
|
|
* src/wordsplit.c (_wsplt_seterr): New static function. Use instead
of assigning value to ws_errno.
(coalesce_segment): Propagate _WSNF_QUOTE flag to the resulting node.
(wordsplit_process_list): Coalesce the list before running
command expansion.
* tests/testsuite.at: Test wordsplit first. If it fails, most other
tests will fail too.
* tests/wordsplit.at: Test command expansion.
|
|
New invocation sequence ensures proper error handling.
This is an incompatible change. Authors using ws_getvar member will
have to rewrite their ws_getvar function accordingly.
* src/wordsplit.c (wordsplit_init0): Call wordsplit_clearerr
on reuse.
(wordsplit_init): Fix ws_errno
(expvar): Change invocation of ws_getvar.
(expcmd): Change invocation of ws_command.
(wordsplit_clearerr): New function.
(wordsplit_strerror): Handle WRDSE_USERERR.
* src/wordsplit.h (ws_getvar): Change return value and signature of
ws_getvar and ws_command.
New member 'ws_usererr'.
(WRDSF_ARGV): New flag.
(WRDSE_OK): New define. Same as WRDSE_EOF.
(WRDSE_USERERR): New error code.
(wordsplit_clearerr): New proto.
* tests/wsp.c (wsp_getvar, wsp_runcmd): Rewrite.
|
|
* src/wordsplit.c (wordsplit_init): Change handling of lacking
WRDSF_NOCMD.
(find_closing_cbrace): Rename to find_closing_paren, take additional
argument. All uses changed.
(node_expand_vars): Rewrite as a generalized function node_expand.
(wordsplit_varexp): Use node_expand.
(expcmd, wordsplit_cmdexp): New functions.
(scan_word): Handle $(command) notation
(wordsplit_process_list): Command expansion.
* src/wordsplit.h (wordsplit) <ws_command>: New member.
* tests/wsp.c (wsp_runcmd): New function.
(main): Set ws_command unless WRDSF_NOCMD flag is set.
|
|
* tests/wsp.c (main): use additional variable to keep the
allocated value in incremental mode.
|
|
|
|
* tests/wsp.c (help): Update.
(wsp_getvar): New function.
(main): Callect all arguments in form X=Y to an array
and set ws_getvar member if this array is not empty.
* tests/wordsplit.at: Add tests for ws_getenv function.
|
|
* src/wordsplit.c (expvar): Ensure value is set.
Pass WRDSF_QUOTE to wordsplit
(scan_word): Treat ${...} as a single word.
* tests/wordsplit.at: Add more tests.
* tests/wsp.c: Fix error message.
|
|
* configure.ac (AC_CONFIG_SRCDIR): Fix argument
* src/Make-inst.am (include_HEADERS): Add grecsopt.h
* tests/.gitignore: Add wsp.c
* tests/Makefile.am: Add new tests.
* tests/testsuite.at: Likewise.
* tests/wordsplit.at: New testcase.
* tests/wsp.c: New test program.
|
|
* doc/grecs-syntax.texi: Document changes to the #include statement.
* doc/grecs_config.5: Likewise.
* src/preproc.c (include_glob, include_pos)
(include_once): New statics.
(pp_list_find): Return 0 if list is NULL.
(incl_copy): New statics.
(source_lookup): Use incl_copy as the copy function.
(pop_source): If more glob expansions are available, push next one
onto input stack.
(isglob): New static.
(parse_include): Support globbing patterns.
* tests/Makefile.am: Add new testes.
* tests/testsuite.at: Likewise.
* tests/incl00.at: New test case.
* tests/incl01.at: New test case.
* tests/incl02.at: New test case.
* tests/incl03.at: New test case.
|