Nabble - Gnu - Grep

[PATCH] fix multi-byte-locale read-beyond-end-of-buffer error

2009-12-23T12:05:52Z

FYI, here's a fix:

tests/fmbtest.sh provokes the failure, too,
so I'm not planning to add a test case.

From 60a7e46a1c3ff186c373e776958ea02437b686ef Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@...>
Date: Wed, 23 Dec 2009 18:59:30 +0100
Subject: [PATCH] fix multi-byte-locale read-beyond-end-of-buffer error

Avoid read-beyond-end-of-buffer errors, evoked by running this:
LC_ALL=en_US.UTF-8 valgrind src/grep -f <(printf 'a\nb\n') <(echo c)

Conditional jump or move depends on uninitialised value(s)
at 0x78136D: __gconv_transform_utf8_internal (in /lib/libc-2.11.so)
by 0x7E7232: mbrtowc (in /lib/libc-2.11.so)
by 0x8055773: dfaexec (dfa.c:2816)
by 0x804D7B0: EGexecute (search.c:353)
by 0x804ACD8: grepbuf (grep.c:1036)
by 0x804B023: grep (grep.c:1156)
by 0x804B460: grepfile (grep.c:1287)
by 0x804CF0D: main (grep.c:2282)

Conditional jump or move depends on uninitialised value(s)
at 0x7E7248: mbrtowc (in /lib/libc-2.11.so)
by 0x8055773: dfaexec (dfa.c:2816)
by 0x804D7B0: EGexecute (search.c:353)
by 0x804ACD8: grepbuf (grep.c:1036)
by 0x804B023: grep (grep.c:1156)
by 0x804B460: grepfile (grep.c:1287)
by 0x804CF0D: main (grep.c:2282)

* src/dfa.c (dfaexec) [MBS_SUPPORT]: Do not access one byte beyond
end of buffer.
---
src/dfa.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/dfa.c b/src/dfa.c
index d1d7f25..b50f98a 100644
--- a/src/dfa.c
+++ b/src/dfa.c
@@ -2808,13 +2808,13 @@ dfaexec (struct dfa *d, char const *begin, size_t size, int *backref)
MALLOC(inputwcs, wchar_t, end - (unsigned char const *)begin + 2);
memset(&mbs, 0, sizeof(mbstate_t));
remain_bytes = 0;
- for (i = 0; i < end - (unsigned char const *)begin + 1; i++)
+ for (i = 0; i < end - (unsigned char const *)begin; i++)
{
if (remain_bytes == 0)
{
remain_bytes
= mbrtowc(inputwcs + i, begin + i,
- end - (unsigned char const *)begin - i + 1, &mbs);
+ end - (unsigned char const *)begin - i, &mbs);
if (remain_bytes <= 1)
{
remain_bytes = 0;
--
1.6.6.rc4.284.gb0b11

[bug #23814] Result wrong in multibyte locale

2009-12-23T08:39:01Z

Follow-up Comment #1, bug #23814 (project grep):

Confirmed:

$ echo $'xa4xb3xa4xb3' | LANG=ja_JP.EUC-JP grep $'xa4xb3'
ここ
$ echo $'xb3xa4xa4xb3' | LANG=ja_JP.EUC-JP grep $'xa4xb3'
海こ
$ echo $'xb3xa4xb3xa4xa4xb3' | LANG=ja_JP.EUC-JP grep $'xa4xb3'
$

Cannot happen with UTF-8 because of how the encoding is done.

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?23814>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #23321] Epsclosure speedup patch

2009-12-23T08:32:12Z

Update of bug #23321 (project grep):

Status: None => Fixed
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #2:

Patch applied, thanks.

Thank you for the bug report. This bug has been fixed in the
development sources, which can be downloaded using git from
git://git.sv.gnu.org/grep.git

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?23321>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #23354] insert speedup patch

2009-12-23T08:31:36Z

Update of bug #23354 (project grep):

Status: None => Fixed
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #1:

Patch applied.

Thank you for the bug report. This bug has been fixed in the
development sources, which can be downloaded using git from
git://git.sv.gnu.org/grep.git

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?23354>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[PATCH 2/2] Speed up insert.

2009-12-23T08:30:35Z

Suggested by Johan Walles <johan.walles@...> (bug 23354).

* src/dfa.c (insert): Use binary search.
---
THANKS | 1 +
src/dfa.c | 29 ++++++++++++++++-------------
2 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/THANKS b/THANKS
index 3c89175..508846f 100644
--- a/THANKS
+++ b/THANKS
@@ -38,6 +38,7 @@ Jim Hand <jhand@...>
Jim Meyering <meyering@...>
Jochen Hein <jochen.hein@...>
Joel N. Weber II <devnull@...>
+Johan Walles <johan.walles@...>
John Hughes <john@...>
Jorge Stolfi <stolfi@...>
Juan Manuel Guerrero <ST001906@...>
diff --git a/src/dfa.c b/src/dfa.c
index 7d97f30..d1d7f25 100644
--- a/src/dfa.c
+++ b/src/dfa.c
@@ -1433,23 +1433,26 @@ copy (position_set const *src, position_set *dst)
static void
insert (position p, position_set *s)
{
- int i;
- position t1, t2;
+ int count = s->nelem;
+ int lo = 0, hi = count;
+ while (lo < hi)
+ {
+ int mid = ((unsigned) lo + (unsigned) hi) >> 1;
+ if (s->elems[mid].index < p.index)
+ lo = mid + 1;
+ else
+ hi = mid;
+ }

- for (i = 0; i < s->nelem && p.index < s->elems[i].index; ++i)
- continue;
- if (i < s->nelem && p.index == s->elems[i].index)
- s->elems[i].constraint |= p.constraint;
+ if (lo < count && p.index == s->elems[lo].index)
+ s->elems[lo].constraint |= p.constraint;
else
{
- t1 = p;
+ int i;
+ for (i = count; i > lo; i--)
+ s->elems[i] = s->elems[i - 1];
+ s->elems[lo] = p;
++s->nelem;
- while (i < s->nelem)
- {
- t2 = s->elems[i];
- s->elems[i++] = t1;
- t1 = t2;
- }
}
}

--
1.6.5.2

[PATCH 1/2] Decrease epsclosure memory usage

2009-12-23T08:30:34Z

From: Johan Walles <johan.walles@...>

Fixes bug 23321.

* src/dfa.c (epsclosure): Make visited an array of char.
---
src/dfa.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/dfa.c b/src/dfa.c
index 47ec093..7d97f30 100644
--- a/src/dfa.c
+++ b/src/dfa.c
@@ -1569,10 +1569,10 @@ static void
epsclosure (position_set *s, struct dfa const *d)
{
int i, j;
- int *visited;
+ char *visited;
position p, old;

- CALLOC(visited, int, d->tindex);
+ CALLOC(visited, char, d->tindex);

for (i = 0; i < s->nelem; ++i)
if (d->tokens[s->elems[i].index] >= NOTCHAR
--
1.6.5.2

[PATCH 0/2] Two patches from the savannah bugs

2009-12-23T08:30:33Z

Pretty safe, and they close two bugs. So now only the really nasty
things remain. :-) I redid the binary search patch because of copyright
problems.

Johan Walles (1):
Decrease epsclosure memory usage

Paolo Bonzini (1):
Speed up insert.

THANKS | 1 +
src/dfa.c | 33 ++++++++++++++++++---------------
2 files changed, 19 insertions(+), 15 deletions(-)

[bug #25063] grep --color clears line when CR matches (because of \33[K Erase Line string)

2009-12-23T02:27:19Z

Update of bug #25063 (project grep):

Status: None => Fixed
Open/Closed: Open => Closed
Summary: grep --color clears line when CR matches (because
of \33[K Erase Line string) => grep --color clears line when CR matches
(because of 33[K Erase Line string)

_______________________________________________________

Follow-up Comment #1:

You can set GREP_COLORS=ne in the environment to achieve the desired effect.

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?25063>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #24925] line enders are not consistent in output file

2009-12-23T02:22:55Z

Follow-up Comment #1, bug #24925 (project grep):

Confirmed, it only occurs when the output is redirected. See the call to
SET_BINARY (1) in src/grep.c

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?24925>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #23401] Add option to skip symlinks

2009-12-23T02:14:35Z

Update of bug #23401 (project grep):

Status: None => Duplicate
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #2:

Duplicate of bug 17623

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?23401>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #20961] hyphen problem: '^[0-9\-_\.]\+$' doesnot work while '^[0-9_\.-]\+$' does

2009-12-23T02:13:35Z

Update of bug #20961 (project grep):

Status: None => Invalid
Open/Closed: Open => Closed
Summary: hyphen problem: '^[0-9\-_\.]\+$' doesnot work while
'^[0-9_\.-]\+$' does => hyphen problem: '^[0-9-_.]+$' doesnot work while
'^[0-9_.-]+$' does

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?20961>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #22681] fix man page of grep

2009-12-22T23:57:17Z

Update of bug #22681 (project grep):

Status: None => Fixed
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #1:

Thank you for the bug report. This bug has been fixed in the
development sources, which can be downloaded using git from
git://git.sv.gnu.org/grep.git

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?22681>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #12128] grep -1 -2 is equivalent to grep -12

2009-12-22T23:57:05Z

Update of bug #12128 (project grep):

Status: None => Fixed
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #3:

Thank you for the bug report. This bug has been fixed in the
development sources, which can be downloaded using git from
git://git.sv.gnu.org/grep.git

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?12128>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #13161] missing check on memchr() return val in EGexecute

2009-12-22T23:56:52Z

Update of bug #13161 (project grep):

Status: None => Fixed
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #1:

Thank you for the bug report. This bug has been fixed in the
development sources, which can be downloaded using git from
git://git.sv.gnu.org/grep.git

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?13161>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #17623] add option to grep -R to NOT follow symlinks to directory

2009-12-22T23:55:26Z

Update of bug #17623 (project grep):

Severity: 3 - Normal => 2 - Minor
Summary: grep -R should NOT follow symlinks => add option to
grep -R to NOT follow symlinks to directory

_______________________________________________________

Follow-up Comment #4:

What about --[no-]dereference-symlink-to-dir (can be shortened to
--[no-]dereference since there's no ambiguous option)? cp's -H could become
--dereference-symlink-to-dir=cmdline (using an optional argument).

-S can be used as a short option.

Karl's proposal to have --directories=recurse-no-symlink is a bit ugly
because there is no quick way to specify it as a short option.

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?17623>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #17623] grep -R should NOT follow symlinks

2009-12-22T14:18:19Z

Follow-up Comment #3, bug #17623 (project grep):

Use the same option name as in coreutils?
--dereference-command-line-symlink-to-dir

And FWIW, personally I'd be happy if
--dereference-command-line-symlink-to-dir became the new default (even me, Mr.
Backward Compatibility Rules :). I suspect that's what users really
want/expect 99% of the time.

Having these things as option values for --directories sounds reasonable,
given that we are talking about following symlinks to dirs, not files
(right?).

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?17623>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #17623] grep -R should NOT follow symlinks

2009-12-22T12:31:02Z

Follow-up Comment #2, bug #17623 (project grep):

Hi Paolo,

I too have long wished for a way to make grep -R not follow symlinks.

My first thought was to handle it like find, cp, du, ls and chown etc.
have done, with their -H, -L and -P options, but all of those are already
in use with grep.

One approach would be to extend
--directories=...
or some new long-option to accept three new values,
corresponding to these (useful only with --recursive, which is
equivalent to --directory=recurse, or perhaps implying that):

H traverse-command-line-symlink-to-dir
L traverse-every-symlink-to-dir
P traverse-no-symlink-to-dir

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?17623>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[PATCH] Use concat-filename in savedir, fix part of bug #21276.

2009-12-22T02:50:23Z

* bootstrap.conf (gnulib_modules): Add concat-filename.
* lib/savedir.c (isdir1): Use concatenated_filename.
---
bootstrap.conf | 1 +
lib/savedir.c | 28 ++++++++--------------------
2 files changed, 9 insertions(+), 20 deletions(-)

diff --git a/bootstrap.conf b/bootstrap.conf
index 41d126e..1099fcf 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -24,6 +24,7 @@ atexit
binary-io
btowc
closeout
+concat-filename
dirent
dirname
do-release-commit-and-tag
diff --git a/lib/savedir.c b/lib/savedir.c
index c60e7c0..531c9a6 100644
--- a/lib/savedir.c
+++ b/lib/savedir.c
@@ -36,29 +36,23 @@
#include <string.h>
#include <fnmatch.h>
#include "savedir.h"
+#include "concat-filename.h"

extern int isdir (const char *path);

-char *path;
-size_t pathlen;
-
static int
isdir1 (const char *dir, const char *file)
{
int status;
- int slash;
- size_t dirlen = strlen (dir);
- size_t filelen = strlen (file);
- if ((dirlen + filelen + 2) > pathlen)
+ char *path;
+
+ status = 0;
+ path = concatenated_filename (dir, file, NULL);
+ if (path)
{
- path = calloc (dirlen + 1 + filelen + 1, sizeof (*path));
- pathlen = dirlen + filelen + 2;
+ status = isdir (path);
+ free (path);
}
- strcpy (path, dir);
- slash = (path[dirlen] != '/');
- path[dirlen] = '/';
- strcpy (path + dirlen + slash , file);
- status = isdir (path);
return status;
}

@@ -149,11 +143,5 @@ savedir (const char *dir, off_t name_size, struct exclude *included_patterns,
free (name_space);
return NULL;
}
- if (path)
- {
- free (path);
- path = NULL;
- pathlen = 0;
- }
return name_space;
}
--
1.6.5

[PATCH] Improve description of --label

2009-12-22T01:19:43Z

Fixes bug 22681.

* doc/grep.1 (--label): Use -H in the example, improve wording.
* doc/grep.texi (Output Line Prefix Control): Likewise.
---
doc/grep.1 | 4 ++--
doc/grep.texi | 8 ++++----
2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/doc/grep.1 b/doc/grep.1
index 950b82c..c0ea754 100644
--- a/doc/grep.1
+++ b/doc/grep.1
@@ -314,10 +314,10 @@ This is the default when there is only one file
.BI \-\^\-label= LABEL
Display input actually coming from standard input as input coming from file
.I LABEL.
-This is especially useful for tools like
+This is especially useful when implementing tools like
.BR zgrep ,
e.g.,
-.B "gzip -cd foo.gz | grep --label=foo something"
+.B "gzip -cd foo.gz | grep --label=foo -H something"
.TP
.BR \-n ", " \-\^\-line\-number
Prefix each line of output with the 1-based line number
diff --git a/doc/grep.texi b/doc/grep.texi
index 2714c58..a4a87f6 100644
--- a/doc/grep.texi
+++ b/doc/grep.texi
@@ -414,12 +414,12 @@ This is the default when there is only one file
@opindex --label
@cindex changing name of standard input
Display input actually coming from standard input
-as input coming from file @var{LABEL}.
-This is especially useful for tools like @command{zgrep};
-e.g.:
+as input coming from file @var{LABEL}. This is
+especially useful when implementing tools like
+@command{zgrep}; e.g.:

@example
-gzip -cd foo.gz | grep --label=foo something
+gzip -cd foo.gz | grep --label=foo -H something
@end example

@item -n
--
1.6.5.2

[bug #17454] Man page should describe parentheses

2009-12-22T01:16:22Z

Update of bug #17454 (project grep):

Status: None => Fixed
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #1:

Precedence
Repetition takes precedence over concatenation, which in turn
takes
precedence over alternation. A whole expression may be enclosed
in
parentheses to override these precedence rules and form
a
subexpression.

Back References and Subexpressions
The back-reference n, where n is a single digit, matches the
substring
previously matched by the nth parenthesized subexpression of
the
regular expression.

Thank you for the bug report. This bug has been fixed in CVS and
in grep-2.5.4. You may download the latest release from:

ftp://ftp.gnu.org/gnu/grep/

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?17454>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #25791] grep-2.5.4 source tarball signed with wrong/inaccessible public key

2009-12-22T01:13:57Z

Update of bug #25791 (project grep):

Status: None => Wont Fix
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #1:

Anyway too late to fix this.

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?25791>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #17623] grep -R should NOT follow symlinks

2009-12-22T01:12:33Z

Follow-up Comment #1, bug #17623 (project grep):

> grep -R follows symbolic links, unlike cp or ls.
> At least, one option should be provided to disable this behaviour.

Jim, do you have an opinion on this? I'm inclined to mark it as invalid or
at least as an enhancement, but you're the expert on these things. :-)

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?17623>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #18666] 'grep --color -i A' does not colour output in non-UTF8 locale

2009-12-22T01:11:30Z

Update of bug #18666 (project grep):

Status: None => Duplicate
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #2:

Dup of bug 12995.

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?18666>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #12995] grep with --color and -i causes color match not to be displayed

2009-12-22T01:11:01Z

Follow-up Comment #7, bug #12995 (project grep):

More testcases from bug 18666:

The following commands all colour the a's and A's in their output:

LC_ALL=en_GB.utf8 grep --color -i a file
LC_ALL=en_GB.utf8 grep --color -i A file
LC_ALL=en_GB grep --color -i a file
LC_ALL=C grep --color -i a file

But these two commands do not:

LC_ALL=en_GB grep --color -i A file
LC_ALL=C grep --color -i A file

The combination of option -i with an uppercase letter in the search pattern
and a non-UTF8 locale makes colourisation fail.

(That is: with grep from CVS. With grep-2.5.1 it works fine.)

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?12995>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #17492] BRE with caret in the middle of pattern doesn't match

2009-12-22T01:08:21Z

Update of bug #17492 (project grep):

Status: None => Fixed
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #2:

Thank you for the bug report. This bug has been fixed in CVS and
in grep-2.5.4. You may download the latest release from:

ftp://ftp.gnu.org/gnu/grep/

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?17492>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[PATCH] Avoid using an invalid memchr result.

2009-12-22T01:01:16Z

Related to bug 13161. I cannot find a testcase, but it is better to be
defensive considering that these bug were found in the past.

This version actually compiles. :-)

* src/search.c (EGexecute, Fexecute): Check for memchr return values.
---
gnulib | 2 +-
src/search.c | 18 ++++++++++++------
2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/gnulib b/gnulib
index c5588be..60b0c35 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit c5588be343f580be8e87d99e043dcdf3d7606759
+Subproject commit 60b0c353fb7bc5f8dd35e65df983793251e9efed
diff --git a/src/search.c b/src/search.c
index e4698ff..571b580 100644
--- a/src/search.c
+++ b/src/search.c
@@ -334,8 +334,10 @@ EXECUTE_FCT(EGexecute)
beg += offset;
/* Narrow down to the line containing the candidate, and
run it through DFA. */
- end = memchr(beg, eol, buflim - beg);
- end++;
+ if ((end = memchr(beg, eol, buflim - beg)) != NULL)
+ end++;
+ else
+ end = buflim;
#ifdef MBS_SUPPORT
if (MB_CUR_MAX > 1 && mb_properties[beg - buf] == 0)
continue;
@@ -355,8 +357,10 @@ EXECUTE_FCT(EGexecute)
break;
/* Narrow down to the line we've found. */
beg += offset;
- end = memchr (beg, eol, buflim - beg);
- end++;
+ if ((end = memchr(beg, eol, buflim - beg)) != NULL)
+ end++;
+ else
+ end = buflim;
while (beg > buf && beg[-1] != eol)
--beg;
}
@@ -594,8 +598,10 @@ EXECUTE_FCT(Fexecute)
goto out;

success:
- end = memchr (beg + len, eol, (buf + size) - (beg + len));
- end++;
+ if ((end = memchr (beg + len, eol, (buf + size) - (beg + len))) != NULL)
+ end++;
+ else
+ end = buf + size;
while (buf < beg && beg[-1] != eol)
--beg;
len = end - beg;
--
1.6.5.2

[PATCH] Make 'grep -1 -2' and 'grep -1v2' equivalent to grep -2

2009-12-22T00:59:26Z

Fixes bug 12128.

* src/grep.c (get_nondigit_option): Reset the buffer every time
a non-digit option is found or a new argument is started.
---
src/grep.c | 26 +++++++++++++++++++-------
tests/yesno.sh | 3 +++
2 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/src/grep.c b/src/grep.c
index 6539d4c..eabd304 100644
--- a/src/grep.c
+++ b/src/grep.c
@@ -1637,19 +1637,27 @@ prepend_default_options (char const *options, int *pargc, char ***pargv)
static int
get_nondigit_option (int argc, char *const *argv, int *default_context)
{
- int opt;
+ static int prev_digit_optind = -1;
+ int opt, this_digit_optind, was_digit;
char buf[sizeof (uintmax_t) * CHAR_BIT + 4];
char *p = buf;

- /* Set buf[0] to anything but '0', for the leading-zero test below. */
- buf[0] = '\0';
-
+ was_digit = 0;
+ this_digit_optind = optind;
while (opt = getopt_long (argc, argv, short_options, long_options, NULL),
'0' <= opt && opt <= '9')
{
- /* Suppress trivial leading zeros, to avoid incorrect
- diagnostic on strings like 00000000000. */
- p -= buf[0] == '0';
+ if (prev_digit_optind != this_digit_optind || !was_digit)
+ {
+ /* Reset to start another context length argument. */
+ p = buf;
+ }
+ else
+ {
+ /* Suppress trivial leading zeros, to avoid incorrect
+ diagnostic on strings like 00000000000. */
+ p -= buf[0] == '0';
+ }

if (p == buf + sizeof buf - 4)
{
@@ -1661,6 +1669,10 @@ get_nondigit_option (int argc, char *const *argv, int *default_context)
break;
}
*p++ = opt;
+
+ was_digit = 1;
+ prev_digit_optind = this_digit_optind;
+ this_digit_optind = optind;
}
if (p != buf)
{
diff --git a/tests/yesno.sh b/tests/yesno.sh
index 4852a33..2e21b5f 100755
--- a/tests/yesno.sh
+++ b/tests/yesno.sh
@@ -64,6 +64,7 @@ set x \
'-o' "$c$d$e$h$i$m$n$z0" \
'-C,1' "$rB$C$D$E$rF$rG$H$I$rJ$s$rL$M$N$z0" \
'-C,1,-o' "$c$d$e$h$i$s$m$n$z0" \
+ '-C,4,-1' "$rB$C$D$E$rF$rG$H$I$rJ$s$rL$M$N$z0" \
'-m,4' "$C$D$E$H$z0$XI$XJ$XK$XL$XM$XN" \
'-m,4,-o' "$c$d$e$h$z0$XI$XJ$XK$XL$XM$XN" \
'-m,4,-C,1' "$rB$C$D$E$rF$rG$H$z0$XI$XJ$XK$XL$XM$XN" \
@@ -79,6 +80,8 @@ set x \
'-v,-o' "$z0" \
'-v,-C,1' "$A$B$rC$s$rE$F$G$rH$rI$J$K$L$rM$z0" \
'-v,-C,1,-o' "$rc$s$re$rh$ri$rm$z0" \
+ '-4,-1' "$rB$C$D$E$rF$rG$H$I$rJ$s$rL$M$N$z0" \
+ '-4,-v,-1' "$A$B$rC$s$rE$F$G$rH$rI$J$K$L$rM$z0" \
'-m,1,-v' "$A$z0$XB$XC$XD$XE$XF$XG$XH$XI$XJ$XK$XL$XM$XN" \
'-m,1,-v,-o' "$z0$XB$XC$XD$XE$XF$XG$XH$XI$XJ$XK$XL$XM$XN" \
'-m,1,-v,-C,1' "$A$z0$XB$XC$XD$XE$XF$XG$XH$XI$XJ$XK$XL$XM$XN" \
--
1.6.5.2

[PATCH] Avoid using an invalid memchr result.

2009-12-22T00:49:28Z

Related to bug 13161. I cannot find a testcase, but it is better to be
defensive considering that these bug were found in the past.

* src/search.c (EGexecute, Fexecute): Check for memchr return values.
---
gnulib | 2 +-
src/search.c | 17 ++++++++++++-----
2 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/gnulib b/gnulib
index c5588be..60b0c35 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit c5588be343f580be8e87d99e043dcdf3d7606759
+Subproject commit 60b0c353fb7bc5f8dd35e65df983793251e9efed
diff --git a/src/search.c b/src/search.c
index e4698ff..ccc83ae 100644
--- a/src/search.c
+++ b/src/search.c
@@ -334,8 +334,10 @@ EXECUTE_FCT(EGexecute)
beg += offset;
/* Narrow down to the line containing the candidate, and
run it through DFA. */
- end = memchr(beg, eol, buflim - beg);
- end++;
+ if ((end = memchr(beg, eol, buflim - beg)) != NULL)
+ end++;
+ else
+ end = buflim;
#ifdef MBS_SUPPORT
if (MB_CUR_MAX > 1 && mb_properties[beg - buf] == 0)
continue;
@@ -355,8 +357,10 @@ EXECUTE_FCT(EGexecute)
break;
/* Narrow down to the line we've found. */
beg += offset;
- end = memchr (beg, eol, buflim - beg);
- end++;
+ if ((end = memchr(beg, eol, buflim - beg)) != NULL)
+ end++;
+ else
+ end = buflim;
while (beg > buf && beg[-1] != eol)
--beg;
}
@@ -595,7 +599,10 @@ EXECUTE_FCT(Fexecute)

success:
end = memchr (beg + len, eol, (buf + size) - (beg + len));
- end++;
+ if ((end = memchr(beg, eol, buflim - beg)) != NULL)
+ end++;
+ else
+ end = buflim;
while (buf < beg && beg[-1] != eol)
--beg;
len = end - beg;
--
1.6.5.2

[bug #28275] grep -P should use PCRE_UTF8

2009-12-22T00:46:11Z

Update of bug #28275 (project grep):

Status: Invalid => Confirmed
Open/Closed: Closed => Open
Summary: Ranges like [a-z] incorrectly match in UTF systems
=> grep -P should use PCRE_UTF8

_______________________________________________________

Follow-up Comment #8:

That would require conversion to UTF-8 before searching (for non-single-byte,
non-UTF-8 character sets) but it is a good idea.

Yet another thing to rewrite. :-)

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?28275>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #28275] Ranges like [a-z] incorrectly match in UTF systems

2009-12-21T06:49:29Z

Follow-up Comment #7, bug #28275 (project grep):

> Regarding Perl mode, that's unfortunately a bug in PCRE, not grep.

pcregrep that comes with PCRE works fine if the -u switch is specified.
Philip Hazel suggested calling pcre_compile() with PCRE_UTF8 flag set, when
UTF-8 is involved. I tried patching grep, but ran into some Gentoo-specific
difficulties.

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?28275>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

enable a few more syntax-checks

2009-12-20T07:32:00Z

From dc57dad0fd9ccc43e65491e7416652d1437dff30 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@...>
Date: Fri, 4 Dec 2009 23:19:07 +0100
Subject: [PATCH 1/4] maint: enable cast_of_alloca_return_value check

* cfg.mk (local-checks-to-skip): Enable sc_cast_of_alloca_return_value.
* .x-sc_cast_of_alloca_return_value: New file.
---
.x-sc_cast_of_alloca_return_value | 1 +
cfg.mk | 1 -
2 files changed, 1 insertions(+), 1 deletions(-)
create mode 100644 .x-sc_cast_of_alloca_return_value

diff --git a/.x-sc_cast_of_alloca_return_value b/.x-sc_cast_of_alloca_return_value
new file mode 100644
index 0000000..748e1b0
--- /dev/null
+++ b/.x-sc_cast_of_alloca_return_value
@@ -0,0 +1 @@
+lib/regex.c
diff --git a/cfg.mk b/cfg.mk
index 7233197..92aee2f 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -22,7 +22,6 @@ gpg_key_ID = B9AB9A16

# Tests not to run as part of "make distcheck".
local-checks-to-skip = \
- sc_cast_of_alloca_return_value \
sc_cast_of_argument_to_free \
sc_cast_of_x_alloc_return_value \
sc_error_message_uppercase \
--
1.6.6.rc3.271.g3d40f

From acd6c7be4dab1cc5d4012f1074691999d4fd3083 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@...>
Date: Fri, 4 Dec 2009 23:31:34 +0100
Subject: [PATCH 2/4] maint: enable cast_of_x_alloc_return_value check

* cfg.mk (local-checks-to-skip): Enable sc_cast_of_x_alloc_return_value.
* .x-sc_cast_of_x_alloc_return_value:
* src/dfa.c (CALLOC, MALLOC, REALLOC): Remove casts.
* src/dosbuf.c (undossify_input): Likewise.
* src/grep.c (print_line_middle, prepend_default_options): Likewise.
---
.x-sc_cast_of_x_alloc_return_value | 1 +
cfg.mk | 1 -
src/dfa.c | 6 +++---
src/dosbuf.c | 7 +++----
src/grep.c | 4 ++--
5 files changed, 9 insertions(+), 10 deletions(-)
create mode 100644 .x-sc_cast_of_x_alloc_return_value

diff --git a/.x-sc_cast_of_x_alloc_return_value b/.x-sc_cast_of_x_alloc_return_value
new file mode 100644
index 0000000..748e1b0
--- /dev/null
+++ b/.x-sc_cast_of_x_alloc_return_value
@@ -0,0 +1 @@
+lib/regex.c
diff --git a/cfg.mk b/cfg.mk
index 92aee2f..b222ff3 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -23,7 +23,6 @@ gpg_key_ID = B9AB9A16
# Tests not to run as part of "make distcheck".
local-checks-to-skip = \
sc_cast_of_argument_to_free \
- sc_cast_of_x_alloc_return_value \
sc_error_message_uppercase \
sc_m4_quote_check \
sc_makefile_TAB_only_indentation \
diff --git a/src/dfa.c b/src/dfa.c
index 6917ceb..47ec093 100644
--- a/src/dfa.c
+++ b/src/dfa.c
@@ -173,9 +173,9 @@ xrealloc (ptr_t p, size_t n)
return r;
}

-#define CALLOC(p, t, n) ((p) = (t *) xcalloc((size_t)(n), sizeof (t)))
-#define MALLOC(p, t, n) ((p) = (t *) xmalloc((n) * sizeof (t)))
-#define REALLOC(p, t, n) ((p) = (t *) xrealloc((ptr_t) (p), (n) * sizeof (t)))
+#define CALLOC(p, t, n) ((p) = xcalloc((size_t)(n), sizeof (t)))
+#define MALLOC(p, t, n) ((p) = xmalloc((n) * sizeof (t)))
+#define REALLOC(p, t, n) ((p) = xrealloc((ptr_t) (p), (n) * sizeof (t)))

/* Reallocate an array of type t if nalloc is too small for index. */
#define REALLOC_IF_NECESSARY(p, t, nalloc, index) \
diff --git a/src/dosbuf.c b/src/dosbuf.c
index be64952..664bb90 100644
--- a/src/dosbuf.c
+++ b/src/dosbuf.c
@@ -125,10 +125,9 @@ undossify_input (register char *buf, size_t buflen)
if (inp_map_idx >= dos_pos_map_size - 1)
{
dos_pos_map_size = inp_map_idx ? inp_map_idx * 2 : 1000;
- dos_pos_map =
- (struct dos_map *)xrealloc((char *)dos_pos_map,
- dos_pos_map_size *
- sizeof(struct dos_map));
+ dos_pos_map = xrealloc((char *)dos_pos_map,
+ dos_pos_map_size *
+ sizeof(struct dos_map));
}

if (!inp_map_idx)
diff --git a/src/grep.c b/src/grep.c
index 6539d4c..564166e 100644
--- a/src/grep.c
+++ b/src/grep.c
@@ -796,7 +796,7 @@ print_line_middle (const char *beg, const char *lim,
{
int i = lim - beg;

- ibeg = buf = (char *) xmalloc(i);
+ ibeg = buf = xmalloc(i);
/* This can't possibly be correct with UTF-8,
but it's equivalent to what was there so far. */
while (--i >= 0)
@@ -1620,7 +1620,7 @@ prepend_default_options (char const *options, int *pargc, char ***pargv)
int prepended = prepend_args (options, buf, (char **) NULL);
int argc = *pargc;
char * const *argv = *pargv;
- char **pp = (char **) xmalloc ((prepended + argc + 1) * sizeof *pp);
+ char **pp = xmalloc ((prepended + argc + 1) * sizeof *pp);
*pargc = prepended + argc;
*pargv = pp;
*pp++ = *argv++;
--
1.6.6.rc3.271.g3d40f

From ecb0e55a4a278752032175ad8c628ec20388734a Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@...>
Date: Fri, 4 Dec 2009 23:35:23 +0100
Subject: [PATCH 3/4] maint: enable prohibit_have_config_h check

* cfg.mk (local-checks-to-skip): Enable sc_prohibit_have_config_h
* lib/regex.c: Remove useless cpp test of HAVE_CONFIG_H.
* lib/savedir.c: Likewise.
* src/grep.c: Likewise.
* src/kwset.c: Likewise.
* src/search.c: Likewise.
---
cfg.mk | 1 -
lib/regex.c | 4 +---
lib/savedir.c | 4 +---
src/grep.c | 4 +---
src/kwset.c | 4 +---
src/search.c | 4 +---
6 files changed, 5 insertions(+), 16 deletions(-)

diff --git a/cfg.mk b/cfg.mk
index b222ff3..364c4fe 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -26,7 +26,6 @@ local-checks-to-skip = \
sc_error_message_uppercase \
sc_m4_quote_check \
sc_makefile_TAB_only_indentation \
- sc_prohibit_have_config_h \
sc_prohibit_magic_number_exit \
sc_prohibit_strcmp \
sc_space_tab \
diff --git a/lib/regex.c b/lib/regex.c
index 6ad5c53..6ec4cb0 100644
--- a/lib/regex.c
+++ b/lib/regex.c
@@ -27,9 +27,7 @@
#undef _GNU_SOURCE
#define _GNU_SOURCE

-#ifdef HAVE_CONFIG_H
-# include <config.h>
-#endif
+#include <config.h>

#ifndef PARAMS
# if defined __GNUC__ || (defined __STDC__ && __STDC__)
diff --git a/lib/savedir.c b/lib/savedir.c
index e2c015a..c60e7c0 100644
--- a/lib/savedir.c
+++ b/lib/savedir.c
@@ -18,9 +18,7 @@

/* Written by David MacKenzie <djm@...>. */

-#if HAVE_CONFIG_H
-# include <config.h>
-#endif
+#include <config.h>

#include <sys/types.h>
#include <unistd.h>
diff --git a/src/grep.c b/src/grep.c
index 564166e..c16c0fb 100644
--- a/src/grep.c
+++ b/src/grep.c
@@ -19,9 +19,7 @@

/* Written July 1992 by Mike Haertel. */

-#ifdef HAVE_CONFIG_H
-# include <config.h>
-#endif
+#include <config.h>
#include <sys/types.h>
#include <sys/stat.h>
#if defined(HAVE_MMAP)
diff --git a/src/kwset.c b/src/kwset.c
index 2d90ca5..40b6a6f 100644
--- a/src/kwset.c
+++ b/src/kwset.c
@@ -29,9 +29,7 @@
String Matching: An Aid to Bibliographic Search," CACM June 1975,
Vol. 18, No. 6, which describes the failure function used below. */

-#ifdef HAVE_CONFIG_H
-# include <config.h>
-#endif
+#include <config.h>
#include <sys/types.h>
#include "system.h"
#include "kwset.h"
diff --git a/src/search.c b/src/search.c
index e4698ff..15516b1 100644
--- a/src/search.c
+++ b/src/search.c
@@ -18,9 +18,7 @@

/* Written August 1992 by Mike Haertel. */

-#ifdef HAVE_CONFIG_H
-# include <config.h>
-#endif
+#include <config.h>

#include <sys/types.h>

--
1.6.6.rc3.271.g3d40f

From d038616cd1dfced11015b8a8a2f52e61f513de7e Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@...>
Date: Fri, 11 Dec 2009 21:45:36 +0100
Subject: [PATCH 4/4] build: update gnulib submodule to latest

---
build-aux/.gitignore | 1 +
gnulib | 2 +-
2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/build-aux/.gitignore b/build-aux/.gitignore
index 016b26a..5734ae4 100644
--- a/build-aux/.gitignore
+++ b/build-aux/.gitignore
@@ -1,4 +1,5 @@
announce-gen
+arg-nonnull.h
config.guess
config.rpath
config.sub
diff --git a/gnulib b/gnulib
index c5588be..a3255b1 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit c5588be343f580be8e87d99e043dcdf3d7606759
+Subproject commit a3255b1b9848874572e0df9d0609e263da50de4b
--
1.6.6.rc3.271.g3d40f

Re: [bug #28329] egrep/fgrep should accept respectively -E/-F

2009-12-19T01:58:43Z

>> I think setting -E in GREP_OPTIONS is an incredibly bad idea
>> :-) as it will break scripts.
>
> Since extended REs are almost completely backward compatible with simple REs,
> I think it is unlikely that this will break many scripts. I haven't found one
> yet where this is a problem.

Any script that greps for + or { or } or ? for example.

>> use "alias grep='grep -E'" instead
>
> I tried this for a while. Then I would spend time figuring out a RE at the
> command prompt only to have it break when I included it in a script.

So now it breaks when you give your scripts to others... (of course
you could add GREP_OPTIONS=-E to those scripts too, that would be a
good idea, but so you can also add an alias there too I think).

Paolo

[bug #28329] egrep/fgrep should accept respectively -E/-F

2009-12-18T20:00:26Z

Follow-up Comment #3, bug #28329 (project grep):

> I think setting -E in GREP_OPTIONS is an incredibly bad idea
> :-) as it will break scripts.

Since extended REs are almost completely backward compatible with simple REs,
I think it is unlikely that this will break many scripts. I haven't found one
yet where this is a problem.

> use "alias grep='grep -E'" instead

I tried this for a while. Then I would spend time figuring out a RE at the
command prompt only to have it break when I included it in a script. After
many frustrating debugging sessions, I set the variable to save my sanity. I'm
trying to get into the habit of always using egrep but until that "takes",
this is my fallback.

I don't have a problem with programs ignoring incompatible options in cases
like this (a warning would be OK) but I'm sure purists do. Years of experience
tends to make one a little less dogmatic about things like this. ;-)

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?28329>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #28329] egrep/fgrep should accept respectively -E/-F

2009-12-18T15:21:50Z

Follow-up Comment #2, bug #28329 (project grep):

Irrelevant to the ticket, but FWIW, I use GREP_OPTIONS to do excludes:
--directories=skip --exclude-dir=.svn --exclude=*~

The only problem I've had is with older greps that don't have the options.
So it goes.

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?28329>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[bug #28329] egrep/fgrep should accept respectively -E/-F

2009-12-18T03:51:46Z

Update of bug #28329 (project grep):

Severity: 3 - Normal => 2 - Minor
Summary: egrep error when GREP_OPTIONS=-E => egrep/fgrep
should accept respectively -E/-F

_______________________________________________________

Follow-up Comment #1:

I think setting -E in GREP_OPTIONS is an incredibly bad idea :-) as it will
break scripts. In fact, there are very few options suitable for GREP_OPTIONS:
--line-buffered, --binary-files (and synonyms), --devices, --mmap, --binary,
and probably nothing else.

If you want to save a letter on every grep invocation, use "alias grep='grep
-E'" instead so that scripts do not have automatic -E. This would fix the
issue, since egrep would not see the -E instead.

That said, adding support for -E in egrep and -F in fgrep does have some
merit, so I'm changing the summary and leaving the bug open.

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/bugs/?28329>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/