This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: sre_parse group limit check missing with 'python -O'
Type: Stage:
Components: Regular Expressions Versions: Python 2.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: effbot Nosy List: Eugene.Voytitsky, effbot, kbriggs, rushing
Priority: normal Keywords:

Created on 2004-11-12 20:46 by rushing, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (4)
msg23114 - (view) Author: Sam Rushing (rushing) * Date: 2004-11-12 20:46
sre_parse.py includes a check to ensure that the
hard-coded limit of 100 groups is not exceeded.
Since the check uses an assert(), it's not present
when running in '-O' mode, so you can easily segfault
the re engine:

rushing@fang:~$ /usr/local/bin/python -O
Python 2.3.4 (#2, Oct 23 2004, 05:24:36) 
[GCC 3.4.2 [FreeBSD] 20040728] on freebsd5
>>> import re
>>> p = re.compile ('(' + '|'.join (['(xxx)']*200) + ')')
>>> p.search ('asdfasdfasdfasdfasdf')
Segmentation fault (core dumped)

The assert() should be changed to an 'if/raise'.

NOTE: I've sent changes to Frederik to remove the
limitation altogether - so if those get in before this
bug is addressed then please ignore.
msg23115 - (view) Author: Fredrik Lundh (effbot) * (Python committer) Date: 2004-11-13 07:02
Logged In: YES 
user_id=38376

This was changed in 2.4b1 (didn't I mail you about this?):

>>> p = re.compile ('(' + '|'.join (['(xxx)']*200) + ')')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/fredrik/NOBACKUP/python-
dev/python/dist/src/Lib/sre.py", line 180, in compile
    return _compile(pattern, flags)
  File "/home/fredrik/NOBACKUP/python-
dev/python/dist/src/Lib/sre.py", line 225, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/home/fredrik/NOBACKUP/python-
dev/python/dist/src/Lib/sre_compile.py", line 506, in compile
    raise AssertionError(
AssertionError: sorry, but this version only supports 100 
named groups

As for the real fix, it arrived too close to the beta release, 
and while it looks pretty solid, I'm not sure if it's a good idea 
to add it this close to a major release.  I'll bring this up on 
python-dev, when I find the time.
msg23116 - (view) Author: Keith Briggs (kbriggs) Date: 2004-12-21 16:23
Logged In: YES 
user_id=888261

There is a further problem here - the error message refers 
to "100 named groups", whereas the exception seems to be 
raised when there are too many groups, whether they are 
named or not.

What is the reason for this limit?  Can it easily be removed?
It is causing me many problems.
msg168680 - (view) Author: Eugene Voytitsky (Eugene.Voytitsky) Date: 2012-08-20 16:48
Hi all, does someone can answer the questions asked by Keith Briggs in 2004:

> What is the reason for this limit?  Can it easily be removed?
> It is causing me many problems.

I also stuck into the problem with this limitation.
Details you can read here - https://groups.google.com/forum/#!topic/ply-hack/iQLnZpTL1GA[1-25]

Thanks.
History
Date User Action Args
2022-04-11 14:56:08adminsetgithub: 41167
2012-08-20 16:48:17Eugene.Voytitskysetnosy: + Eugene.Voytitsky
messages: + msg168680
2004-11-12 20:46:23rushingcreate