Introduction
This document describes how to add regular expression search support to your roundup tracker. The syntax for the regular expressions is of course the same syntax as used in Python's RE module. For more information about the syntax, we refer to the documentation belonging to that module.
Requirements
The DatabaseWrapper is required if you want to add the regular expression search to your tracker.
Implementation
The regular expression feature consists out of a new search HTML template and a new action handler that supports regular expressions.
Search HTML Template
The new template is based on the issue.search.html template and it came
out of the roundup 1.1.1 distribution. To see the changes, please run a
diff against the new issue.search.html template and the out of the box
one from 1.1.1. Don't get scared. The changes are simple and not too much.
Action Handler
The action handler is also based on a standard out of the box action handler
of roundup 1.1.1. That action handler is SearchAction in actions.py.
The new handler supports all the features of the original handler as long
as the user didn't ask for a regular expression search.
To see the changes, please run a diff against the original action handler.
Best regards,
Marlon van den Berg
PS: Be aware that a regular expression search can consume some time when you run it on a slow server or in a tracker with a huge number of issues and messages.
Source Code
Here is the new issue.search.html template:
<tal:block metal:use-macro="templates/page/macros/icing">
<title metal:fill-slot="head_title" i18n:translate="">Issue searching - <span
i18n:name="tracker" tal:replace="config/TRACKER_NAME" /></title>
<span metal:fill-slot="body_title" tal:omit-tag="python:1"
i18n:translate="">Issue searching</span>
<td class="content" metal:fill-slot="content">
<disabled script language="javascript" type="text/javascript">
//<!--
function onClick_RegExp() {
if (self.document.itemSynopsis['reg_exp'].checked) {
self.document.itemSynopsis['re_ignorecase'].disabled = false;
} else {
self.document.itemSynopsis['re_ignorecase'].checked = false;
self.document.itemSynopsis['re_ignorecase'].disabled = true;
}
}
//-->
<disabled /script>
<form method="GET" name="itemSynopsis"
tal:attributes="action request/classname">
<table class="form" tal:define="
cols python:request.columns or 'id activity title status assignedto'.split();
sort_on python:request.sort[1] or 'activity';
group_on python:request.group[1] or 'priority';
search_input templates/page/macros/search_input;
column_input templates/page/macros/column_input;
sort_input templates/page/macros/sort_input;
group_input templates/page/macros/group_input;
search_select templates/page/macros/search_select;
search_multiselect templates/page/macros/search_multiselect;">
<tr tal:define="name string:@search_text">
<th i18n:translate="">All text*:</th>
<td>
<input size="45"
tal:attributes="value python:request.form.getvalue(name) or nothing;
name name">
</td>
<td nowrap colspan="3">
<input type="checkbox" name="reg_exp" value="1" onClick="javascript: onClick_RegExp()"
tal:attributes="checked reguest/form/reg_exp/value | nothing">
Regular Expression<br>
<input type="checkbox" name="re_ignorecase" value="1"
tal:attributes="checked reguest/form/re_ignorecase/value | nothing">
Ignore case<br>
<br>
</td>
</tr>
<tr>
<th class="header"> </th>
<th class="header" i18n:translate="">Filter on</th>
<th class="header" i18n:translate="">Display</th>
<th class="header" i18n:translate="">Sort on</th>
<th class="header" i18n:translate="">Group on</th>
</tr>
<tr tal:define="name string:title">
<th i18n:translate="">Title:</th>
<td metal:use-macro="search_input"></td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td> </td>
</tr>
<tr tal:define="name string:topic;
db_klass string:keyword;
db_content string:name;">
<th i18n:translate="">Topic:</th>
<td metal:use-macro="search_select"></td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:id">
<th i18n:translate="">ID:</th>
<td metal:use-macro="search_input"></td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td> </td>
</tr>
<tr tal:define="name string:creation">
<th i18n:translate="">Creation Date:</th>
<td metal:use-macro="search_input"></td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:creator;
db_klass string:user;
db_content string:username;"
tal:condition="db/user/is_view_ok">
<th i18n:translate="">Creator:</th>
<td metal:use-macro="search_select">
<option metal:fill-slot="extra_options" i18n:translate=""
tal:attributes="value request/user/id">created by me</option>
</td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:activity">
<th i18n:translate="">Activity:</th>
<td metal:use-macro="search_input"></td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td> </td>
</tr>
<tr tal:define="name string:actor;
db_klass string:user;
db_content string:username;"
tal:condition="db/user/is_view_ok">
<th i18n:translate="">Actor:</th>
<td metal:use-macro="search_select">
<option metal:fill-slot="extra_options" i18n:translate=""
tal:attributes="value request/user/id">done by me</option>
</td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td> </td>
</tr>
<tr tal:define="name string:priority;
db_klass string:priority;
db_content string:name;">
<th i18n:translate="">Priority:</th>
<td metal:use-macro="search_select">
<option metal:fill-slot="extra_options" value="-1" i18n:translate=""
tal:attributes="selected python:value == '-1'">not selected</option>
</td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:status;
db_klass string:status;
db_content string:name;">
<th i18n:translate="">Status:</th>
<td metal:use-macro="search_select">
<tal:block metal:fill-slot="extra_options">
<option value="-1,1,2,3,4,5,6,7" i18n:translate=""
tal:attributes="selected python:value == '-1,1,2,3,4,5,6,7'">not resolved</option>
<option value="-1" i18n:translate=""
tal:attributes="selected python:value == '-1'">not selected</option>
</tal:block>
</td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td metal:use-macro="group_input"></td>
</tr>
<tr tal:define="name string:assignedto;
db_klass string:user;
db_content string:username;"
tal:condition="db/user/is_view_ok">
<th i18n:translate="">Assigned to:</th>
<td metal:use-macro="search_select">
<tal:block metal:fill-slot="extra_options">
<option tal:attributes="value request/user/id"
i18n:translate="">assigned to me</option>
<option value="-1" tal:attributes="selected python:value == '-1'"
i18n:translate="">unassigned</option>
</tal:block>
</td>
<td metal:use-macro="column_input"></td>
<td metal:use-macro="sort_input"></td>
<td metal:use-macro="group_input"></td>
</tr>
<tr>
<th i18n:translate="">No Sort or group:</th>
<td> </td>
<td> </td>
<td><input type="radio" name="@sort" value=""></td>
<td><input type="radio" name="@group" value=""></td>
</tr>
<tr>
<th i18n:translate="">Pagesize:</th>
<td><input name="@pagesize" size="3" value="50"
tal:attributes="value request/form/@pagesize/value | default"></td>
</tr>
<tr>
<th i18n:translate="">Start With:</th>
<td><input name="@startwith" size="3" value="0"
tal:attributes="value request/form/@startwith/value | default"></td>
</tr>
<tr>
<th i18n:translate="">Sort Descending:</th>
<td><input type="checkbox" name="@sortdir"
tal:attributes="checked python:request.sort[0] == '-' or request.sort[0] is None">
</td>
</tr>
<tr>
<th i18n:translate="">Group Descending:</th>
<td><input type="checkbox" name="@groupdir"
tal:attributes="checked python:request.group[0] == '-'">
</td>
</tr>
<tr tal:condition="python:request.user.hasPermission('Edit', 'query')">
<th i18n:translate="">Query name**:</th>
<td tal:define="value request/form/@queryname/value | nothing">
<input name="@queryname" tal:attributes="value value">
<input type="hidden" name="@old-queryname" tal:attributes="value value">
</td>
</tr>
<tr>
<td>
<input type="hidden" name="@action" value="regexp_search">
</td>
<td><input type="submit" value="Search" i18n:attributes="value"></td>
</tr>
<tr><td> </td>
<td colspan="4" class="help" i18n:translate="">
*: The "all text" field will look in message bodies and issue titles<br>
<span tal:condition="python:request.user.hasPermission('Edit', 'query')">
**: If you supply a name, the query will be saved off and available as a
link in the sidebar
</span>
</td>
</tr>
</table>
</form>
<disabled script language="javascript" type="text/javascript">
//<!--
onClick_RegExp();
//-->
<disabled /script>
</td>
</tal:block>
<!-- SHA: d7999f394badc861c290fe332cb72634191352fc -->
And here is the new action handler source code:
import cgi, types
# templating import required for roundup 1.3.2, at a minimum
# from roundup.cgi import templating
from roundup.cgi.actions import SearchAction
from roundup.cgi.exceptions import Redirect
from roundup.extensions import dbwrapper
class RegExpSearchAction(SearchAction):
def handle(self):
"""\
Mangle some of the form variables.
Set the form ":filter" variable based on the values of the filter
variables - if they're set to anything other than "dontcare" then add
them to :filter.
Handle the ":queryname" variable and save off the query to the user's
query list.
Split any String query values on whitespace and comma.
"""
self.fakeFilterVars()
queryname = self.getQueryName()
# Prepare for regular expression
if self.form.has_key('reg_exp'):
self.form.value.remove(self.form['reg_exp'])
if queryname:
error_message = '''Sorry, regular expression searches can't be saved yet'''
self.client.error_message.append(error_message)
self.client.template = 'search'
return
elif not self.form.has_key('@search_text'):
error_message = '''Missing regular expression in 'All text' field'''
self.client.error_message.append(error_message)
self.client.template = 'search'
return
else:
reg_exp = self.form['@search_text'].value
if self.form.has_key('re_ignorecase'):
re_ignorecase = int(self.form['re_ignorecase'].value)
else:
re_ignorecase = 0
else:
reg_exp = None
# editing existing query name?
old_queryname = ''
for key in ('@old-queryname', ':old-queryname'):
if self.form.has_key(key):
old_queryname = self.form[key].value.strip()
# handle saving the query params
if queryname:
# parse the environment and figure what the query _is_
req = templating.HTMLRequest(self.client)
# The [1:] strips off the '?' character, it isn't part of the
# query string.
url = req.indexargs_url('', {})[1:]
key = self.db.query.getkey()
if key:
# edit the old way, only one query per name
try:
qid = self.db.query.lookup(old_queryname)
if not self.hasPermission('Edit', 'query', itemid=qid):
raise exceptions.Unauthorised, self._(
"You do not have permission to edit queries")
self.db.query.set(qid, klass=self.classname, url=url)
except KeyError:
# create a query
if not self.hasPermission('Create', 'query'):
raise exceptions.Unauthorised, self._(
"You do not have permission to store queries")
qid = self.db.query.create(name=queryname,
klass=self.classname, url=url)
else:
# edit the new way, query name not a key any more
# see if we match an existing private query
uid = self.db.getuid()
qids = self.db.query.filter(None, {'name': old_queryname,
'private_for': uid})
if not qids:
# ok, so there's not a private query for the current user
# - see if there's one created by them
qids = self.db.query.filter(None, {'name': old_queryname,
'creator': uid})
if qids:
# edit query - make sure we get an exact match on the name
for qid in qids:
if old_queryname != self.db.query.get(qid, 'name'):
continue
if not self.hasPermission('Edit', 'query', itemid=qid):
raise exceptions.Unauthorised, self._(
"You do not have permission to edit queries")
self.db.query.set(qid, klass=self.classname,
url=url, name=queryname)
else:
# create a query
if not self.hasPermission('Create', 'query'):
raise exceptions.Unauthorised, self._(
"You do not have permission to store queries")
qid = self.db.query.create(name=queryname,
klass=self.classname, url=url, private_for=uid)
# and add it to the user's query multilink
queries = self.db.user.get(self.userid, 'queries')
if qid not in queries:
queries.append(qid)
self.db.user.set(self.userid, queries=queries)
# commit the query change to the database
self.db.commit()
# check for regular expression search
if reg_exp and self.classname == 'issue':
# not needed anymore
if self.form.has_key('@search_text'):
self.form.value.remove(self.form['@search_text'])
# open dbwrapper
dbw = dbwrapper.DBWrapper(self.db)
if re_ignorecase:
flags = dbwrapper.IGNORECASE
else:
flags = 0
flags += dbwrapper.LOCALE
issues = [ issue.id for issue, res in dbw.issue.re.search(reg_exp, flags, 'title') ]
flags += (dbwrapper.MULTILINE + dbwrapper.DOTALL)
messages = dbw.msg.re.search(reg_exp, flags, 'content')
results = dbw.issue.find(messages=messages)
for issue in results:
if not issue.id in issues:
issues.append(issue.id)
if self.form.has_key('id'):
if isinstance(self.form['id'], types.ListType):
form_issues = []
for minifield in self.form['id']:
if minifield.value:
form_issues += re.split('[+, ]', minifield.value)
else:
form_issues = re.split('[+, ]', self.form['id'].value)
self.form.value.remove(self.form['id'])
found_issues = []
for issue_id in form_issues:
if int(issue_id) in issues:
found_issues.append(int(issue_id))
issues = found_issues
if issues:
issues.sort()
else:
issues.append(-1)
self.form.value.append(cgi.MiniFieldStorage('id', '+'.join( [str(nodeid) for nodeid in issues] )))
args = []
for column in self.form.keys():
if isinstance(self.form[column], types.ListType):
values = []
for minifield in self.form[column]:
if minifield.value:
values.append(minifield.value)
else:
values = [self.form[column].value]
args.append('%s=%s'%(column, ','.join(values)))
raise Redirect, '%s%s?&%s'%(self.base, self.classname, '&'.join(args))
def init(instance):
instance.registerAction('regexp_search', RegExpSearchAction)