Mercurial > kallithea
annotate rhodecode/controllers/search.py @ 936:e1c9903d7e38 beta
fixed wrong clone url for anonymous access
author | Marcin Kuzminski <marcin@python-works.com> |
---|---|
date | Mon, 10 Jan 2011 20:37:46 +0100 |
parents | 07a6e8c65526 |
children | 3fc9183e05dd |
rev | line source |
---|---|
861
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
1 # -*- coding: utf-8 -*- |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
2 """ |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
3 rhodecode.controllers.search |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
4 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
5 |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
6 Search controller for rhodecode |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
7 |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
8 :created_on: Aug 7, 2010 |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
9 :author: marcink |
902
07a6e8c65526
fixed copyright year to 2011
Marcin Kuzminski <marcin@python-works.com>
parents:
861
diff
changeset
|
10 :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com> |
861
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
11 :license: GPLv3, see COPYING for more details. |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
12 """ |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
13 # This program is free software; you can redistribute it and/or |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
14 # modify it under the terms of the GNU General Public License |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
15 # as published by the Free Software Foundation; version 2 |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
16 # of the License or (at your opinion) any later version of the license. |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
17 # |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
18 # This program is distributed in the hope that it will be useful, |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
19 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
20 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
21 # GNU General Public License for more details. |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
22 # |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
23 # You should have received a copy of the GNU General Public License |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
24 # along with this program; if not, write to the Free Software |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
25 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
26 # MA 02110-1301, USA. |
861
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
27 import logging |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
28 import traceback |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
29 |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
30 from pylons.i18n.translation import _ |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
31 from pylons import request, response, config, session, tmpl_context as c, url |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
32 from pylons.controllers.util import abort, redirect |
861
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
33 |
547
1e757ac98988
renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
525
diff
changeset
|
34 from rhodecode.lib.auth import LoginRequired |
1e757ac98988
renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
525
diff
changeset
|
35 from rhodecode.lib.base import BaseController, render |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
36 from rhodecode.lib.indexers import SCHEMA, IDX_NAME, ResultWrapper |
861
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
37 |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
38 from webhelpers.paginate import Page |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
39 from webhelpers.util import update_params |
861
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
40 |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
41 from whoosh.index import open_dir, EmptyIndexError |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
42 from whoosh.qparser import QueryParser, QueryParserError |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
43 from whoosh.query import Phrase |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
44 |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
45 log = logging.getLogger(__name__) |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
46 |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
47 class SearchController(BaseController): |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
48 |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
49 @LoginRequired() |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
50 def __before__(self): |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
51 super(SearchController, self).__before__() |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
52 |
525
87d80c84df09
added search in specific repository
Marcin Kuzminski <marcin@python-works.com>
parents:
478
diff
changeset
|
53 def index(self, search_repo=None): |
87d80c84df09
added search in specific repository
Marcin Kuzminski <marcin@python-works.com>
parents:
478
diff
changeset
|
54 c.repo_name = search_repo |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
55 c.formated_results = [] |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
56 c.runtime = '' |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
57 c.cur_query = request.GET.get('q', None) |
556
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
58 c.cur_type = request.GET.get('type', 'source') |
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
59 c.cur_search = search_type = {'content':'content', |
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
60 'commit':'content', |
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
61 'path':'path', |
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
62 'repository':'repository'}\ |
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
63 .get(c.cur_type, 'content') |
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
64 |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
65 |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
66 if c.cur_query: |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
67 cur_query = c.cur_query.lower() |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
68 |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
69 if c.cur_query: |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
70 p = int(request.params.get('page', 1)) |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
71 highlight_items = set() |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
72 try: |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
73 idx = open_dir(config['app_conf']['index_dir'] |
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
74 , indexname=IDX_NAME) |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
75 searcher = idx.searcher() |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
76 |
556
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
77 qp = QueryParser(search_type, schema=SCHEMA) |
525
87d80c84df09
added search in specific repository
Marcin Kuzminski <marcin@python-works.com>
parents:
478
diff
changeset
|
78 if c.repo_name: |
87d80c84df09
added search in specific repository
Marcin Kuzminski <marcin@python-works.com>
parents:
478
diff
changeset
|
79 cur_query = u'repository:%s %s' % (c.repo_name, cur_query) |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
80 try: |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
81 query = qp.parse(unicode(cur_query)) |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
82 |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
83 if isinstance(query, Phrase): |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
84 highlight_items.update(query.words) |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
85 else: |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
86 for i in query.all_terms(): |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
87 if i[0] == 'content': |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
88 highlight_items.add(i[1]) |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
89 |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
90 matcher = query.matcher(searcher) |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
91 |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
92 log.debug(query) |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
93 log.debug(highlight_items) |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
94 results = searcher.search(query) |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
95 res_ln = len(results) |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
96 c.runtime = '%s results (%.3f seconds)' \ |
556
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
97 % (res_ln, results.runtime) |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
98 |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
99 def url_generator(**kw): |
556
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
100 return update_params("?q=%s&type=%s" \ |
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
101 % (c.cur_query, c.cur_search), **kw) |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
102 |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
103 c.formated_results = Page( |
556
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
104 ResultWrapper(search_type, searcher, matcher, |
65b2f150beb7
Added searching for file names within the repository in rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
105 highlight_items), |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
106 page=p, item_count=res_ln, |
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
107 items_per_page=10, url=url_generator) |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
108 |
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
109 |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
110 except QueryParserError: |
410
9a7ae16ff53e
fixes translations, style updates.
Marcin Kuzminski <marcin@python-works.com>
parents:
406
diff
changeset
|
111 c.runtime = _('Invalid search query. Try quoting it.') |
478
7010af6efde5
Reimplemented searching for speed on large files and added paging for search results
Marcin Kuzminski <marcin@python-works.com>
parents:
410
diff
changeset
|
112 searcher.close() |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
113 except (EmptyIndexError, IOError): |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
114 log.error(traceback.format_exc()) |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
115 log.error('Empty Index data') |
861
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
116 c.runtime = _('There is no index to search in. ' |
fd2ea6ceadc8
updated docs on every controller
Marcin Kuzminski <marcin@python-works.com>
parents:
688
diff
changeset
|
117 'Please run whoosh indexer') |
688
8acbfa837180
Tests rewrite for 1.2 added some globals configs to make tests easier.
Marcin Kuzminski <marcin@python-works.com>
parents:
556
diff
changeset
|
118 |
406
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
119 # Return a rendered template |
b153a51b1d3b
Implemented search using whoosh. Still as experimental option.
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
120 return render('/search/search.html') |