Mercurial > kallithea

annotate rhodecode/model/scm.py @ 712:131c1e335fa7 beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added ldap info on admin users, added bool2icon helper for nicer representation of True/False values. Fixed lock open icon to be more readable
author Marcin Kuzminski <marcin@python-works.com>
date Thu, 18 Nov 2010 03:05:29 +0100
parents e2f3c8e6939d
children 066af351c6d8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
58
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
1 #!/usr/bin/env python
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
2 # encoding: utf-8
569
000b675e7c1d bumped version, some spelling fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
3 # Model for RhodeCode
252
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
4 # Copyright (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>
373
3171614c0067 Added permissions check on repo switcher,
Marcin Kuzminski <marcin@python-works.com>
parents: 368
diff changeset
5 #
252
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
6 # This program is free software; you can redistribute it and/or
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
7 # modify it under the terms of the GNU General Public License
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
8 # as published by the Free Software Foundation; version 2
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
9 # of the License or (at your opinion) any later version of the license.
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
10 #
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
11 # This program is distributed in the hope that it will be useful,
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
14 # GNU General Public License for more details.
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
15 #
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
16 # You should have received a copy of the GNU General Public License
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
17 # along with this program; if not, write to the Free Software
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
18 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
19 # MA 02110-1301, USA.
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
20 """
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
21 Created on April 9, 2010
569
000b675e7c1d bumped version, some spelling fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
22 Model for RhodeCode
58
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
23 @author: marcink
252
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
24 """
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
25 from beaker.cache import cache_region, region_invalidate
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
26 from mercurial import ui
710
e2f3c8e6939d Disable git support due to large problems with dulwich.
Marcin Kuzminski <marcin@python-works.com>
parents: 693
diff changeset
27 from rhodecode import BACKENDS
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
28 from rhodecode.lib import helpers as h
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
29 from rhodecode.lib.auth import HasRepoPermissionAny
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
30 from rhodecode.lib.utils import get_repos
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
31 from rhodecode.model import meta
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
32 from rhodecode.model.db import Repository, User, RhodeCodeUi, CacheInvalidation
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
33 from rhodecode.model.caching_query import FromCache
535
72778dda34cf some fixups in cache, added fallback and cache invalidation when key not found in cached repos list,
Marcin Kuzminski <marcin@python-works.com>
parents: 531
diff changeset
34 from sqlalchemy.orm import joinedload
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
35 from sqlalchemy.orm.session import make_transient
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
36 from vcs import get_backend
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
37 from vcs.utils.helpers import get_scm
373
3171614c0067 Added permissions check on repo switcher,
Marcin Kuzminski <marcin@python-works.com>
parents: 368
diff changeset
38 from vcs.exceptions import RepositoryError, VCSError
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
39 from vcs.utils.lazy import LazyProperty
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
40 import traceback
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
41 import logging
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
42 import os
633
fcf599cd3404 Move changes for git implementation
Marcin Kuzminski <marcin@python-works.com>
parents: 631
diff changeset
43 import time
fcf599cd3404 Move changes for git implementation
Marcin Kuzminski <marcin@python-works.com>
parents: 631
diff changeset
44
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
45 log = logging.getLogger(__name__)
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
46
691
7486da5f0628 Refactor codes for scm model
Marcin Kuzminski <marcin@python-works.com>
parents: 690
diff changeset
47 class ScmModel(object):
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
48 """
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
49 Mercurial Model
58
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
50 """
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
51
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
52 def __init__(self):
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
53 self.sa = meta.Session()
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
54
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
55 @LazyProperty
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
56 def repos_path(self):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
57 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
58 Get's the repositories root path from database
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
59 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
60 q = self.sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
61
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
62 return q.ui_value
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
63
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
64 def repo_scan(self, repos_path, baseui, initial=False):
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
65 """
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
66 Listing of repositories in given path. This path should not be a
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
67 repository itself. Return a dictionary of repository objects
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
68
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
69 :param repos_path: path to directory containing repositories
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
70 :param baseui
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
71 :param initial: initial scan
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
72 """
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
73 log.info('scanning for repositories in %s', repos_path)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
74
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
75 if not isinstance(baseui, ui.ui):
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
76 baseui = ui.ui()
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
77 repos_list = {}
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
78
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
79 for name, path in get_repos(repos_path):
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
80 try:
248
fb7f066126cc Added support for repository located in subdirectories.
Marcin Kuzminski <marcin@python-works.com>
parents: 245
diff changeset
81 if repos_list.has_key(name):
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
82 raise RepositoryError('Duplicate repository name %s '
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
83 'found in %s' % (name, path))
248
fb7f066126cc Added support for repository located in subdirectories.
Marcin Kuzminski <marcin@python-works.com>
parents: 245
diff changeset
84 else:
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
85
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
86 klass = get_backend(path[0])
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
87
710
e2f3c8e6939d Disable git support due to large problems with dulwich.
Marcin Kuzminski <marcin@python-works.com>
parents: 693
diff changeset
88 if path[0] == 'hg' and path[0] in BACKENDS.keys():
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
89 repos_list[name] = klass(path[1], baseui=baseui)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
90
710
e2f3c8e6939d Disable git support due to large problems with dulwich.
Marcin Kuzminski <marcin@python-works.com>
parents: 693
diff changeset
91 if path[0] == 'git' and path[0] in BACKENDS.keys():
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
92 repos_list[name] = klass(path[1])
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
93 except OSError:
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
94 continue
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
95
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
96 return repos_list
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
97
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
98 def get_repos(self, all_repos=None):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
99 """
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
100 Get all repos from db and for each repo create it's backend instance.
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
101 and fill that backed with information from database
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
102
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
103 :param all_repos: give specific repositories list, good for filtering
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
104 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
105 if not all_repos:
693
90ac3255c117 fixed sorting for repo switcher, fixed length for code stats
Marcin Kuzminski <marcin@python-works.com>
parents: 692
diff changeset
106 all_repos = self.sa.query(Repository)\
90ac3255c117 fixed sorting for repo switcher, fixed length for code stats
Marcin Kuzminski <marcin@python-works.com>
parents: 692
diff changeset
107 .order_by(Repository.repo_name).all()
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
108
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
109 for r in all_repos:
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
110
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
111 repo = self.get(r.repo_name)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
112
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
113 if repo is not None:
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
114 last_change = repo.last_change
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
115 tip = h.get_changeset_safe(repo, 'tip')
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
116
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
117 tmp_d = {}
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
118 tmp_d['name'] = repo.name
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
119 tmp_d['name_sort'] = tmp_d['name'].lower()
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
120 tmp_d['description'] = repo.dbrepo.description
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
121 tmp_d['description_sort'] = tmp_d['description']
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
122 tmp_d['last_change'] = last_change
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
123 tmp_d['last_change_sort'] = time.mktime(last_change.timetuple())
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
124 tmp_d['tip'] = tip.raw_id
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
125 tmp_d['tip_sort'] = tip.revision
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
126 tmp_d['rev'] = tip.revision
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
127 tmp_d['contact'] = repo.dbrepo.user.full_contact
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
128 tmp_d['contact_sort'] = tmp_d['contact']
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
129 tmp_d['repo_archives'] = list(repo._get_archives())
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
130 tmp_d['last_msg'] = tip.message
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
131 tmp_d['repo'] = repo
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
132 yield tmp_d
74
cdf4fda66dd9 Started summary page. Added filters to templates. used by n,self.f.filtername prefixed by n to disable other filters. Few other fixes found
Marcin Kuzminski <marcin@python-blog.com>
parents: 73
diff changeset
133
cdf4fda66dd9 Started summary page. Added filters to templates. used by n,self.f.filtername prefixed by n to disable other filters. Few other fixes found
Marcin Kuzminski <marcin@python-blog.com>
parents: 73
diff changeset
134 def get_repo(self, repo_name):
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
135 return self.get(repo_name)
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
136
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
137 def get(self, repo_name):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
138 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
139 Get's repository from given name, creates BackendInstance and
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
140 propagates it's data from database with all additional information
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
141 :param repo_name:
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
142 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
143 if not HasRepoPermissionAny('repository.read', 'repository.write',
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
144 'repository.admin')(repo_name, 'get repo check'):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
145 return
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
146
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
147 @cache_region('long_term')
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
148 def _get_repo(repo_name):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
149
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
150 repo_path = os.path.join(self.repos_path, repo_name)
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
151 alias = get_scm(repo_path)[0]
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
152
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
153 log.debug('Creating instance of %s repository', alias)
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
154 backend = get_backend(alias)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
155
710
e2f3c8e6939d Disable git support due to large problems with dulwich.
Marcin Kuzminski <marcin@python-works.com>
parents: 693
diff changeset
156 #TODO: get the baseui from somewhere for this
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
157 if alias == 'hg':
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
158 repo = backend(repo_path, create=False, baseui=None)
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
159 #skip hidden web repository
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
160 if repo._get_hidden():
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
161 return
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
162 else:
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
163 repo = backend(repo_path, create=False)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
164
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
165 dbrepo = self.sa.query(Repository)\
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
166 .options(joinedload(Repository.fork))\
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
167 .options(joinedload(Repository.user))\
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
168 .filter(Repository.repo_name == repo_name)\
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
169 .scalar()
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
170 make_transient(dbrepo)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
171 repo.dbrepo = dbrepo
535
72778dda34cf some fixups in cache, added fallback and cache invalidation when key not found in cached repos list,
Marcin Kuzminski <marcin@python-works.com>
parents: 531
diff changeset
172 return repo
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
173
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
174 invalidate = self._should_invalidate(repo_name)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
175 if invalidate:
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
176 log.info('invalidating cache for repository %s', repo_name)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
177 region_invalidate(_get_repo, None, repo_name)
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
178 self._mark_invalidated(invalidate)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
179
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
180 return _get_repo(repo_name)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
181
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
182
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
183
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
184 def mark_for_invalidation(self, repo_name):
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
185 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
186 Puts cache invalidation task into db for
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
187 further global cache invalidation
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
188
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
189 :param repo_name: this repo that should invalidation take place
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
190 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
191 log.debug('marking %s for invalidation', repo_name)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
192 cache = self.sa.query(CacheInvalidation)\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
193 .filter(CacheInvalidation.cache_key == repo_name).scalar()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
194
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
195 if cache:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
196 #mark this cache as inactive
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
197 cache.cache_active = False
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
198 else:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
199 log.debug('cache key not found in invalidation db -> creating one')
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
200 cache = CacheInvalidation(repo_name)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
201
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
202 try:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
203 self.sa.add(cache)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
204 self.sa.commit()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
205 except:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
206 log.error(traceback.format_exc())
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
207 self.sa.rollback()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
208
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
209
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
210
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
211
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
212
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
213 def _should_invalidate(self, repo_name):
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
214 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
215 Looks up database for invalidation signals for this repo_name
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
216 :param repo_name:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
217 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
218
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
219 ret = self.sa.query(CacheInvalidation)\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
220 .options(FromCache('sql_cache_short',
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
221 'get_invalidation_%s' % repo_name))\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
222 .filter(CacheInvalidation.cache_key == repo_name)\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
223 .filter(CacheInvalidation.cache_active == False)\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
224 .scalar()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
225
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
226 return ret
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
227
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
228 def _mark_invalidated(self, cache_key):
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
229 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
230 Marks all occurences of cache to invaldation as already invalidated
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
231 @param repo_name:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
232 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
233 if cache_key:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
234 log.debug('marking %s as already invalidated', cache_key)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
235 try:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
236 cache_key.cache_active = True
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
237 self.sa.add(cache_key)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
238 self.sa.commit()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
239 except:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
240 log.error(traceback.format_exc())
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
241 self.sa.rollback()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
242