Mercurial > kallithea

annotate rhodecode/model/scm.py @ 692:cb0d9ce6ac5c beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
#50 on point cache invalidation changes. Created cacheInvalidation table cleaned up sa sessions from models, since it wasn't really needed.
author Marcin Kuzminski <marcin@python-works.com>
date Mon, 15 Nov 2010 02:26:19 +0100
parents 7486da5f0628
children 90ac3255c117
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
58
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
1 #!/usr/bin/env python
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
2 # encoding: utf-8
569
000b675e7c1d bumped version, some spelling fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
3 # Model for RhodeCode
252
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
4 # Copyright (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com>
373
3171614c0067 Added permissions check on repo switcher,
Marcin Kuzminski <marcin@python-works.com>
parents: 368
diff changeset
5 #
252
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
6 # This program is free software; you can redistribute it and/or
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
7 # modify it under the terms of the GNU General Public License
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
8 # as published by the Free Software Foundation; version 2
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
9 # of the License or (at your opinion) any later version of the license.
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
10 #
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
11 # This program is distributed in the hope that it will be useful,
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
14 # GNU General Public License for more details.
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
15 #
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
16 # You should have received a copy of the GNU General Public License
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
17 # along with this program; if not, write to the Free Software
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
18 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
19 # MA 02110-1301, USA.
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
20 """
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
21 Created on April 9, 2010
569
000b675e7c1d bumped version, some spelling fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
22 Model for RhodeCode
58
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
23 @author: marcink
252
3782a6d698af licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents: 248
diff changeset
24 """
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
25 from beaker.cache import cache_region, region_invalidate
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
26 from mercurial import ui
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
27 from rhodecode.lib import helpers as h
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
28 from rhodecode.lib.auth import HasRepoPermissionAny
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
29 from rhodecode.lib.utils import get_repos
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
30 from rhodecode.model import meta
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
31 from rhodecode.model.db import Repository, User, RhodeCodeUi, CacheInvalidation
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
32 from rhodecode.model.caching_query import FromCache
535
72778dda34cf some fixups in cache, added fallback and cache invalidation when key not found in cached repos list,
Marcin Kuzminski <marcin@python-works.com>
parents: 531
diff changeset
33 from sqlalchemy.orm import joinedload
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
34 from sqlalchemy.orm.session import make_transient
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
35 from vcs import get_backend
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
36 from vcs.utils.helpers import get_scm
373
3171614c0067 Added permissions check on repo switcher,
Marcin Kuzminski <marcin@python-works.com>
parents: 368
diff changeset
37 from vcs.exceptions import RepositoryError, VCSError
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
38 from vcs.utils.lazy import LazyProperty
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
39 import traceback
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
40 import logging
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
41 import os
633
fcf599cd3404 Move changes for git implementation
Marcin Kuzminski <marcin@python-works.com>
parents: 631
diff changeset
42 import time
fcf599cd3404 Move changes for git implementation
Marcin Kuzminski <marcin@python-works.com>
parents: 631
diff changeset
43
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
44 log = logging.getLogger(__name__)
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
45
691
7486da5f0628 Refactor codes for scm model
Marcin Kuzminski <marcin@python-works.com>
parents: 690
diff changeset
46 class ScmModel(object):
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
47 """
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
48 Mercurial Model
58
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
49 """
8fb1abd4178a Added hg model,implemented removal of repos, added HgModel for fetching repos(with generator)
Marcin Kuzminski <marcin@python-blog.com>
parents:
diff changeset
50
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
51 def __init__(self):
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
52 self.sa = meta.Session()
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
53
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
54 @LazyProperty
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
55 def repos_path(self):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
56 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
57 Get's the repositories root path from database
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
58 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
59 q = self.sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
60
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
61 return q.ui_value
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
62
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
63 def repo_scan(self, repos_path, baseui, initial=False):
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
64 """
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
65 Listing of repositories in given path. This path should not be a
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
66 repository itself. Return a dictionary of repository objects
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
67
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
68 :param repos_path: path to directory containing repositories
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
69 :param baseui
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
70 :param initial: initial scan
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
71 """
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
72 log.info('scanning for repositories in %s', repos_path)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
73
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
74 if not isinstance(baseui, ui.ui):
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
75 baseui = ui.ui()
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
76 repos_list = {}
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
77
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
78 for name, path in get_repos(repos_path):
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
79 try:
248
fb7f066126cc Added support for repository located in subdirectories.
Marcin Kuzminski <marcin@python-works.com>
parents: 245
diff changeset
80 if repos_list.has_key(name):
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
81 raise RepositoryError('Duplicate repository name %s '
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
82 'found in %s' % (name, path))
248
fb7f066126cc Added support for repository located in subdirectories.
Marcin Kuzminski <marcin@python-works.com>
parents: 245
diff changeset
83 else:
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
84
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
85 klass = get_backend(path[0])
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
86
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
87 if path[0] == 'hg':
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
88 repos_list[name] = klass(path[1], baseui=baseui)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
89
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
90 if path[0] == 'git':
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
91 repos_list[name] = klass(path[1])
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
92 except OSError:
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
93 continue
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
94
245
a83a1799480c Reimplemented way of caching repos list, hg model now get's repos objects right from cached dict, this way we skipp creating instances of MercurialRepository and gain performance. Some import cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 222
diff changeset
95 return repos_list
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
96
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
97 def get_repos(self, all_repos=None):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
98 """
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
99 Get all repos from db and for each repo create it's backend instance.
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
100 and fill that backed with information from database
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
101
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
102 :param all_repos: give specific repositories list, good for filtering
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
103 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
104 if not all_repos:
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
105 all_repos = self.sa.query(Repository).all()
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
106
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
107 for r in all_repos:
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
108
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
109 repo = self.get(r.repo_name)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
110
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
111 if repo is not None:
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
112 last_change = repo.last_change
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
113 tip = h.get_changeset_safe(repo, 'tip')
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
114
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
115 tmp_d = {}
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
116 tmp_d['name'] = repo.name
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
117 tmp_d['name_sort'] = tmp_d['name'].lower()
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
118 tmp_d['description'] = repo.dbrepo.description
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
119 tmp_d['description_sort'] = tmp_d['description']
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
120 tmp_d['last_change'] = last_change
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
121 tmp_d['last_change_sort'] = time.mktime(last_change.timetuple())
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
122 tmp_d['tip'] = tip.raw_id
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
123 tmp_d['tip_sort'] = tip.revision
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
124 tmp_d['rev'] = tip.revision
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
125 tmp_d['contact'] = repo.dbrepo.user.full_contact
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
126 tmp_d['contact_sort'] = tmp_d['contact']
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
127 tmp_d['repo_archives'] = list(repo._get_archives())
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
128 tmp_d['last_msg'] = tip.message
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
129 tmp_d['repo'] = repo
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
130 yield tmp_d
74
cdf4fda66dd9 Started summary page. Added filters to templates. used by n,self.f.filtername prefixed by n to disable other filters. Few other fixes found
Marcin Kuzminski <marcin@python-blog.com>
parents: 73
diff changeset
131
cdf4fda66dd9 Started summary page. Added filters to templates. used by n,self.f.filtername prefixed by n to disable other filters. Few other fixes found
Marcin Kuzminski <marcin@python-blog.com>
parents: 73
diff changeset
132 def get_repo(self, repo_name):
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
133 return self.get(repo_name)
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
134
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
135 def get(self, repo_name):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
136 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
137 Get's repository from given name, creates BackendInstance and
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
138 propagates it's data from database with all additional information
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
139 :param repo_name:
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
140 """
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
141 if not HasRepoPermissionAny('repository.read', 'repository.write',
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
142 'repository.admin')(repo_name, 'get repo check'):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
143 return
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
144
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
145 @cache_region('long_term')
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
146 def _get_repo(repo_name):
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
147
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
148 repo_path = os.path.join(self.repos_path, repo_name)
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
149 alias = get_scm(repo_path)[0]
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
150
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
151 log.debug('Creating instance of %s repository', alias)
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
152 backend = get_backend(alias)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
153
690
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
154 if alias == 'hg':
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
155 repo = backend(repo_path, create=False, baseui=None)
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
156 #skip hidden web repository
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
157 if repo._get_hidden():
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
158 return
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
159 else:
4685f3eafd35 Fixed sumamry page description bug
Marcin Kuzminski <marcin@python-works.com>
parents: 665
diff changeset
160 repo = backend(repo_path, create=False)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
161
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
162 dbrepo = self.sa.query(Repository)\
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
163 .options(joinedload(Repository.fork))\
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
164 .options(joinedload(Repository.user))\
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
165 .filter(Repository.repo_name == repo_name)\
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
166 .scalar()
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
167 make_transient(dbrepo)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
168 repo.dbrepo = dbrepo
535
72778dda34cf some fixups in cache, added fallback and cache invalidation when key not found in cached repos list,
Marcin Kuzminski <marcin@python-works.com>
parents: 531
diff changeset
169 return repo
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
170
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
171 invalidate = self._should_invalidate(repo_name)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
172 if invalidate:
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
173 log.info('invalidating cache for repository %s', repo_name)
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
174 region_invalidate(_get_repo, None, repo_name)
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
175 self._mark_invalidated(invalidate)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
176
665
070f32743632 Moved out reposcan into hg Model.
Marcin Kuzminski <marcin@python-works.com>
parents: 636
diff changeset
177 return _get_repo(repo_name)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 569
diff changeset
178
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
179
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
180
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
181 def mark_for_invalidation(self, repo_name):
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
182 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
183 Puts cache invalidation task into db for
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
184 further global cache invalidation
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
185
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
186 :param repo_name: this repo that should invalidation take place
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
187 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
188 log.debug('marking %s for invalidation', repo_name)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
189 cache = self.sa.query(CacheInvalidation)\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
190 .filter(CacheInvalidation.cache_key == repo_name).scalar()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
191
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
192 if cache:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
193 #mark this cache as inactive
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
194 cache.cache_active = False
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
195 else:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
196 log.debug('cache key not found in invalidation db -> creating one')
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
197 cache = CacheInvalidation(repo_name)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
198
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
199 try:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
200 self.sa.add(cache)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
201 self.sa.commit()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
202 except:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
203 log.error(traceback.format_exc())
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
204 self.sa.rollback()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
205
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
206
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
207
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
208
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
209
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
210 def _should_invalidate(self, repo_name):
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
211 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
212 Looks up database for invalidation signals for this repo_name
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
213 :param repo_name:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
214 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
215
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
216 ret = self.sa.query(CacheInvalidation)\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
217 .options(FromCache('sql_cache_short',
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
218 'get_invalidation_%s' % repo_name))\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
219 .filter(CacheInvalidation.cache_key == repo_name)\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
220 .filter(CacheInvalidation.cache_active == False)\
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
221 .scalar()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
222
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
223 return ret
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
224
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
225 def _mark_invalidated(self, cache_key):
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
226 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
227 Marks all occurences of cache to invaldation as already invalidated
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
228 @param repo_name:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
229 """
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
230 if cache_key:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
231 log.debug('marking %s as already invalidated', cache_key)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
232 try:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
233 cache_key.cache_active = True
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
234 self.sa.add(cache_key)
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
235 self.sa.commit()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
236 except:
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
237 log.error(traceback.format_exc())
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
238 self.sa.rollback()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
239