Mercurial > kallithea

annotate rhodecode/lib/celerylib/tasks.py @ 2031:82a88013a3fd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
merge 1.3 into stable
author Marcin Kuzminski <marcin@python-works.com>
date Sun, 26 Feb 2012 17:25:09 +0200
parents afe8cfa32a0f 324ac367a4da
children dc2584ba5fbc
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
903
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
1 # -*- coding: utf-8 -*-
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
2 """
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
3 rhodecode.lib.celerylib.tasks
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
4 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
903
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
5
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
6 RhodeCode task modules, containing all task that suppose to be run
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
7 by celery daemon
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1169
diff changeset
8
903
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
9 :created_on: Oct 6, 2010
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
10 :author: marcink
1824
89efedac4e6c 2012 copyrights
Marcin Kuzminski <marcin@python-works.com>
parents: 1818
diff changeset
11 :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
903
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
12 :license: GPLv3, see COPYING for more details.
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
13 """
1206
a671db5bdd58 fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
14 # This program is free software: you can redistribute it and/or modify
a671db5bdd58 fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
15 # it under the terms of the GNU General Public License as published by
a671db5bdd58 fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
16 # the Free Software Foundation, either version 3 of the License, or
a671db5bdd58 fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
17 # (at your option) any later version.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1169
diff changeset
18 #
903
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
19 # This program is distributed in the hope that it will be useful,
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
20 # but WITHOUT ANY WARRANTY; without even the implied warranty of
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
21 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
22 # GNU General Public License for more details.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1169
diff changeset
23 #
903
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
24 # You should have received a copy of the GNU General Public License
1206
a671db5bdd58 fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
25 # along with this program. If not, see <http://www.gnu.org/licenses/>.
467
3fc3ce53659b starting celery branch
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
26 from celery.decorators import task
555
03676d39dd0a added fault tolerant case when celeryconfig is not present in the directory.
Marcin Kuzminski <marcin@python-works.com>
parents: 549
diff changeset
27
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
28 import os
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
29 import traceback
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
30 import logging
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
31 from os.path import join as jn
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
32
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
33 from time import mktime
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
34 from operator import itemgetter
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
35 from string import lower
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
36
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
37 from pylons import config, url
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
38 from pylons.i18n.translation import _
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
39
2007
324ac367a4da Added VCS into rhodecode core for faster and easier deployments of new versions
Marcin Kuzminski <marcin@python-works.com>
parents: 1976
diff changeset
40 from rhodecode.lib.vcs import get_backend
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
41
1723
64e91067b996 - refactoring to overcome poor usage of global pylons config
Marcin Kuzminski <marcin@python-works.com>
parents: 1722
diff changeset
42 from rhodecode import CELERY_ON
1401
b7563ad4e7ee Unicode fixes, added safe_str method for global str() operations +better test sandboxing
Marcin Kuzminski <marcin@python-works.com>
parents: 1366
diff changeset
43 from rhodecode.lib import LANGUAGES_EXTENSIONS_MAP, safe_str
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
44 from rhodecode.lib.celerylib import run_task, locked_task, dbsession, \
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
45 str2bool, __get_lockkey, LockHeld, DaemonLock, get_session
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
46 from rhodecode.lib.helpers import person
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
47 from rhodecode.lib.rcmail.smtp_mailer import SmtpMailer
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
48 from rhodecode.lib.utils import add_cache, action_logger
1514
87ec80c280bb fixed issues with python2.5
Marcin Kuzminski <marcin@python-works.com>
parents: 1417
diff changeset
49 from rhodecode.lib.compat import json, OrderedDict
87ec80c280bb fixed issues with python2.5
Marcin Kuzminski <marcin@python-works.com>
parents: 1417
diff changeset
50
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
51 from rhodecode.model.db import Statistics, Repository, User
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
52
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
53
777
aac24db58ce8 fixed cache problem,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
54 add_cache(config)
aac24db58ce8 fixed cache problem,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
55
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
56 __all__ = ['whoosh_index', 'get_commits_stats',
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
57 'reset_user_password', 'send_email']
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
58
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
59
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
60 def get_logger(cls):
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
61 if CELERY_ON:
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
62 try:
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
63 log = cls.get_logger()
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
64 except:
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
65 log = logging.getLogger(__name__)
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
66 else:
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
67 log = logging.getLogger(__name__)
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
68
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
69 return log
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
70
1807
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
71
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
72 @task(ignore_result=True)
504
d280aa1c85c6 removed pidlock from whoosh and added it as locked_task decorator
Marcin Kuzminski <marcin@python-works.com>
parents: 502
diff changeset
73 @locked_task
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
74 @dbsession
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
75 def whoosh_index(repo_location, full_index):
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
76 from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
77 log = whoosh_index.get_logger(whoosh_index)
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
78 DBS = get_session()
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
79
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
80 index_location = config['index_dir']
662
373ee7031003 fixed annotation bug, added history to annotation.
Marcin Kuzminski <marcin@python-works.com>
parents: 659
diff changeset
81 WhooshIndexingDaemon(index_location=index_location,
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
82 repo_location=repo_location, sa=DBS)\
777
aac24db58ce8 fixed cache problem,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
83 .run(full_index=full_index)
497
fb0c3af6031b Implemented locking for task, to prevent for running the same tasks,
Marcin Kuzminski <marcin@python-works.com>
parents: 493
diff changeset
84
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
85
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
86 @task(ignore_result=True)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
87 @dbsession
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
88 def get_commits_stats(repo_name, ts_min_y, ts_max_y):
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
89 log = get_logger(get_commits_stats)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
90 DBS = get_session()
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
91 lockkey = __get_lockkey('get_commits_stats', repo_name, ts_min_y,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
92 ts_max_y)
1540
191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable
Marcin Kuzminski <marcin@python-works.com>
parents: 1514
diff changeset
93 lockkey_path = config['here']
191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable
Marcin Kuzminski <marcin@python-works.com>
parents: 1514
diff changeset
94
1976
Marcin Kuzminski <marcin@python-works.com>
parents: 1944
diff changeset
95 log.info('running task with lockkey %s' % lockkey)
1807
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
96
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
97 try:
1540
191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable
Marcin Kuzminski <marcin@python-works.com>
parents: 1514
diff changeset
98 lock = l = DaemonLock(file_=jn(lockkey_path, lockkey))
699
52da7cba88a6 Code refactor for auth func, preparing for ldap support
Marcin Kuzminski <marcin@python-works.com>
parents: 692
diff changeset
99
1976
Marcin Kuzminski <marcin@python-works.com>
parents: 1944
diff changeset
100 # for js data compatibility cleans the key for person from '
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
101 akc = lambda k: person(k).replace('"', "")
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
102
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
103 co_day_auth_aggr = {}
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
104 commits_by_day_aggregate = {}
1749
8ecc6b8229a5 commit less models
Marcin Kuzminski <marcin@python-works.com>
parents: 1747
diff changeset
105 repo = Repository.get_by_repo_name(repo_name)
8ecc6b8229a5 commit less models
Marcin Kuzminski <marcin@python-works.com>
parents: 1747
diff changeset
106 if repo is None:
8ecc6b8229a5 commit less models
Marcin Kuzminski <marcin@python-works.com>
parents: 1747
diff changeset
107 return True
1818
cf51bbfb120e auto white-space removal
Marcin Kuzminski <marcin@python-works.com>
parents: 1807
diff changeset
108
1749
8ecc6b8229a5 commit less models
Marcin Kuzminski <marcin@python-works.com>
parents: 1747
diff changeset
109 repo = repo.scm_instance
1807
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
110 repo_size = repo.count()
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
111 # return if repo have no revisions
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
112 if repo_size < 1:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
113 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
114 return True
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
115
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
116 skip_date_limit = True
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
117 parse_limit = int(config['app_conf'].get('commit_parse_limit'))
1807
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
118 last_rev = None
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
119 last_cs = None
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
120 timegetter = itemgetter('time')
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
121
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
122 dbrepo = DBS.query(Repository)\
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
123 .filter(Repository.repo_name == repo_name).scalar()
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
124 cur_stats = DBS.query(Statistics)\
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
125 .filter(Statistics.repository == dbrepo).scalar()
1105
61c5f0ab24b9 Added branch filter to repo pager
Marcin Kuzminski <marcin@python-works.com>
parents: 1076
diff changeset
126
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
127 if cur_stats is not None:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
128 last_rev = cur_stats.stat_on_revision
1105
61c5f0ab24b9 Added branch filter to repo pager
Marcin Kuzminski <marcin@python-works.com>
parents: 1076
diff changeset
129
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
130 if last_rev == repo.get_changeset().revision and repo_size > 1:
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
131 # pass silently without any work if we're not on first revision or
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
132 # current state of parsing revision(from db marker) is the
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
133 # last revision
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
134 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
135 return True
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
136
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
137 if cur_stats:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
138 commits_by_day_aggregate = OrderedDict(json.loads(
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
139 cur_stats.commit_activity_combined))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
140 co_day_auth_aggr = json.loads(cur_stats.commit_activity)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
141
1976
Marcin Kuzminski <marcin@python-works.com>
parents: 1944
diff changeset
142 log.debug('starting parsing %s' % parse_limit)
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
143 lmktime = mktime
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
144
1807
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
145 last_rev = last_rev + 1 if last_rev >= 0 else 0
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
146 log.debug('Getting revisions from %s to %s' % (
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
147 last_rev, last_rev + parse_limit)
1635a21485d6 - fixed issue with missing commits on some repos commands
Marcin Kuzminski <marcin@python-works.com>
parents: 1749
diff changeset
148 )
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
149 for cs in repo[last_rev:last_rev + parse_limit]:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
150 last_cs = cs # remember last parsed changeset
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
151 k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
152 cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0])
1076
db71228a4c23 moved statistics parse_limit into .ini files
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
153
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
154 if akc(cs.author) in co_day_auth_aggr:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
155 try:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
156 l = [timegetter(x) for x in
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
157 co_day_auth_aggr[akc(cs.author)]['data']]
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
158 time_pos = l.index(k)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
159 except ValueError:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
160 time_pos = False
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
161
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
162 if time_pos >= 0 and time_pos is not False:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
163
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
164 datadict = \
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
165 co_day_auth_aggr[akc(cs.author)]['data'][time_pos]
804
b92d9a0321e0 code stats speed improvments
Marcin Kuzminski <marcin@python-works.com>
parents: 803
diff changeset
166
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
167 datadict["commits"] += 1
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
168 datadict["added"] += len(cs.added)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
169 datadict["changed"] += len(cs.changed)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
170 datadict["removed"] += len(cs.removed)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
171
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
172 else:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
173 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
174
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
175 datadict = {"time": k,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
176 "commits": 1,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
177 "added": len(cs.added),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
178 "changed": len(cs.changed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
179 "removed": len(cs.removed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
180 }
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
181 co_day_auth_aggr[akc(cs.author)]['data']\
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
182 .append(datadict)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
183
467
3fc3ce53659b starting celery branch
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
184 else:
487
b12ea84fb906 Some fixes to summary, and total rewrite of summary graphs implemented more interactive graph.
Marcin Kuzminski <marcin@python-works.com>
parents: 486
diff changeset
185 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
186 co_day_auth_aggr[akc(cs.author)] = {
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
187 "label": akc(cs.author),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
188 "data": [{"time":k,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
189 "commits":1,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
190 "added":len(cs.added),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
191 "changed":len(cs.changed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
192 "removed":len(cs.removed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
193 }],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
194 "schema": ["commits"],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
195 }
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
196
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
197 #gather all data by day
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
198 if k in commits_by_day_aggregate:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
199 commits_by_day_aggregate[k] += 1
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
200 else:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
201 commits_by_day_aggregate[k] = 1
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
202
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
203 overview_data = sorted(commits_by_day_aggregate.items(),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
204 key=itemgetter(0))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
205
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
206 if not co_day_auth_aggr:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
207 co_day_auth_aggr[akc(repo.contact)] = {
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
208 "label": akc(repo.contact),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
209 "data": [0, 1],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
210 "schema": ["commits"],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
211 }
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
212
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
213 stats = cur_stats if cur_stats else Statistics()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
214 stats.commit_activity = json.dumps(co_day_auth_aggr)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
215 stats.commit_activity_combined = json.dumps(overview_data)
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
216
1976
Marcin Kuzminski <marcin@python-works.com>
parents: 1944
diff changeset
217 log.debug('last revison %s' % last_rev)
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
218 leftovers = len(repo.revisions[last_rev:])
1976
Marcin Kuzminski <marcin@python-works.com>
parents: 1944
diff changeset
219 log.debug('revisions to parse %s' % leftovers)
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
220
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
221 if last_rev == 0 or leftovers < parse_limit:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
222 log.debug('getting code trending stats')
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
223 stats.languages = json.dumps(__get_codes_stats(repo_name))
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
224
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
225 try:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
226 stats.repository = dbrepo
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
227 stats.stat_on_revision = last_cs.revision if last_cs else 0
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
228 DBS.add(stats)
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
229 DBS.commit()
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
230 except:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
231 log.error(traceback.format_exc())
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
232 DBS.rollback()
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
233 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
234 return False
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
235
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
236 #final release
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
237 lock.release()
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
238
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
239 #execute another task if celery is enabled
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
240 if len(repo.revisions) > 1 and CELERY_ON:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
241 run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
242 return True
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
243 except LockHeld:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
244 log.info('LockHeld')
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
245 return 'Task with key %s already running' % lockkey
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
246
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
247 @task(ignore_result=True)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
248 @dbsession
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
249 def send_password_link(user_email):
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
250 from rhodecode.model.notification import EmailNotificationModel
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
251
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
252 log = get_logger(send_password_link)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
253 DBS = get_session()
1944
5fc9c92025c1 white space cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 1941
diff changeset
254
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
255 try:
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
256 user = User.get_by_email(user_email)
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
257 if user:
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
258 log.debug('password reset user found %s' % user)
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
259 link = url('reset_password_confirmation', key=user.api_key,
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
260 qualified=True)
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
261 reg_type = EmailNotificationModel.TYPE_PASSWORD_RESET
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
262 body = EmailNotificationModel().get_email_tmpl(reg_type,
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
263 **{'user':user.short_contact,
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
264 'reset_url':link})
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
265 log.debug('sending email')
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
266 run_task(send_email, user_email,
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
267 _("password reset link"), body)
1976
Marcin Kuzminski <marcin@python-works.com>
parents: 1944
diff changeset
268 log.info('send new password mail to %s' % user_email)
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
269 else:
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
270 log.debug("password reset email %s not found" % user_email)
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
271 except:
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
272 log.error(traceback.format_exc())
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
273 return False
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
274
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
275 return True
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
276
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
277 @task(ignore_result=True)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
278 @dbsession
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
279 def reset_user_password(user_email):
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
280 from rhodecode.lib import auth
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
281
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
282 log = get_logger(reset_user_password)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
283 DBS = get_session()
1944
5fc9c92025c1 white space cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 1941
diff changeset
284
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
285 try:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
286 try:
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
287 user = User.get_by_email(user_email)
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
288 new_passwd = auth.PasswordGenerator().gen_password(8,
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
289 auth.PasswordGenerator.ALPHABETS_BIG_SMALL)
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
290 if user:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
291 user.password = auth.get_crypt_password(new_passwd)
1116
716911af91e1 Added api_key into user, api key get's generated again after password change
Marcin Kuzminski <marcin@python-works.com>
parents: 1105
diff changeset
292 user.api_key = auth.generate_api_key(user.username)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
293 DBS.add(user)
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
294 DBS.commit()
1976
Marcin Kuzminski <marcin@python-works.com>
parents: 1944
diff changeset
295 log.info('change password for %s' % user_email)
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
296 if new_passwd is None:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
297 raise Exception('unable to generate new password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
298 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
299 log.error(traceback.format_exc())
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
300 DBS.rollback()
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
301
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
302 run_task(send_email, user_email,
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
303 'Your new password',
1417
5875955def39 fixes #223 improve password reset form
Marcin Kuzminski <marcin@python-works.com>
parents: 1401
diff changeset
304 'Your new RhodeCode password:%s' % (new_passwd))
1976
Marcin Kuzminski <marcin@python-works.com>
parents: 1944
diff changeset
305 log.info('send new password mail to %s' % user_email)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
306
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
307 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
308 log.error('Failed to update user password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
309 log.error(traceback.format_exc())
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
310
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
311 return True
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
312
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
313
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
314 @task(ignore_result=True)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
315 @dbsession
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
316 def send_email(recipients, subject, body, html_body=''):
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
317 """
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
318 Sends an email with defined parameters from the .ini files.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1169
diff changeset
319
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
320 :param recipients: list of recipients, it this is empty the defined email
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
321 address from field 'email_to' is used instead
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
322 :param subject: subject of the mail
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
323 :param body: body of the mail
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
324 :param html_body: html version of body
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
325 """
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
326 log = get_logger(send_email)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
327 DBS = get_session()
1944
5fc9c92025c1 white space cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 1941
diff changeset
328
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
329 email_config = config
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
330 subject = "%s %s" % (email_config.get('email_prefix'), subject)
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
331 if not recipients:
1612
5cb398de935e implements #291 email notification sent to all admin users
Marcin Kuzminski <marcin@python-works.com>
parents: 1581
diff changeset
332 # if recipients are not defined we send to email_config + all admins
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
333 admins = [u.email for u in User.query()
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
334 .filter(User.admin == True).all()]
1612
5cb398de935e implements #291 email notification sent to all admin users
Marcin Kuzminski <marcin@python-works.com>
parents: 1581
diff changeset
335 recipients = [email_config.get('email_to')] + admins
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
336
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
337 mail_from = email_config.get('app_email_from', 'RhodeCode')
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
338 user = email_config.get('smtp_username')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
339 passwd = email_config.get('smtp_password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
340 mail_server = email_config.get('smtp_server')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
341 mail_port = email_config.get('smtp_port')
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
342 tls = str2bool(email_config.get('smtp_use_tls'))
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
343 ssl = str2bool(email_config.get('smtp_use_ssl'))
1169
f6dca275c5a8 control mailer debug with the .ini file
Marcin Kuzminski <marcin@python-works.com>
parents: 1116
diff changeset
344 debug = str2bool(config.get('debug'))
1581
67377fd685be applied smth_auth options update patch
Les Peabody <lpeabody@ccom.unh.edu>
parents: 1540
diff changeset
345 smtp_auth = email_config.get('smtp_auth')
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
346
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
347 try:
1673
333b3e7c991f Added email sending test site
Marcin Kuzminski <marcin@python-works.com>
parents: 1612
diff changeset
348 m = SmtpMailer(mail_from, user, passwd, mail_server, smtp_auth,
1169
f6dca275c5a8 control mailer debug with the .ini file
Marcin Kuzminski <marcin@python-works.com>
parents: 1116
diff changeset
349 mail_port, ssl, tls, debug=debug)
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
350 m.send(recipients, subject, body, html_body)
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
351 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
352 log.error('Mail sending failed')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
353 log.error(traceback.format_exc())
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
354 return False
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
355 return True
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
356
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
357
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
358 @task(ignore_result=True)
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
359 @dbsession
530
a08f610e545e Implemented server side forks
Marcin Kuzminski <marcin@python-works.com>
parents: 509
diff changeset
360 def create_repo_fork(form_data, cur_user):
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
361 """
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
362 Creates a fork of repository using interval VCS methods
1818
cf51bbfb120e auto white-space removal
Marcin Kuzminski <marcin@python-works.com>
parents: 1807
diff changeset
363
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
364 :param form_data:
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
365 :param cur_user:
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
366 """
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
367 from rhodecode.model.repo import RepoModel
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
368
1717
7ff304d3028f Notification fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 1713
diff changeset
369 log = get_logger(create_repo_fork)
1941
58c684bacdad fix leftover bad code on fork function
Marcin Kuzminski <marcin@python-works.com>
parents: 1929
diff changeset
370 DBS = get_session()
58c684bacdad fix leftover bad code on fork function
Marcin Kuzminski <marcin@python-works.com>
parents: 1929
diff changeset
371
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
372 base_path = Repository.base_path()
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
373
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
374 RepoModel(DBS).create(form_data, cur_user, just_db=True, fork=True)
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
375
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
376 alias = form_data['repo_type']
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
377 org_repo_name = form_data['org_path']
1730
ce0b47534c36 fixed fork journal entry
Marcin Kuzminski <marcin@python-works.com>
parents: 1728
diff changeset
378 fork_name = form_data['repo_name_full']
1742
40c4f7351574 added option to do a checkout after cloning a repository
Marcin Kuzminski <marcin@python-works.com>
parents: 1730
diff changeset
379 update_after_clone = form_data['update_after_clone']
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
380 source_repo_path = os.path.join(base_path, org_repo_name)
1730
ce0b47534c36 fixed fork journal entry
Marcin Kuzminski <marcin@python-works.com>
parents: 1728
diff changeset
381 destination_fork_path = os.path.join(base_path, fork_name)
1264
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task.
Marcin Kuzminski <marcin@python-works.com>
parents: 1244
diff changeset
382
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
383 log.info('creating fork of %s as %s', source_repo_path,
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
384 destination_fork_path)
659
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type.
Marcin Kuzminski <marcin@python-works.com>
parents: 635
diff changeset
385 backend = get_backend(alias)
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
386 backend(safe_str(destination_fork_path), create=True,
1742
40c4f7351574 added option to do a checkout after cloning a repository
Marcin Kuzminski <marcin@python-works.com>
parents: 1730
diff changeset
387 src_url=safe_str(source_repo_path),
40c4f7351574 added option to do a checkout after cloning a repository
Marcin Kuzminski <marcin@python-works.com>
parents: 1730
diff changeset
388 update_after_clone=update_after_clone)
1730
ce0b47534c36 fixed fork journal entry
Marcin Kuzminski <marcin@python-works.com>
parents: 1728
diff changeset
389 action_logger(cur_user, 'user_forked_repo:%s' % fork_name,
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
390 org_repo_name, '', DBS)
1818
cf51bbfb120e auto white-space removal
Marcin Kuzminski <marcin@python-works.com>
parents: 1807
diff changeset
391
1747
8804715404c9 implements #193 journal stores information about deleting of repos
Marcin Kuzminski <marcin@python-works.com>
parents: 1742
diff changeset
392 action_logger(cur_user, 'user_created_fork:%s' % fork_name,
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
393 fork_name, '', DBS)
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
394 # finally commit at latest possible stage
1929
cd8a7e3698bc fixes #340 session cleanup for celery tasks
Marcin Kuzminski <marcin@python-works.com>
parents: 1824
diff changeset
395 DBS.commit()
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
396
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
397 def __get_codes_stats(repo_name):
1722
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
398 repo = Repository.get_by_repo_name(repo_name).scm_instance
e7eef7a1db6a #235 forking page repo group selection
Marcin Kuzminski <marcin@python-works.com>
parents: 1717
diff changeset
399
603
95a502d94860 removed soon deprecated walk method on repository instance
Marcin Kuzminski <marcin@python-works.com>
parents: 564
diff changeset
400 tip = repo.get_changeset()
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
401 code_stats = {}
630
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
402
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
403 def aggregate(cs):
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
404 for f in cs[2]:
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
405 ext = lower(f.extension)
789
9fec2207f91c removed binary files from trending sources
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
406 if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary:
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
407 if ext in code_stats:
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
408 code_stats[ext] += 1
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
409 else:
1244
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away.
Marcin Kuzminski <marcin@python-works.com>
parents: 1206
diff changeset
410 code_stats[ext] = 1
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
411
630
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
412 map(aggregate, tip.walk('/'))
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
413
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
414 return code_stats or {}