kallithea: rhodecode/lib/celerylib/tasks.py annotate

annotate rhodecode/lib/celerylib/tasks.py @ 1887:3246fcce2402

backported fixes for statistics from beta branch, try to fix issue #271 - version bump

author	Marcin Kuzminski <marcin@python-works.com>
date	Thu, 12 Jan 2012 06:21:48 +0200
parents	99c70e97ce48
children	afe8cfa32a0f

rev	line source
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	1 # -- coding: utf-8 --
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	2 """
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	3 rhodecode.lib.celerylib.tasks
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	4 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	5
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	6 RhodeCode task modules, containing all task that suppose to be run
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	7 by celery daemon
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	8
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	9 :created_on: Oct 6, 2010
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	10 :author: marcink
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	11 :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	12 :license: GPLv3, see COPYING for more details.
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	13 """
1206 a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	14 # This program is free software: you can redistribute it and/or modify
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	15 # it under the terms of the GNU General Public License as published by
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	16 # the Free Software Foundation, either version 3 of the License, or
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	17 # (at your option) any later version.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	18 #
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	19 # This program is distributed in the hope that it will be useful,
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	20 # but WITHOUT ANY WARRANTY; without even the implied warranty of
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	21 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	22 # GNU General Public License for more details.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	23 #
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	24 # You should have received a copy of the GNU General Public License
1206 a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	25 # along with this program. If not, see <http://www.gnu.org/licenses/>.
467 3fc3ce53659b starting celery branch Marcin Kuzminski <marcin@python-works.com> parents: diff changeset	26 from celery.decorators import task
555 03676d39dd0a added fault tolerant case when celeryconfig is not present in the directory. Marcin Kuzminski <marcin@python-works.com> parents: 549 diff changeset	27
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	28 import os
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	29 import traceback
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	30 import logging
1354 ed309b1fbaa4 fixes issue #197 Relative paths for pidlocks Marcin Kuzminski <marcin@python-works.com> parents: 1337 diff changeset	31 from os.path import dirname as dn, join as jn
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	32
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	33 from time import mktime
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	34 from operator import itemgetter
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	35 from string import lower
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	36
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	37 from pylons import config, url
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	38 from pylons.i18n.translation import _
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	39
1401 b7563ad4e7ee Unicode fixes, added safe_str method for global str() operations +better test sandboxing Marcin Kuzminski <marcin@python-works.com> parents: 1366 diff changeset	40 from rhodecode.lib import LANGUAGES_EXTENSIONS_MAP, safe_str
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	41 from rhodecode.lib.celerylib import run_task, locked_task, str2bool, \
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	42 __get_lockkey, LockHeld, DaemonLock
547 1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	43 from rhodecode.lib.helpers import person
1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	44 from rhodecode.lib.smtp_mailer import SmtpMailer
1337 37625d304a16 Changed OrderedDict implementation to pypy odict, in general it's the fastest and most reliable solution. Added OrderedTuple from python foundation. Marcin Kuzminski <marcin@python-works.com> parents: 1302 diff changeset	45 from rhodecode.lib.utils import add_cache
1514 87ec80c280bb fixed issues with python2.5 Marcin Kuzminski <marcin@python-works.com> parents: 1417 diff changeset	46 from rhodecode.lib.compat import json, OrderedDict
87ec80c280bb fixed issues with python2.5 Marcin Kuzminski <marcin@python-works.com> parents: 1417 diff changeset	47
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	48 from rhodecode.model import init_model
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	49 from rhodecode.model import meta
1642 c0d8171ade36 implements #291 email notification sent to all admin users Marcin Kuzminski <marcin@python-works.com> parents: 1639 diff changeset	50 from rhodecode.model.db import RhodeCodeUi, Statistics, Repository, User
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	51
631 05528ad948c4 Hacking for git support,and new faster repo scan Marcin Kuzminski <marcin@python-works.com> parents: 630 diff changeset	52 from vcs.backends import get_repo
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	53
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	54 from sqlalchemy import engine_from_config
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	55
777 aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	56 add_cache(config)
aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	57
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	58 __all__ = ['whoosh_index', 'get_commits_stats',
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	59 'reset_user_password', 'send_email']
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	60
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	61 CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	62
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	63
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	64 def get_session():
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	65 if CELERY_ON:
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	66 engine = engine_from_config(config, 'sqlalchemy.db1.')
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	67 init_model(engine)
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	68
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	69 sa = meta.Session()
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	70 return sa
467 3fc3ce53659b starting celery branch Marcin Kuzminski <marcin@python-works.com> parents: diff changeset	71
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	72
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	73 def get_repos_path():
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	74 sa = get_session()
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	75 q = sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	76 return q.ui_value
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	77
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	78
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	79 @task(ignore_result=True)
504 d280aa1c85c6 removed pidlock from whoosh and added it as locked_task decorator Marcin Kuzminski <marcin@python-works.com> parents: 502 diff changeset	80 @locked_task
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	81 def whoosh_index(repo_location, full_index):
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	82 #log = whoosh_index.get_logger()
547 1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	83 from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	84 index_location = config['index_dir']
662 373ee7031003 fixed annotation bug, added history to annotation. Marcin Kuzminski <marcin@python-works.com> parents: 659 diff changeset	85 WhooshIndexingDaemon(index_location=index_location,
777 aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	86 repo_location=repo_location, sa=get_session())\
aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	87 .run(full_index=full_index)
497 fb0c3af6031b Implemented locking for task, to prevent for running the same tasks, Marcin Kuzminski <marcin@python-works.com> parents: 493 diff changeset	88
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	89
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	90 @task(ignore_result=True)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	91 def get_commits_stats(repo_name, ts_min_y, ts_max_y):
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	92 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	93 log = get_commits_stats.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	94 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	95 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	96
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	97 lockkey = __get_lockkey('get_commits_stats', repo_name, ts_min_y,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	98 ts_max_y)
1540 191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	99 lockkey_path = config['here']
191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	100
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	101 log.info('running task with lockkey %s', lockkey)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	102 try:
1887 3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	103 sa = get_session()
1540 191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	104 lock = l = DaemonLock(file_=jn(lockkey_path, lockkey))
699 52da7cba88a6 Code refactor for auth func, preparing for ldap support Marcin Kuzminski <marcin@python-works.com> parents: 692 diff changeset	105
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	106 # for js data compatibilty cleans the key for person from '
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	107 akc = lambda k: person(k).replace('"', "")
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	108
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	109 co_day_auth_aggr = {}
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	110 commits_by_day_aggregate = {}
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	111 repos_path = get_repos_path()
1401 b7563ad4e7ee Unicode fixes, added safe_str method for global str() operations +better test sandboxing Marcin Kuzminski <marcin@python-works.com> parents: 1366 diff changeset	112 repo = get_repo(safe_str(os.path.join(repos_path, repo_name)))
1887 3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	113 repo_size = repo.count()
3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	114 # return if repo have no revisions
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	115 if repo_size < 1:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	116 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	117 return True
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	118
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	119 skip_date_limit = True
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	120 parse_limit = int(config['app_conf'].get('commit_parse_limit'))
1887 3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	121 last_rev = None
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	122 last_cs = None
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	123 timegetter = itemgetter('time')
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	124
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	125 dbrepo = sa.query(Repository)\
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	126 .filter(Repository.repo_name == repo_name).scalar()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	127 cur_stats = sa.query(Statistics)\
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	128 .filter(Statistics.repository == dbrepo).scalar()
1105 61c5f0ab24b9 Added branch filter to repo pager Marcin Kuzminski <marcin@python-works.com> parents: 1076 diff changeset	129
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	130 if cur_stats is not None:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	131 last_rev = cur_stats.stat_on_revision
1105 61c5f0ab24b9 Added branch filter to repo pager Marcin Kuzminski <marcin@python-works.com> parents: 1076 diff changeset	132
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	133 if last_rev == repo.get_changeset().revision and repo_size > 1:
1887 3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	134 # pass silently without any work if we're not on first revision or
3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	135 # current state of parsing revision(from db marker) is the
3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	136 # last revision
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	137 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	138 return True
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	139
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	140 if cur_stats:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	141 commits_by_day_aggregate = OrderedDict(json.loads(
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	142 cur_stats.commit_activity_combined))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	143 co_day_auth_aggr = json.loads(cur_stats.commit_activity)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	144
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	145 log.debug('starting parsing %s', parse_limit)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	146 lmktime = mktime
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	147
1887 3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	148 last_rev = last_rev + 1 if last_rev >= 0 else 0
3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	149 log.debug('Getting revisions from %s to %s' % (
3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	150 last_rev, last_rev + parse_limit)
3246fcce2402 backported fixes for statistics from beta branch, try to fix issue #271 Marcin Kuzminski <marcin@python-works.com> parents: 1799 diff changeset	151 )
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	152 for cs in repo[last_rev:last_rev + parse_limit]:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	153 last_cs = cs # remember last parsed changeset
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	154 k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	155 cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0])
1076 db71228a4c23 moved statistics parse_limit into .ini files Marcin Kuzminski <marcin@python-works.com> parents: 1002 diff changeset	156
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	157 if akc(cs.author) in co_day_auth_aggr:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	158 try:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	159 l = [timegetter(x) for x in
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	160 co_day_auth_aggr[akc(cs.author)]['data']]
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	161 time_pos = l.index(k)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	162 except ValueError:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	163 time_pos = False
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	164
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	165 if time_pos >= 0 and time_pos is not False:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	166
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	167 datadict = \
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	168 co_day_auth_aggr[akc(cs.author)]['data'][time_pos]
804 b92d9a0321e0 code stats speed improvments Marcin Kuzminski <marcin@python-works.com> parents: 803 diff changeset	169
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	170 datadict["commits"] += 1
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	171 datadict["added"] += len(cs.added)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	172 datadict["changed"] += len(cs.changed)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	173 datadict["removed"] += len(cs.removed)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	174
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	175 else:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	176 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	177
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	178 datadict = {"time": k,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	179 "commits": 1,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	180 "added": len(cs.added),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	181 "changed": len(cs.changed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	182 "removed": len(cs.removed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	183 }
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	184 co_day_auth_aggr[akc(cs.author)]['data']\
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	185 .append(datadict)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	186
467 3fc3ce53659b starting celery branch Marcin Kuzminski <marcin@python-works.com> parents: diff changeset	187 else:
487 b12ea84fb906 Some fixes to summary, and total rewrite of summary graphs implemented more interactive graph. Marcin Kuzminski <marcin@python-works.com> parents: 486 diff changeset	188 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	189 co_day_auth_aggr[akc(cs.author)] = {
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	190 "label": akc(cs.author),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	191 "data": [{"time":k,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	192 "commits":1,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	193 "added":len(cs.added),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	194 "changed":len(cs.changed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	195 "removed":len(cs.removed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	196 }],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	197 "schema": ["commits"],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	198 }
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	199
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	200 #gather all data by day
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	201 if k in commits_by_day_aggregate:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	202 commits_by_day_aggregate[k] += 1
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	203 else:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	204 commits_by_day_aggregate[k] = 1
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	205
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	206 overview_data = sorted(commits_by_day_aggregate.items(),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	207 key=itemgetter(0))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	208
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	209 if not co_day_auth_aggr:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	210 co_day_auth_aggr[akc(repo.contact)] = {
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	211 "label": akc(repo.contact),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	212 "data": [0, 1],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	213 "schema": ["commits"],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	214 }
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	215
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	216 stats = cur_stats if cur_stats else Statistics()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	217 stats.commit_activity = json.dumps(co_day_auth_aggr)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	218 stats.commit_activity_combined = json.dumps(overview_data)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	219
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	220 log.debug('last revison %s', last_rev)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	221 leftovers = len(repo.revisions[last_rev:])
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	222 log.debug('revisions to parse %s', leftovers)
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	223
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	224 if last_rev == 0 or leftovers < parse_limit:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	225 log.debug('getting code trending stats')
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	226 stats.languages = json.dumps(__get_codes_stats(repo_name))
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	227
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	228 try:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	229 stats.repository = dbrepo
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	230 stats.stat_on_revision = last_cs.revision if last_cs else 0
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	231 sa.add(stats)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	232 sa.commit()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	233 except:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	234 log.error(traceback.format_exc())
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	235 sa.rollback()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	236 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	237 return False
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	238
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	239 # final release
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	240 lock.release()
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	241
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	242 # execute another task if celery is enabled
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	243 if len(repo.revisions) > 1 and CELERY_ON:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	244 run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	245 return True
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	246 except LockHeld:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	247 log.info('LockHeld')
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	248 return 'Task with key %s already running' % lockkey
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	249
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	250 @task(ignore_result=True)
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	251 def send_password_link(user_email):
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	252 try:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	253 log = reset_user_password.get_logger()
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	254 except:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	255 log = logging.getLogger(__name__)
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	256
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	257 from rhodecode.lib import auth
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	258 from rhodecode.model.db import User
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	259
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	260 try:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	261 sa = get_session()
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	262 user = sa.query(User).filter(User.email == user_email).scalar()
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	263
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	264 if user:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	265 link = url('reset_password_confirmation', key=user.api_key,
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	266 qualified=True)
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	267 tmpl = """
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	268 Hello %s
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	269
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	270 We received a request to create a new password for your account.
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	271
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	272 You can generate it by clicking following URL:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	273
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	274 %s
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	275
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	276 If you didn't request new password please ignore this email.
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	277 """
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	278 run_task(send_email, user_email,
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	279 "RhodeCode password reset link",
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	280 tmpl % (user.short_contact, link))
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	281 log.info('send new password mail to %s', user_email)
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	282
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	283 except:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	284 log.error('Failed to update user password')
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	285 log.error(traceback.format_exc())
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	286 return False
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	287
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	288 return True
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	289
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	290 @task(ignore_result=True)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	291 def reset_user_password(user_email):
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	292 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	293 log = reset_user_password.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	294 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	295 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	296
547 1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	297 from rhodecode.lib import auth
1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	298 from rhodecode.model.db import User
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	299
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	300 try:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	301 try:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	302 sa = get_session()
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	303 user = sa.query(User).filter(User.email == user_email).scalar()
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	304 new_passwd = auth.PasswordGenerator().gen_password(8,
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	305 auth.PasswordGenerator.ALPHABETS_BIG_SMALL)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	306 if user:
2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	307 user.password = auth.get_crypt_password(new_passwd)
1116 716911af91e1 Added api_key into user, api key get's generated again after password change Marcin Kuzminski <marcin@python-works.com> parents: 1105 diff changeset	308 user.api_key = auth.generate_api_key(user.username)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	309 sa.add(user)
2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	310 sa.commit()
2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	311 log.info('change password for %s', user_email)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	312 if new_passwd is None:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	313 raise Exception('unable to generate new password')
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	314
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	315 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	316 log.error(traceback.format_exc())
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	317 sa.rollback()
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	318
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	319 run_task(send_email, user_email,
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	320 "Your new RhodeCode password",
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	321 'Your new RhodeCode password:%s' % (new_passwd))
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	322 log.info('send new password mail to %s', user_email)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	323
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	324 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	325 log.error('Failed to update user password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	326 log.error(traceback.format_exc())
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	327
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	328 return True
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	329
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	330
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	331 @task(ignore_result=True)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	332 def send_email(recipients, subject, body):
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	333 """
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	334 Sends an email with defined parameters from the .ini files.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	335
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	336 :param recipients: list of recipients, it this is empty the defined email
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	337 address from field 'email_to' is used instead
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	338 :param subject: subject of the mail
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	339 :param body: body of the mail
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	340 """
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	341 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	342 log = send_email.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	343 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	344 log = logging.getLogger(__name__)
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	345
99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	346 sa = get_session()
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	347 email_config = config
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	348
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	349 if not recipients:
1642 c0d8171ade36 implements #291 email notification sent to all admin users Marcin Kuzminski <marcin@python-works.com> parents: 1639 diff changeset	350 # if recipients are not defined we send to email_config + all admins
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	351 admins = [
99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	352 u.email for u in sa.query(User).filter(User.admin==True).all()
99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	353 ]
1642 c0d8171ade36 implements #291 email notification sent to all admin users Marcin Kuzminski <marcin@python-works.com> parents: 1639 diff changeset	354 recipients = [email_config.get('email_to')] + admins
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	355
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	356 mail_from = email_config.get('app_email_from')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	357 user = email_config.get('smtp_username')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	358 passwd = email_config.get('smtp_password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	359 mail_server = email_config.get('smtp_server')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	360 mail_port = email_config.get('smtp_port')
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	361 tls = str2bool(email_config.get('smtp_use_tls'))
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	362 ssl = str2bool(email_config.get('smtp_use_ssl'))
1169 f6dca275c5a8 control mailer debug with the .ini file Marcin Kuzminski <marcin@python-works.com> parents: 1116 diff changeset	363 debug = str2bool(config.get('debug'))
1581 67377fd685be applied smth_auth options update patch Les Peabody <lpeabody@ccom.unh.edu> parents: 1540 diff changeset	364 smtp_auth = email_config.get('smtp_auth')
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	365
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	366 try:
1581 67377fd685be applied smth_auth options update patch Les Peabody <lpeabody@ccom.unh.edu> parents: 1540 diff changeset	367 m = SmtpMailer(mail_from, user, passwd, mail_server,smtp_auth,
1169 f6dca275c5a8 control mailer debug with the .ini file Marcin Kuzminski <marcin@python-works.com> parents: 1116 diff changeset	368 mail_port, ssl, tls, debug=debug)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	369 m.send(recipients, subject, body)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	370 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	371 log.error('Mail sending failed')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	372 log.error(traceback.format_exc())
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	373 return False
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	374 return True
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	375
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	376
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	377 @task(ignore_result=True)
530 a08f610e545e Implemented server side forks Marcin Kuzminski <marcin@python-works.com> parents: 509 diff changeset	378 def create_repo_fork(form_data, cur_user):
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	379 from rhodecode.model.repo import RepoModel
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	380 from vcs import get_backend
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	381
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	382 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	383 log = create_repo_fork.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	384 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	385 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	386
752 89b9037d68b7 fixed Example celery config to ampq, Marcin Kuzminski <marcin@python-works.com> parents: 722 diff changeset	387 repo_model = RepoModel(get_session())
630 a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	388 repo_model.create(form_data, cur_user, just_db=True, fork=True)
659 758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	389 repo_name = form_data['repo_name']
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	390 repos_path = get_repos_path()
659 758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	391 repo_path = os.path.join(repos_path, repo_name)
530 a08f610e545e Implemented server side forks Marcin Kuzminski <marcin@python-works.com> parents: 509 diff changeset	392 repo_fork_path = os.path.join(repos_path, form_data['fork_name'])
659 758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	393 alias = form_data['repo_type']
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	394
659 758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	395 log.info('creating repo fork %s as %s', repo_name, repo_path)
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	396 backend = get_backend(alias)
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	397 backend(str(repo_fork_path), create=True, src_url=str(repo_path))
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	398
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	399
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	400 def __get_codes_stats(repo_name):
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	401 repos_path = get_repos_path()
1401 b7563ad4e7ee Unicode fixes, added safe_str method for global str() operations +better test sandboxing Marcin Kuzminski <marcin@python-works.com> parents: 1366 diff changeset	402 repo = get_repo(safe_str(os.path.join(repos_path, repo_name)))
603 95a502d94860 removed soon deprecated walk method on repository instance Marcin Kuzminski <marcin@python-works.com> parents: 564 diff changeset	403 tip = repo.get_changeset()
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	404 code_stats = {}
630 a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	405
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	406 def aggregate(cs):
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	407 for f in cs[2]:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	408 ext = lower(f.extension)
789 9fec2207f91c removed binary files from trending sources Marcin Kuzminski <marcin@python-works.com> parents: 785 diff changeset	409 if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	410 if ext in code_stats:
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	411 code_stats[ext] += 1
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	412 else:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	413 code_stats[ext] = 1
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	414
630 a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	415 map(aggregate, tip.walk('/'))
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	416
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	417 return code_stats or {}

Mercurial > kallithea

annotate rhodecode/lib/celerylib/tasks.py @ 1887:3246fcce2402