kallithea: rhodecode/lib/celerylib/tasks.py annotate

annotate rhodecode/lib/celerylib/tasks.py @ 1799:99c70e97ce48

fixes issue with unbound session, thanks to slafs for patch

author	Marcin Kuzminski <marcin@python-works.com>
date	Wed, 21 Dec 2011 02:47:39 +0200
parents	c0d8171ade36
children	3246fcce2402

rev	line source
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	1 # -- coding: utf-8 --
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	2 """
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	3 rhodecode.lib.celerylib.tasks
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	4 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	5
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	6 RhodeCode task modules, containing all task that suppose to be run
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	7 by celery daemon
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	8
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	9 :created_on: Oct 6, 2010
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	10 :author: marcink
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	11 :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	12 :license: GPLv3, see COPYING for more details.
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	13 """
1206 a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	14 # This program is free software: you can redistribute it and/or modify
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	15 # it under the terms of the GNU General Public License as published by
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	16 # the Free Software Foundation, either version 3 of the License, or
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	17 # (at your option) any later version.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	18 #
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	19 # This program is distributed in the hope that it will be useful,
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	20 # but WITHOUT ANY WARRANTY; without even the implied warranty of
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	21 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	22 # GNU General Public License for more details.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	23 #
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	24 # You should have received a copy of the GNU General Public License
1206 a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	25 # along with this program. If not, see <http://www.gnu.org/licenses/>.
467 3fc3ce53659b starting celery branch Marcin Kuzminski <marcin@python-works.com> parents: diff changeset	26 from celery.decorators import task
555 03676d39dd0a added fault tolerant case when celeryconfig is not present in the directory. Marcin Kuzminski <marcin@python-works.com> parents: 549 diff changeset	27
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	28 import os
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	29 import traceback
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	30 import logging
1354 ed309b1fbaa4 fixes issue #197 Relative paths for pidlocks Marcin Kuzminski <marcin@python-works.com> parents: 1337 diff changeset	31 from os.path import dirname as dn, join as jn
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	32
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	33 from time import mktime
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	34 from operator import itemgetter
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	35 from string import lower
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	36
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	37 from pylons import config, url
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	38 from pylons.i18n.translation import _
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	39
1401 b7563ad4e7ee Unicode fixes, added safe_str method for global str() operations +better test sandboxing Marcin Kuzminski <marcin@python-works.com> parents: 1366 diff changeset	40 from rhodecode.lib import LANGUAGES_EXTENSIONS_MAP, safe_str
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	41 from rhodecode.lib.celerylib import run_task, locked_task, str2bool, \
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	42 __get_lockkey, LockHeld, DaemonLock
547 1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	43 from rhodecode.lib.helpers import person
1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	44 from rhodecode.lib.smtp_mailer import SmtpMailer
1337 37625d304a16 Changed OrderedDict implementation to pypy odict, in general it's the fastest and most reliable solution. Added OrderedTuple from python foundation. Marcin Kuzminski <marcin@python-works.com> parents: 1302 diff changeset	45 from rhodecode.lib.utils import add_cache
1514 87ec80c280bb fixed issues with python2.5 Marcin Kuzminski <marcin@python-works.com> parents: 1417 diff changeset	46 from rhodecode.lib.compat import json, OrderedDict
87ec80c280bb fixed issues with python2.5 Marcin Kuzminski <marcin@python-works.com> parents: 1417 diff changeset	47
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	48 from rhodecode.model import init_model
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	49 from rhodecode.model import meta
1642 c0d8171ade36 implements #291 email notification sent to all admin users Marcin Kuzminski <marcin@python-works.com> parents: 1639 diff changeset	50 from rhodecode.model.db import RhodeCodeUi, Statistics, Repository, User
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	51
631 05528ad948c4 Hacking for git support,and new faster repo scan Marcin Kuzminski <marcin@python-works.com> parents: 630 diff changeset	52 from vcs.backends import get_repo
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	53
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	54 from sqlalchemy import engine_from_config
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	55
777 aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	56 add_cache(config)
aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	57
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	58 __all__ = ['whoosh_index', 'get_commits_stats',
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	59 'reset_user_password', 'send_email']
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	60
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	61 CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	62
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	63
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	64 def get_session():
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	65 if CELERY_ON:
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	66 engine = engine_from_config(config, 'sqlalchemy.db1.')
f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	67 init_model(engine)
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	68
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	69 sa = meta.Session()
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	70 return sa
467 3fc3ce53659b starting celery branch Marcin Kuzminski <marcin@python-works.com> parents: diff changeset	71
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	72
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	73 def get_repos_path():
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	74 sa = get_session()
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	75 q = sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	76 return q.ui_value
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	77
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	78
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	79 @task(ignore_result=True)
504 d280aa1c85c6 removed pidlock from whoosh and added it as locked_task decorator Marcin Kuzminski <marcin@python-works.com> parents: 502 diff changeset	80 @locked_task
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	81 def whoosh_index(repo_location, full_index):
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	82 #log = whoosh_index.get_logger()
547 1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	83 from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	84 index_location = config['index_dir']
662 373ee7031003 fixed annotation bug, added history to annotation. Marcin Kuzminski <marcin@python-works.com> parents: 659 diff changeset	85 WhooshIndexingDaemon(index_location=index_location,
777 aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	86 repo_location=repo_location, sa=get_session())\
aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	87 .run(full_index=full_index)
497 fb0c3af6031b Implemented locking for task, to prevent for running the same tasks, Marcin Kuzminski <marcin@python-works.com> parents: 493 diff changeset	88
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	89
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	90 @task(ignore_result=True)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	91 def get_commits_stats(repo_name, ts_min_y, ts_max_y):
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	92 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	93 log = get_commits_stats.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	94 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	95 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	96
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	97 lockkey = __get_lockkey('get_commits_stats', repo_name, ts_min_y,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	98 ts_max_y)
1540 191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	99 lockkey_path = config['here']
191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	100
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	101 log.info('running task with lockkey %s', lockkey)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	102 try:
1540 191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	103 lock = l = DaemonLock(file_=jn(lockkey_path, lockkey))
699 52da7cba88a6 Code refactor for auth func, preparing for ldap support Marcin Kuzminski <marcin@python-works.com> parents: 692 diff changeset	104
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	105 # for js data compatibilty cleans the key for person from '
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	106 akc = lambda k: person(k).replace('"', "")
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	107
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	108 co_day_auth_aggr = {}
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	109 commits_by_day_aggregate = {}
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	110 repos_path = get_repos_path()
1401 b7563ad4e7ee Unicode fixes, added safe_str method for global str() operations +better test sandboxing Marcin Kuzminski <marcin@python-works.com> parents: 1366 diff changeset	111 repo = get_repo(safe_str(os.path.join(repos_path, repo_name)))
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	112 repo_size = len(repo.revisions)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	113 #return if repo have no revisions
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	114 if repo_size < 1:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	115 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	116 return True
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	117
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	118 skip_date_limit = True
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	119 parse_limit = int(config['app_conf'].get('commit_parse_limit'))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	120 last_rev = 0
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	121 last_cs = None
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	122 timegetter = itemgetter('time')
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	123
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	124 sa = get_session()
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	125
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	126 dbrepo = sa.query(Repository)\
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	127 .filter(Repository.repo_name == repo_name).scalar()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	128 cur_stats = sa.query(Statistics)\
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	129 .filter(Statistics.repository == dbrepo).scalar()
1105 61c5f0ab24b9 Added branch filter to repo pager Marcin Kuzminski <marcin@python-works.com> parents: 1076 diff changeset	130
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	131 if cur_stats is not None:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	132 last_rev = cur_stats.stat_on_revision
1105 61c5f0ab24b9 Added branch filter to repo pager Marcin Kuzminski <marcin@python-works.com> parents: 1076 diff changeset	133
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	134 if last_rev == repo.get_changeset().revision and repo_size > 1:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	135 #pass silently without any work if we're not on first revision or
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	136 #current state of parsing revision(from db marker) is the
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	137 #last revision
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	138 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	139 return True
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	140
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	141 if cur_stats:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	142 commits_by_day_aggregate = OrderedDict(json.loads(
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	143 cur_stats.commit_activity_combined))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	144 co_day_auth_aggr = json.loads(cur_stats.commit_activity)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	145
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	146 log.debug('starting parsing %s', parse_limit)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	147 lmktime = mktime
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	148
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	149 last_rev = last_rev + 1 if last_rev > 0 else last_rev
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	150
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	151 for cs in repo[last_rev:last_rev + parse_limit]:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	152 last_cs = cs # remember last parsed changeset
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	153 k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	154 cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0])
1076 db71228a4c23 moved statistics parse_limit into .ini files Marcin Kuzminski <marcin@python-works.com> parents: 1002 diff changeset	155
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	156 if akc(cs.author) in co_day_auth_aggr:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	157 try:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	158 l = [timegetter(x) for x in
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	159 co_day_auth_aggr[akc(cs.author)]['data']]
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	160 time_pos = l.index(k)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	161 except ValueError:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	162 time_pos = False
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	163
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	164 if time_pos >= 0 and time_pos is not False:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	165
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	166 datadict = \
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	167 co_day_auth_aggr[akc(cs.author)]['data'][time_pos]
804 b92d9a0321e0 code stats speed improvments Marcin Kuzminski <marcin@python-works.com> parents: 803 diff changeset	168
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	169 datadict["commits"] += 1
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	170 datadict["added"] += len(cs.added)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	171 datadict["changed"] += len(cs.changed)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	172 datadict["removed"] += len(cs.removed)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	173
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	174 else:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	175 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	176
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	177 datadict = {"time": k,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	178 "commits": 1,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	179 "added": len(cs.added),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	180 "changed": len(cs.changed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	181 "removed": len(cs.removed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	182 }
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	183 co_day_auth_aggr[akc(cs.author)]['data']\
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	184 .append(datadict)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	185
467 3fc3ce53659b starting celery branch Marcin Kuzminski <marcin@python-works.com> parents: diff changeset	186 else:
487 b12ea84fb906 Some fixes to summary, and total rewrite of summary graphs implemented more interactive graph. Marcin Kuzminski <marcin@python-works.com> parents: 486 diff changeset	187 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	188 co_day_auth_aggr[akc(cs.author)] = {
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	189 "label": akc(cs.author),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	190 "data": [{"time":k,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	191 "commits":1,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	192 "added":len(cs.added),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	193 "changed":len(cs.changed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	194 "removed":len(cs.removed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	195 }],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	196 "schema": ["commits"],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	197 }
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	198
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	199 #gather all data by day
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	200 if k in commits_by_day_aggregate:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	201 commits_by_day_aggregate[k] += 1
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	202 else:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	203 commits_by_day_aggregate[k] = 1
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	204
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	205 overview_data = sorted(commits_by_day_aggregate.items(),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	206 key=itemgetter(0))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	207
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	208 if not co_day_auth_aggr:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	209 co_day_auth_aggr[akc(repo.contact)] = {
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	210 "label": akc(repo.contact),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	211 "data": [0, 1],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	212 "schema": ["commits"],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	213 }
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	214
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	215 stats = cur_stats if cur_stats else Statistics()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	216 stats.commit_activity = json.dumps(co_day_auth_aggr)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	217 stats.commit_activity_combined = json.dumps(overview_data)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	218
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	219 log.debug('last revison %s', last_rev)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	220 leftovers = len(repo.revisions[last_rev:])
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	221 log.debug('revisions to parse %s', leftovers)
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	222
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	223 if last_rev == 0 or leftovers < parse_limit:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	224 log.debug('getting code trending stats')
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	225 stats.languages = json.dumps(__get_codes_stats(repo_name))
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	226
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	227 try:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	228 stats.repository = dbrepo
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	229 stats.stat_on_revision = last_cs.revision if last_cs else 0
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	230 sa.add(stats)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	231 sa.commit()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	232 except:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	233 log.error(traceback.format_exc())
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	234 sa.rollback()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	235 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	236 return False
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	237
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	238 # final release
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	239 lock.release()
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	240
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	241 # execute another task if celery is enabled
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	242 if len(repo.revisions) > 1 and CELERY_ON:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	243 run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	244 return True
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	245 except LockHeld:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	246 log.info('LockHeld')
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	247 return 'Task with key %s already running' % lockkey
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	248
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	249 @task(ignore_result=True)
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	250 def send_password_link(user_email):
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	251 try:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	252 log = reset_user_password.get_logger()
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	253 except:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	254 log = logging.getLogger(__name__)
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	255
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	256 from rhodecode.lib import auth
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	257 from rhodecode.model.db import User
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	258
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	259 try:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	260 sa = get_session()
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	261 user = sa.query(User).filter(User.email == user_email).scalar()
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	262
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	263 if user:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	264 link = url('reset_password_confirmation', key=user.api_key,
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	265 qualified=True)
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	266 tmpl = """
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	267 Hello %s
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	268
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	269 We received a request to create a new password for your account.
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	270
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	271 You can generate it by clicking following URL:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	272
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	273 %s
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	274
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	275 If you didn't request new password please ignore this email.
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	276 """
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	277 run_task(send_email, user_email,
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	278 "RhodeCode password reset link",
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	279 tmpl % (user.short_contact, link))
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	280 log.info('send new password mail to %s', user_email)
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	281
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	282 except:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	283 log.error('Failed to update user password')
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	284 log.error(traceback.format_exc())
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	285 return False
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	286
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	287 return True
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	288
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	289 @task(ignore_result=True)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	290 def reset_user_password(user_email):
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	291 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	292 log = reset_user_password.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	293 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	294 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	295
547 1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	296 from rhodecode.lib import auth
1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	297 from rhodecode.model.db import User
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	298
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	299 try:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	300 try:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	301 sa = get_session()
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	302 user = sa.query(User).filter(User.email == user_email).scalar()
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	303 new_passwd = auth.PasswordGenerator().gen_password(8,
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	304 auth.PasswordGenerator.ALPHABETS_BIG_SMALL)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	305 if user:
2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	306 user.password = auth.get_crypt_password(new_passwd)
1116 716911af91e1 Added api_key into user, api key get's generated again after password change Marcin Kuzminski <marcin@python-works.com> parents: 1105 diff changeset	307 user.api_key = auth.generate_api_key(user.username)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	308 sa.add(user)
2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	309 sa.commit()
2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	310 log.info('change password for %s', user_email)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	311 if new_passwd is None:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	312 raise Exception('unable to generate new password')
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	313
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	314 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	315 log.error(traceback.format_exc())
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	316 sa.rollback()
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	317
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	318 run_task(send_email, user_email,
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	319 "Your new RhodeCode password",
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	320 'Your new RhodeCode password:%s' % (new_passwd))
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	321 log.info('send new password mail to %s', user_email)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	322
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	323 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	324 log.error('Failed to update user password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	325 log.error(traceback.format_exc())
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	326
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	327 return True
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	328
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	329
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	330 @task(ignore_result=True)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	331 def send_email(recipients, subject, body):
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	332 """
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	333 Sends an email with defined parameters from the .ini files.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	334
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	335 :param recipients: list of recipients, it this is empty the defined email
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	336 address from field 'email_to' is used instead
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	337 :param subject: subject of the mail
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	338 :param body: body of the mail
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	339 """
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	340 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	341 log = send_email.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	342 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	343 log = logging.getLogger(__name__)
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	344
99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	345 sa = get_session()
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	346 email_config = config
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	347
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	348 if not recipients:
1642 c0d8171ade36 implements #291 email notification sent to all admin users Marcin Kuzminski <marcin@python-works.com> parents: 1639 diff changeset	349 # if recipients are not defined we send to email_config + all admins
1799 99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	350 admins = [
99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	351 u.email for u in sa.query(User).filter(User.admin==True).all()
99c70e97ce48 fixes issue with unbound session, thanks to slafs for patch Marcin Kuzminski <marcin@python-works.com> parents: 1642 diff changeset	352 ]
1642 c0d8171ade36 implements #291 email notification sent to all admin users Marcin Kuzminski <marcin@python-works.com> parents: 1639 diff changeset	353 recipients = [email_config.get('email_to')] + admins
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	354
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	355 mail_from = email_config.get('app_email_from')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	356 user = email_config.get('smtp_username')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	357 passwd = email_config.get('smtp_password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	358 mail_server = email_config.get('smtp_server')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	359 mail_port = email_config.get('smtp_port')
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	360 tls = str2bool(email_config.get('smtp_use_tls'))
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	361 ssl = str2bool(email_config.get('smtp_use_ssl'))
1169 f6dca275c5a8 control mailer debug with the .ini file Marcin Kuzminski <marcin@python-works.com> parents: 1116 diff changeset	362 debug = str2bool(config.get('debug'))
1581 67377fd685be applied smth_auth options update patch Les Peabody <lpeabody@ccom.unh.edu> parents: 1540 diff changeset	363 smtp_auth = email_config.get('smtp_auth')
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	364
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	365 try:
1581 67377fd685be applied smth_auth options update patch Les Peabody <lpeabody@ccom.unh.edu> parents: 1540 diff changeset	366 m = SmtpMailer(mail_from, user, passwd, mail_server,smtp_auth,
1169 f6dca275c5a8 control mailer debug with the .ini file Marcin Kuzminski <marcin@python-works.com> parents: 1116 diff changeset	367 mail_port, ssl, tls, debug=debug)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	368 m.send(recipients, subject, body)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	369 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	370 log.error('Mail sending failed')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	371 log.error(traceback.format_exc())
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	372 return False
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	373 return True
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	374
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	375
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	376 @task(ignore_result=True)
530 a08f610e545e Implemented server side forks Marcin Kuzminski <marcin@python-works.com> parents: 509 diff changeset	377 def create_repo_fork(form_data, cur_user):
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	378 from rhodecode.model.repo import RepoModel
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	379 from vcs import get_backend
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	380
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	381 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	382 log = create_repo_fork.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	383 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	384 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	385
752 89b9037d68b7 fixed Example celery config to ampq, Marcin Kuzminski <marcin@python-works.com> parents: 722 diff changeset	386 repo_model = RepoModel(get_session())
630 a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	387 repo_model.create(form_data, cur_user, just_db=True, fork=True)
659 758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	388 repo_name = form_data['repo_name']
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	389 repos_path = get_repos_path()
659 758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	390 repo_path = os.path.join(repos_path, repo_name)
530 a08f610e545e Implemented server side forks Marcin Kuzminski <marcin@python-works.com> parents: 509 diff changeset	391 repo_fork_path = os.path.join(repos_path, form_data['fork_name'])
659 758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	392 alias = form_data['repo_type']
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	393
659 758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	394 log.info('creating repo fork %s as %s', repo_name, repo_path)
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	395 backend = get_backend(alias)
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type. Marcin Kuzminski <marcin@python-works.com> parents: 635 diff changeset	396 backend(str(repo_fork_path), create=True, src_url=str(repo_path))
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	397
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	398
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	399 def __get_codes_stats(repo_name):
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	400 repos_path = get_repos_path()
1401 b7563ad4e7ee Unicode fixes, added safe_str method for global str() operations +better test sandboxing Marcin Kuzminski <marcin@python-works.com> parents: 1366 diff changeset	401 repo = get_repo(safe_str(os.path.join(repos_path, repo_name)))
603 95a502d94860 removed soon deprecated walk method on repository instance Marcin Kuzminski <marcin@python-works.com> parents: 564 diff changeset	402 tip = repo.get_changeset()
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	403 code_stats = {}
630 a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	404
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	405 def aggregate(cs):
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	406 for f in cs[2]:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	407 ext = lower(f.extension)
789 9fec2207f91c removed binary files from trending sources Marcin Kuzminski <marcin@python-works.com> parents: 785 diff changeset	408 if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	409 if ext in code_stats:
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	410 code_stats[ext] += 1
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	411 else:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	412 code_stats[ext] = 1
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	413
630 a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	414 map(aggregate, tip.walk('/'))
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	415
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	416 return code_stats or {}

Mercurial > kallithea

annotate rhodecode/lib/celerylib/tasks.py @ 1799:99c70e97ce48