kallithea: rhodecode/lib/celerylib/tasks.py annotate

annotate rhodecode/lib/celerylib/tasks.py @ 2939:dbe3cfb81446 beta

Added Errormator and Sentry support part of pull request #71

author	Marcin Kuzminski <marcin@python-works.com>
date	Thu, 18 Oct 2012 23:54:46 +0200
parents	25b10f9d23b4
children	f7a52d548fd0 578a152f5ae3

rev	line source
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	1 # -- coding: utf-8 --
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	2 """
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	3 rhodecode.lib.celerylib.tasks
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	4 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	5
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	6 RhodeCode task modules, containing all task that suppose to be run
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	7 by celery daemon
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	8
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	9 :created_on: Oct 6, 2010
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	10 :author: marcink
1824 89efedac4e6c 2012 copyrights Marcin Kuzminski <marcin@python-works.com> parents: 1818 diff changeset	11 :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com>
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	12 :license: GPLv3, see COPYING for more details.
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	13 """
1206 a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	14 # This program is free software: you can redistribute it and/or modify
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	15 # it under the terms of the GNU General Public License as published by
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	16 # the Free Software Foundation, either version 3 of the License, or
a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	17 # (at your option) any later version.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	18 #
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	19 # This program is distributed in the hope that it will be useful,
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	20 # but WITHOUT ANY WARRANTY; without even the implied warranty of
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	21 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	22 # GNU General Public License for more details.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	23 #
903 04c9bb9ca6d6 code docs, updates Marcin Kuzminski <marcin@python-works.com> parents: 854 diff changeset	24 # You should have received a copy of the GNU General Public License
1206 a671db5bdd58 fixed license issue #149 Marcin Kuzminski <marcin@python-works.com> parents: 1203 diff changeset	25 # along with this program. If not, see <http://www.gnu.org/licenses/>.
467 3fc3ce53659b starting celery branch Marcin Kuzminski <marcin@python-works.com> parents: diff changeset	26 from celery.decorators import task
555 03676d39dd0a added fault tolerant case when celeryconfig is not present in the directory. Marcin Kuzminski <marcin@python-works.com> parents: 549 diff changeset	27
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	28 import os
cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	29 import traceback
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	30 import logging
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	31 from os.path import join as jn
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	32
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	33 from time import mktime
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	34 from operator import itemgetter
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	35 from string import lower
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	36
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	37 from pylons import config, url
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	38 from pylons.i18n.translation import _
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	39
2007 324ac367a4da Added VCS into rhodecode core for faster and easier deployments of new versions Marcin Kuzminski <marcin@python-works.com> parents: 1976 diff changeset	40 from rhodecode.lib.vcs import get_backend
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	41
2299 e2dbdaf13562 Don't clear dbsessions when celery_eager is turned on Marcin Kuzminski <marcin@python-works.com> parents: 2185 diff changeset	42 from rhodecode import CELERY_ON, CELERY_EAGER
2109 8ecfed1d8f8b utils/conf Marcin Kuzminski <marcin@python-works.com> parents: 2007 diff changeset	43 from rhodecode.lib.utils2 import safe_str
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	44 from rhodecode.lib.celerylib import run_task, locked_task, dbsession, \
cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	45 str2bool, __get_lockkey, LockHeld, DaemonLock, get_session
547 1e757ac98988 renamed project to rhodecode Marcin Kuzminski <marcin@python-works.com> parents: 535 diff changeset	46 from rhodecode.lib.helpers import person
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	47 from rhodecode.lib.rcmail.smtp_mailer import SmtpMailer
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	48 from rhodecode.lib.utils import add_cache, action_logger
1514 87ec80c280bb fixed issues with python2.5 Marcin Kuzminski <marcin@python-works.com> parents: 1417 diff changeset	49 from rhodecode.lib.compat import json, OrderedDict
2185 eac0d6194783 fixed repo_create hooks for forks Marcin Kuzminski <marcin@python-works.com> parents: 2154 diff changeset	50 from rhodecode.lib.hooks import log_create_repository
1514 87ec80c280bb fixed issues with python2.5 Marcin Kuzminski <marcin@python-works.com> parents: 1417 diff changeset	51
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	52 from rhodecode.model.db import Statistics, Repository, User
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	53
692 cb0d9ce6ac5c #50 on point cache invalidation changes. Marcin Kuzminski <marcin@python-works.com> parents: 691 diff changeset	54
777 aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	55 add_cache(config)
aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	56
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	57 __all__ = ['whoosh_index', 'get_commits_stats',
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	58 'reset_user_password', 'send_email']
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	59
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	60
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	61 def get_logger(cls):
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	62 if CELERY_ON:
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	63 try:
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	64 log = cls.get_logger()
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	65 except:
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	66 log = logging.getLogger(__name__)
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	67 else:
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	68 log = logging.getLogger(__name__)
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	69
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	70 return log
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	71
1807 1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	72
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	73 @task(ignore_result=True)
504 d280aa1c85c6 removed pidlock from whoosh and added it as locked_task decorator Marcin Kuzminski <marcin@python-works.com> parents: 502 diff changeset	74 @locked_task
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	75 @dbsession
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	76 def whoosh_index(repo_location, full_index):
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	77 from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
2344 70126ffbd575 fixed issue #459. Changed the way of obtaining logger in reindex task. Marcin Kuzminski <marcin@python-works.com> parents: 2299 diff changeset	78 log = get_logger(whoosh_index)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	79 DBS = get_session()
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	80
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	81 index_location = config['index_dir']
662 373ee7031003 fixed annotation bug, added history to annotation. Marcin Kuzminski <marcin@python-works.com> parents: 659 diff changeset	82 WhooshIndexingDaemon(index_location=index_location,
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	83 repo_location=repo_location, sa=DBS)\
777 aac24db58ce8 fixed cache problem, Marcin Kuzminski <marcin@python-works.com> parents: 776 diff changeset	84 .run(full_index=full_index)
497 fb0c3af6031b Implemented locking for task, to prevent for running the same tasks, Marcin Kuzminski <marcin@python-works.com> parents: 493 diff changeset	85
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	86
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	87 @task(ignore_result=True)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	88 @dbsession
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	89 def get_commits_stats(repo_name, ts_min_y, ts_max_y):
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	90 log = get_logger(get_commits_stats)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	91 DBS = get_session()
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	92 lockkey = __get_lockkey('get_commits_stats', repo_name, ts_min_y,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	93 ts_max_y)
1540 191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	94 lockkey_path = config['here']
191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	95
1976 a76e9bacbedc garden Marcin Kuzminski <marcin@python-works.com> parents: 1944 diff changeset	96 log.info('running task with lockkey %s' % lockkey)
1807 1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	97
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	98 try:
1540 191f3f08236d fixes #258 RhodeCode 1.2 assumes egg folder is writable Marcin Kuzminski <marcin@python-works.com> parents: 1514 diff changeset	99 lock = l = DaemonLock(file_=jn(lockkey_path, lockkey))
699 52da7cba88a6 Code refactor for auth func, preparing for ldap support Marcin Kuzminski <marcin@python-works.com> parents: 692 diff changeset	100
1976 a76e9bacbedc garden Marcin Kuzminski <marcin@python-works.com> parents: 1944 diff changeset	101 # for js data compatibility cleans the key for person from '
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	102 akc = lambda k: person(k).replace('"', "")
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	103
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	104 co_day_auth_aggr = {}
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	105 commits_by_day_aggregate = {}
1749 8ecc6b8229a5 commit less models Marcin Kuzminski <marcin@python-works.com> parents: 1747 diff changeset	106 repo = Repository.get_by_repo_name(repo_name)
8ecc6b8229a5 commit less models Marcin Kuzminski <marcin@python-works.com> parents: 1747 diff changeset	107 if repo is None:
8ecc6b8229a5 commit less models Marcin Kuzminski <marcin@python-works.com> parents: 1747 diff changeset	108 return True
1818 cf51bbfb120e auto white-space removal Marcin Kuzminski <marcin@python-works.com> parents: 1807 diff changeset	109
1749 8ecc6b8229a5 commit less models Marcin Kuzminski <marcin@python-works.com> parents: 1747 diff changeset	110 repo = repo.scm_instance
1807 1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	111 repo_size = repo.count()
1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	112 # return if repo have no revisions
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	113 if repo_size < 1:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	114 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	115 return True
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	116
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	117 skip_date_limit = True
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	118 parse_limit = int(config['app_conf'].get('commit_parse_limit'))
1807 1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	119 last_rev = None
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	120 last_cs = None
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	121 timegetter = itemgetter('time')
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	122
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	123 dbrepo = DBS.query(Repository)\
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	124 .filter(Repository.repo_name == repo_name).scalar()
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	125 cur_stats = DBS.query(Statistics)\
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	126 .filter(Statistics.repository == dbrepo).scalar()
1105 61c5f0ab24b9 Added branch filter to repo pager Marcin Kuzminski <marcin@python-works.com> parents: 1076 diff changeset	127
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	128 if cur_stats is not None:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	129 last_rev = cur_stats.stat_on_revision
1105 61c5f0ab24b9 Added branch filter to repo pager Marcin Kuzminski <marcin@python-works.com> parents: 1076 diff changeset	130
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	131 if last_rev == repo.get_changeset().revision and repo_size > 1:
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	132 # pass silently without any work if we're not on first revision or
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	133 # current state of parsing revision(from db marker) is the
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	134 # last revision
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	135 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	136 return True
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	137
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	138 if cur_stats:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	139 commits_by_day_aggregate = OrderedDict(json.loads(
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	140 cur_stats.commit_activity_combined))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	141 co_day_auth_aggr = json.loads(cur_stats.commit_activity)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	142
1976 a76e9bacbedc garden Marcin Kuzminski <marcin@python-works.com> parents: 1944 diff changeset	143 log.debug('starting parsing %s' % parse_limit)
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	144 lmktime = mktime
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	145
1807 1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	146 last_rev = last_rev + 1 if last_rev >= 0 else 0
1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	147 log.debug('Getting revisions from %s to %s' % (
1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	148 last_rev, last_rev + parse_limit)
1635a21485d6 - fixed issue with missing commits on some repos commands Marcin Kuzminski <marcin@python-works.com> parents: 1749 diff changeset	149 )
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	150 for cs in repo[last_rev:last_rev + parse_limit]:
2109 8ecfed1d8f8b utils/conf Marcin Kuzminski <marcin@python-works.com> parents: 2007 diff changeset	151 log.debug('parsing %s' % cs)
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	152 last_cs = cs # remember last parsed changeset
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	153 k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	154 cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0])
1076 db71228a4c23 moved statistics parse_limit into .ini files Marcin Kuzminski <marcin@python-works.com> parents: 1002 diff changeset	155
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	156 if akc(cs.author) in co_day_auth_aggr:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	157 try:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	158 l = [timegetter(x) for x in
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	159 co_day_auth_aggr[akc(cs.author)]['data']]
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	160 time_pos = l.index(k)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	161 except ValueError:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	162 time_pos = False
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	163
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	164 if time_pos >= 0 and time_pos is not False:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	165
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	166 datadict = \
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	167 co_day_auth_aggr[akc(cs.author)]['data'][time_pos]
804 b92d9a0321e0 code stats speed improvments Marcin Kuzminski <marcin@python-works.com> parents: 803 diff changeset	168
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	169 datadict["commits"] += 1
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	170 datadict["added"] += len(cs.added)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	171 datadict["changed"] += len(cs.changed)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	172 datadict["removed"] += len(cs.removed)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	173
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	174 else:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	175 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	176
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	177 datadict = {"time": k,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	178 "commits": 1,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	179 "added": len(cs.added),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	180 "changed": len(cs.changed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	181 "removed": len(cs.removed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	182 }
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	183 co_day_auth_aggr[akc(cs.author)]['data']\
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	184 .append(datadict)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	185
467 3fc3ce53659b starting celery branch Marcin Kuzminski <marcin@python-works.com> parents: diff changeset	186 else:
487 b12ea84fb906 Some fixes to summary, and total rewrite of summary graphs implemented more interactive graph. Marcin Kuzminski <marcin@python-works.com> parents: 486 diff changeset	187 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	188 co_day_auth_aggr[akc(cs.author)] = {
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	189 "label": akc(cs.author),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	190 "data": [{"time":k,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	191 "commits":1,
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	192 "added":len(cs.added),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	193 "changed":len(cs.changed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	194 "removed":len(cs.removed),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	195 }],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	196 "schema": ["commits"],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	197 }
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	198
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	199 #gather all data by day
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	200 if k in commits_by_day_aggregate:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	201 commits_by_day_aggregate[k] += 1
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	202 else:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	203 commits_by_day_aggregate[k] = 1
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	204
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	205 overview_data = sorted(commits_by_day_aggregate.items(),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	206 key=itemgetter(0))
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	207
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	208 if not co_day_auth_aggr:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	209 co_day_auth_aggr[akc(repo.contact)] = {
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	210 "label": akc(repo.contact),
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	211 "data": [0, 1],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	212 "schema": ["commits"],
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	213 }
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	214
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	215 stats = cur_stats if cur_stats else Statistics()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	216 stats.commit_activity = json.dumps(co_day_auth_aggr)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	217 stats.commit_activity_combined = json.dumps(overview_data)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	218
1976 a76e9bacbedc garden Marcin Kuzminski <marcin@python-works.com> parents: 1944 diff changeset	219 log.debug('last revison %s' % last_rev)
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	220 leftovers = len(repo.revisions[last_rev:])
1976 a76e9bacbedc garden Marcin Kuzminski <marcin@python-works.com> parents: 1944 diff changeset	221 log.debug('revisions to parse %s' % leftovers)
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	222
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	223 if last_rev == 0 or leftovers < parse_limit:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	224 log.debug('getting code trending stats')
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	225 stats.languages = json.dumps(__get_codes_stats(repo_name))
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	226
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	227 try:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	228 stats.repository = dbrepo
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	229 stats.stat_on_revision = last_cs.revision if last_cs else 0
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	230 DBS.add(stats)
cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	231 DBS.commit()
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	232 except:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	233 log.error(traceback.format_exc())
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	234 DBS.rollback()
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	235 lock.release()
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	236 return False
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	237
2109 8ecfed1d8f8b utils/conf Marcin Kuzminski <marcin@python-works.com> parents: 2007 diff changeset	238 # final release
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	239 lock.release()
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	240
2109 8ecfed1d8f8b utils/conf Marcin Kuzminski <marcin@python-works.com> parents: 2007 diff changeset	241 # execute another task if celery is enabled
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	242 if len(repo.revisions) > 1 and CELERY_ON:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	243 run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y)
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	244 return True
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	245 except LockHeld:
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	246 log.info('LockHeld')
0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	247 return 'Task with key %s already running' % lockkey
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	248
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	249 @task(ignore_result=True)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	250 @dbsession
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	251 def send_password_link(user_email):
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	252 from rhodecode.model.notification import EmailNotificationModel
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	253
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	254 log = get_logger(send_password_link)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	255 DBS = get_session()
1944 5fc9c92025c1 white space cleanup Marcin Kuzminski <marcin@python-works.com> parents: 1941 diff changeset	256
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	257 try:
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	258 user = User.get_by_email(user_email)
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	259 if user:
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	260 log.debug('password reset user found %s' % user)
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	261 link = url('reset_password_confirmation', key=user.api_key,
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	262 qualified=True)
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	263 reg_type = EmailNotificationModel.TYPE_PASSWORD_RESET
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	264 body = EmailNotificationModel().get_email_tmpl(reg_type,
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	265 **{'user':user.short_contact,
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	266 'reset_url':link})
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	267 log.debug('sending email')
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	268 run_task(send_email, user_email,
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	269 _("password reset link"), body)
1976 a76e9bacbedc garden Marcin Kuzminski <marcin@python-works.com> parents: 1944 diff changeset	270 log.info('send new password mail to %s' % user_email)
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	271 else:
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	272 log.debug("password reset email %s not found" % user_email)
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	273 except:
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	274 log.error(traceback.format_exc())
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	275 return False
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	276
5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	277 return True
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	278
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	279 @task(ignore_result=True)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	280 @dbsession
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	281 def reset_user_password(user_email):
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	282 from rhodecode.lib import auth
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	283
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	284 log = get_logger(reset_user_password)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	285 DBS = get_session()
1944 5fc9c92025c1 white space cleanup Marcin Kuzminski <marcin@python-works.com> parents: 1941 diff changeset	286
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	287 try:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	288 try:
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	289 user = User.get_by_email(user_email)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	290 new_passwd = auth.PasswordGenerator().gen_password(8,
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	291 auth.PasswordGenerator.ALPHABETS_BIG_SMALL)
493 2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	292 if user:
2256c78afe53 implemented basic autoupdating statistics fetched from database Marcin Kuzminski <marcin@python-works.com> parents: 487 diff changeset	293 user.password = auth.get_crypt_password(new_passwd)
1116 716911af91e1 Added api_key into user, api key get's generated again after password change Marcin Kuzminski <marcin@python-works.com> parents: 1105 diff changeset	294 user.api_key = auth.generate_api_key(user.username)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	295 DBS.add(user)
cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	296 DBS.commit()
1976 a76e9bacbedc garden Marcin Kuzminski <marcin@python-works.com> parents: 1944 diff changeset	297 log.info('change password for %s' % user_email)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	298 if new_passwd is None:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	299 raise Exception('unable to generate new password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	300 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	301 log.error(traceback.format_exc())
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	302 DBS.rollback()
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	303
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	304 run_task(send_email, user_email,
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	305 'Your new password',
1417 5875955def39 fixes #223 improve password reset form Marcin Kuzminski <marcin@python-works.com> parents: 1401 diff changeset	306 'Your new RhodeCode password:%s' % (new_passwd))
1976 a76e9bacbedc garden Marcin Kuzminski <marcin@python-works.com> parents: 1944 diff changeset	307 log.info('send new password mail to %s' % user_email)
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	308
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	309 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	310 log.error('Failed to update user password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	311 log.error(traceback.format_exc())
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	312
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	313 return True
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	314
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	315
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	316 @task(ignore_result=True)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	317 @dbsession
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	318 def send_email(recipients, subject, body, html_body=''):
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	319 """
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	320 Sends an email with defined parameters from the .ini files.
1203 6832ef664673 source code cleanup: remove trailing white space, normalize file endings Marcin Kuzminski <marcin@python-works.com> parents: 1169 diff changeset	321
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	322 :param recipients: list of recipients, it this is empty the defined email
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	323 address from field 'email_to' is used instead
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	324 :param subject: subject of the mail
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	325 :param body: body of the mail
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	326 :param html_body: html version of body
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	327 """
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	328 log = get_logger(send_email)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	329 DBS = get_session()
1944 5fc9c92025c1 white space cleanup Marcin Kuzminski <marcin@python-works.com> parents: 1941 diff changeset	330
776 f6c613fba757 Celery is configured by the .ini files and run from paster now Marcin Kuzminski <marcin@python-works.com> parents: 752 diff changeset	331 email_config = config
2154 747c1c7040a8 fixed email prefix sent as None Marcin Kuzminski <marcin@python-works.com> parents: 2109 diff changeset	332 subject = "%s %s" % (email_config.get('email_prefix', ''), subject)
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	333 if not recipients:
1612 5cb398de935e implements #291 email notification sent to all admin users Marcin Kuzminski <marcin@python-works.com> parents: 1581 diff changeset	334 # if recipients are not defined we send to email_config + all admins
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	335 admins = [u.email for u in User.query()
7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	336 .filter(User.admin == True).all()]
1612 5cb398de935e implements #291 email notification sent to all admin users Marcin Kuzminski <marcin@python-works.com> parents: 1581 diff changeset	337 recipients = [email_config.get('email_to')] + admins
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	338
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	339 mail_from = email_config.get('app_email_from', 'RhodeCode')
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	340 user = email_config.get('smtp_username')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	341 passwd = email_config.get('smtp_password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	342 mail_server = email_config.get('smtp_server')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	343 mail_port = email_config.get('smtp_port')
689 ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	344 tls = str2bool(email_config.get('smtp_use_tls'))
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer Marcin Kuzminski <marcin@python-works.com> parents: 685 diff changeset	345 ssl = str2bool(email_config.get('smtp_use_ssl'))
1169 f6dca275c5a8 control mailer debug with the .ini file Marcin Kuzminski <marcin@python-works.com> parents: 1116 diff changeset	346 debug = str2bool(config.get('debug'))
1581 67377fd685be applied smth_auth options update patch Les Peabody <lpeabody@ccom.unh.edu> parents: 1540 diff changeset	347 smtp_auth = email_config.get('smtp_auth')
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	348
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	349 try:
1673 333b3e7c991f Added email sending test site Marcin Kuzminski <marcin@python-works.com> parents: 1612 diff changeset	350 m = SmtpMailer(mail_from, user, passwd, mail_server, smtp_auth,
1169 f6dca275c5a8 control mailer debug with the .ini file Marcin Kuzminski <marcin@python-works.com> parents: 1116 diff changeset	351 mail_port, ssl, tls, debug=debug)
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	352 m.send(recipients, subject, body, html_body)
474 a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	353 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	354 log.error('Mail sending failed')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	355 log.error(traceback.format_exc())
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	356 return False
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks. Marcin Kuzminski <marcin@python-works.com> parents: 467 diff changeset	357 return True
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	358
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	359
1002 3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results) Marcin Kuzminski <marcin@python-works.com> parents: 903 diff changeset	360 @task(ignore_result=True)
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	361 @dbsession
530 a08f610e545e Implemented server side forks Marcin Kuzminski <marcin@python-works.com> parents: 509 diff changeset	362 def create_repo_fork(form_data, cur_user):
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	363 """
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	364 Creates a fork of repository using interval VCS methods
1818 cf51bbfb120e auto white-space removal Marcin Kuzminski <marcin@python-works.com> parents: 1807 diff changeset	365
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	366 :param form_data:
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	367 :param cur_user:
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	368 """
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	369 from rhodecode.model.repo import RepoModel
2652 532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	370 from rhodecode.model.user import UserModel
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	371
1717 7ff304d3028f Notification fixes Marcin Kuzminski <marcin@python-works.com> parents: 1713 diff changeset	372 log = get_logger(create_repo_fork)
1941 58c684bacdad fix leftover bad code on fork function Marcin Kuzminski <marcin@python-works.com> parents: 1929 diff changeset	373 DBS = get_session()
58c684bacdad fix leftover bad code on fork function Marcin Kuzminski <marcin@python-works.com> parents: 1929 diff changeset	374
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	375 base_path = Repository.base_path()
2652 532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	376 cur_user = UserModel(DBS)._get_user(cur_user)
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	377
1730 ce0b47534c36 fixed fork journal entry Marcin Kuzminski <marcin@python-works.com> parents: 1728 diff changeset	378 fork_name = form_data['repo_name_full']
2652 532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	379 repo_type = form_data['repo_type']
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	380 description = form_data['description']
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	381 owner = cur_user
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	382 private = form_data['private']
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	383 clone_uri = form_data.get('clone_uri')
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	384 repos_group = form_data['repo_group']
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	385 landing_rev = form_data['landing_rev']
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	386 copy_fork_permissions = form_data.get('copy_permissions')
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	387 fork_of = RepoModel(DBS)._get_repo(form_data.get('fork_parent_id'))
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	388
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	389 fork_repo = RepoModel(DBS).create_repo(
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	390 fork_name, repo_type, description, owner, private, clone_uri,
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	391 repos_group, landing_rev, just_db=True, fork_of=fork_of,
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	392 copy_fork_permissions=copy_fork_permissions
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	393 )
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	394
1742 40c4f7351574 added option to do a checkout after cloning a repository Marcin Kuzminski <marcin@python-works.com> parents: 1730 diff changeset	395 update_after_clone = form_data['update_after_clone']
2652 532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	396
532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	397 source_repo_path = os.path.join(base_path, fork_of.repo_name)
1730 ce0b47534c36 fixed fork journal entry Marcin Kuzminski <marcin@python-works.com> parents: 1728 diff changeset	398 destination_fork_path = os.path.join(base_path, fork_name)
1264 0c43c6671815 moved locking of commit stats into the task itself to remove race conditions when lock was not removed before starting another task. Marcin Kuzminski <marcin@python-works.com> parents: 1244 diff changeset	399
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	400 log.info('creating fork of %s as %s', source_repo_path,
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	401 destination_fork_path)
2652 532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	402 backend = get_backend(repo_type)
2813 25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	403
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	404 if repo_type == 'git':
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	405 backend(safe_str(destination_fork_path), create=True,
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	406 src_url=safe_str(source_repo_path),
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	407 update_after_clone=update_after_clone,
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	408 bare=True)
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	409 elif repo_type == 'hg':
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	410 backend(safe_str(destination_fork_path), create=True,
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	411 src_url=safe_str(source_repo_path),
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	412 update_after_clone=update_after_clone)
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	413 else:
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	414 raise Exception('Unknown backend type %s' % repo_type)
25b10f9d23b4 Fixed mercurial backend doesn't take bare parameter when forking Marcin Kuzminski <marcin@python-works.com> parents: 2807 diff changeset	415
2185 eac0d6194783 fixed repo_create hooks for forks Marcin Kuzminski <marcin@python-works.com> parents: 2154 diff changeset	416 log_create_repository(fork_repo.get_dict(), created_by=cur_user.username)
eac0d6194783 fixed repo_create hooks for forks Marcin Kuzminski <marcin@python-works.com> parents: 2154 diff changeset	417
1730 ce0b47534c36 fixed fork journal entry Marcin Kuzminski <marcin@python-works.com> parents: 1728 diff changeset	418 action_logger(cur_user, 'user_forked_repo:%s' % fork_name,
2652 532ee86cf9b1 Refactored create fork function to use new RepoModel functions instead of old Marcin Kuzminski <marcin@python-works.com> parents: 2524 diff changeset	419 fork_of.repo_name, '', DBS)
1818 cf51bbfb120e auto white-space removal Marcin Kuzminski <marcin@python-works.com> parents: 1807 diff changeset	420
1747 8804715404c9 implements #193 journal stores information about deleting of repos Marcin Kuzminski <marcin@python-works.com> parents: 1742 diff changeset	421 action_logger(cur_user, 'user_created_fork:%s' % fork_name,
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	422 fork_name, '', DBS)
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	423 # finally commit at latest possible stage
1929 cd8a7e3698bc fixes #340 session cleanup for celery tasks Marcin Kuzminski <marcin@python-works.com> parents: 1824 diff changeset	424 DBS.commit()
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	425
2185 eac0d6194783 fixed repo_create hooks for forks Marcin Kuzminski <marcin@python-works.com> parents: 2154 diff changeset	426
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	427 def __get_codes_stats(repo_name):
2109 8ecfed1d8f8b utils/conf Marcin Kuzminski <marcin@python-works.com> parents: 2007 diff changeset	428 from rhodecode.config.conf import LANGUAGES_EXTENSIONS_MAP
1722 e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	429 repo = Repository.get_by_repo_name(repo_name).scm_instance
e7eef7a1db6a #235 forking page repo group selection Marcin Kuzminski <marcin@python-works.com> parents: 1717 diff changeset	430
603 95a502d94860 removed soon deprecated walk method on repository instance Marcin Kuzminski <marcin@python-works.com> parents: 564 diff changeset	431 tip = repo.get_changeset()
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	432 code_stats = {}
630 a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	433
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	434 def aggregate(cs):
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	435 for f in cs[2]:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	436 ext = lower(f.extension)
789 9fec2207f91c removed binary files from trending sources Marcin Kuzminski <marcin@python-works.com> parents: 785 diff changeset	437 if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	438 if ext in code_stats:
0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	439 code_stats[ext] += 1
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	440 else:
1244 0eceb478c720 fixed issue #165 trending source files are now stored in cache as ext only, and translated to description only when displaying, so future changes of mappings will take affect right away. Marcin Kuzminski <marcin@python-works.com> parents: 1206 diff changeset	441 code_stats[ext] = 1
629 7e536d1af60d Code refactoring,models renames Marcin Kuzminski <marcin@python-works.com> parents: 603 diff changeset	442
630 a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	443 map(aggregate, tip.walk('/'))
a9e72f7148c8 some small fixes Marcin Kuzminski <marcin@python-works.com> parents: 629 diff changeset	444
506 d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions! Marcin Kuzminski <marcin@python-works.com> parents: 504 diff changeset	445 return code_stats or {}

Mercurial > kallithea

annotate rhodecode/lib/celerylib/tasks.py @ 2939:dbe3cfb81446 beta