Mercurial > kallithea

annotate rhodecode/lib/celerylib/tasks.py @ 1002:3a7f5b1a19dd beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
made rhodecode work with celery 2.2, made some tasks optimizations(forget results) added celeryconfig.py with just the definitions of hosts, it seams just this is needed to get celery working nice, all other config options are taken from .ini files. This is a temp workaround until i get the proper soltuion to this problem.
author Marcin Kuzminski <marcin@python-works.com>
date Tue, 08 Feb 2011 02:57:21 +0100
parents 04c9bb9ca6d6
children db71228a4c23
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
903
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
1 # -*- coding: utf-8 -*-
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
2 """
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
3 rhodecode.lib.celerylib.tasks
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
4 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
903
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
5
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
6 RhodeCode task modules, containing all task that suppose to be run
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
7 by celery daemon
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
8
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
9 :created_on: Oct 6, 2010
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
10 :author: marcink
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
11 :copyright: (C) 2009-2011 Marcin Kuzminski <marcin@python-works.com>
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
12 :license: GPLv3, see COPYING for more details.
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
13 """
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
14 # This program is free software; you can redistribute it and/or
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
15 # modify it under the terms of the GNU General Public License
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
16 # as published by the Free Software Foundation; version 2
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
17 # of the License or (at your opinion) any later version of the license.
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
18 #
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
19 # This program is distributed in the hope that it will be useful,
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
20 # but WITHOUT ANY WARRANTY; without even the implied warranty of
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
21 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
22 # GNU General Public License for more details.
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
23 #
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
24 # You should have received a copy of the GNU General Public License
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
25 # along with this program; if not, write to the Free Software
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
26 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
04c9bb9ca6d6 code docs, updates
Marcin Kuzminski <marcin@python-works.com>
parents: 854
diff changeset
27 # MA 02110-1301, USA.
467
3fc3ce53659b starting celery branch
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
28 from celery.decorators import task
555
03676d39dd0a added fault tolerant case when celeryconfig is not present in the directory.
Marcin Kuzminski <marcin@python-works.com>
parents: 549
diff changeset
29
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
30 import os
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
31 import traceback
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
32 import logging
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
33
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
34 from time import mktime
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
35 from operator import itemgetter
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
36
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
37 from pylons import config
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
38 from pylons.i18n.translation import _
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
39
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
40 from rhodecode.lib.celerylib import run_task, locked_task, str2bool
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
41 from rhodecode.lib.helpers import person
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
42 from rhodecode.lib.smtp_mailer import SmtpMailer
785
277427ac29a9 complete rewrite of paster commands,
Marcin Kuzminski <marcin@python-works.com>
parents: 777
diff changeset
43 from rhodecode.lib.utils import OrderedDict, add_cache
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
44 from rhodecode.model import init_model
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
45 from rhodecode.model import meta
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
46 from rhodecode.model.db import RhodeCodeUi
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
47
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 630
diff changeset
48 from vcs.backends import get_repo
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
49
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
50 from sqlalchemy import engine_from_config
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
51
777
aac24db58ce8 fixed cache problem,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
52 add_cache(config)
aac24db58ce8 fixed cache problem,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
53
555
03676d39dd0a added fault tolerant case when celeryconfig is not present in the directory.
Marcin Kuzminski <marcin@python-works.com>
parents: 549
diff changeset
54 try:
564
ba7e24cd4786 refactor codes and setup for python 2.5
Marcin Kuzminski <marcin@python-works.com>
parents: 562
diff changeset
55 import json
ba7e24cd4786 refactor codes and setup for python 2.5
Marcin Kuzminski <marcin@python-works.com>
parents: 562
diff changeset
56 except ImportError:
ba7e24cd4786 refactor codes and setup for python 2.5
Marcin Kuzminski <marcin@python-works.com>
parents: 562
diff changeset
57 #python 2.5 compatibility
ba7e24cd4786 refactor codes and setup for python 2.5
Marcin Kuzminski <marcin@python-works.com>
parents: 562
diff changeset
58 import simplejson as json
ba7e24cd4786 refactor codes and setup for python 2.5
Marcin Kuzminski <marcin@python-works.com>
parents: 562
diff changeset
59
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
60 __all__ = ['whoosh_index', 'get_commits_stats',
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
61 'reset_user_password', 'send_email']
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
62
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
63 CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
64
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
65 def get_session():
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
66 if CELERY_ON:
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
67 engine = engine_from_config(config, 'sqlalchemy.db1.')
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
68 init_model(engine)
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
69 sa = meta.Session()
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
70 return sa
467
3fc3ce53659b starting celery branch
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
71
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
72 def get_repos_path():
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
73 sa = get_session()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
74 q = sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
75 return q.ui_value
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
76
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
77 @task(ignore_result=True)
504
d280aa1c85c6 removed pidlock from whoosh and added it as locked_task decorator
Marcin Kuzminski <marcin@python-works.com>
parents: 502
diff changeset
78 @locked_task
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
79 def whoosh_index(repo_location, full_index):
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
80 #log = whoosh_index.get_logger()
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
81 from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
82 index_location = config['index_dir']
662
373ee7031003 fixed annotation bug, added history to annotation.
Marcin Kuzminski <marcin@python-works.com>
parents: 659
diff changeset
83 WhooshIndexingDaemon(index_location=index_location,
777
aac24db58ce8 fixed cache problem,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
84 repo_location=repo_location, sa=get_session())\
aac24db58ce8 fixed cache problem,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
85 .run(full_index=full_index)
497
fb0c3af6031b Implemented locking for task, to prevent for running the same tasks,
Marcin Kuzminski <marcin@python-works.com>
parents: 493
diff changeset
86
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
87 @task(ignore_result=True)
502
ac32a026c306 simplified task locking, and fixed some bugs for keyworded arguments
Marcin Kuzminski <marcin@python-works.com>
parents: 497
diff changeset
88 @locked_task
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
89 def get_commits_stats(repo_name, ts_min_y, ts_max_y):
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
90 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
91 log = get_commits_stats.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
92 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
93 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
94
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
95 from rhodecode.model.db import Statistics, Repository
699
52da7cba88a6 Code refactor for auth func, preparing for ldap support
Marcin Kuzminski <marcin@python-works.com>
parents: 692
diff changeset
96
52da7cba88a6 Code refactor for auth func, preparing for ldap support
Marcin Kuzminski <marcin@python-works.com>
parents: 692
diff changeset
97 #for js data compatibilty
52da7cba88a6 Code refactor for auth func, preparing for ldap support
Marcin Kuzminski <marcin@python-works.com>
parents: 692
diff changeset
98 author_key_cleaner = lambda k: person(k).replace('"', "")
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
99
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
100 commits_by_day_author_aggregate = {}
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
101 commits_by_day_aggregate = {}
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
102 repos_path = get_repos_path()
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 630
diff changeset
103 p = os.path.join(repos_path, repo_name)
635
fd63782c4426 Fixed age, for new vcs implementation. Removed all obsolete date formatters
Marcin Kuzminski <marcin@python-works.com>
parents: 631
diff changeset
104 repo = get_repo(p)
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
105
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
106 skip_date_limit = True
630
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
107 parse_limit = 250 #limit for single task changeset parsing optimal for
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
108 last_rev = 0
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
109 last_cs = None
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
110 timegetter = itemgetter('time')
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
111
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
112 sa = get_session()
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
113
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
114 dbrepo = sa.query(Repository)\
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
115 .filter(Repository.repo_name == repo_name).scalar()
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
116 cur_stats = sa.query(Statistics)\
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
117 .filter(Statistics.repository == dbrepo).scalar()
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
118 if cur_stats:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
119 last_rev = cur_stats.stat_on_revision
509
b50e79b4257a fixes to pidlock, to not raise unneded execptions
Marcin Kuzminski <marcin@python-works.com>
parents: 506
diff changeset
120 if not repo.revisions:
b50e79b4257a fixes to pidlock, to not raise unneded execptions
Marcin Kuzminski <marcin@python-works.com>
parents: 506
diff changeset
121 return True
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
122
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
123 if last_rev == repo.revisions[-1] and len(repo.revisions) > 1:
699
52da7cba88a6 Code refactor for auth func, preparing for ldap support
Marcin Kuzminski <marcin@python-works.com>
parents: 692
diff changeset
124 #pass silently without any work if we're not on first revision or
52da7cba88a6 Code refactor for auth func, preparing for ldap support
Marcin Kuzminski <marcin@python-works.com>
parents: 692
diff changeset
125 #current state of parsing revision(from db marker) is the last revision
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
126 return True
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
127
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
128 if cur_stats:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
129 commits_by_day_aggregate = OrderedDict(
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
130 json.loads(
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
131 cur_stats.commit_activity_combined))
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
132 commits_by_day_author_aggregate = json.loads(cur_stats.commit_activity)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
133
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
134 log.debug('starting parsing %s', parse_limit)
630
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
135 lmktime = mktime
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
136
803
317ba4c84160 some small fixes to stats function
Marcin Kuzminski <marcin@python-works.com>
parents: 789
diff changeset
137 last_rev = last_rev + 1 if last_rev > 0 else last_rev
317ba4c84160 some small fixes to stats function
Marcin Kuzminski <marcin@python-works.com>
parents: 789
diff changeset
138 for rev in repo.revisions[last_rev:last_rev + parse_limit]:
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
139 last_cs = cs = repo.get_changeset(rev)
804
b92d9a0321e0 code stats speed improvments
Marcin Kuzminski <marcin@python-works.com>
parents: 803
diff changeset
140 k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
b92d9a0321e0 code stats speed improvments
Marcin Kuzminski <marcin@python-works.com>
parents: 803
diff changeset
141 cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0])
b92d9a0321e0 code stats speed improvments
Marcin Kuzminski <marcin@python-works.com>
parents: 803
diff changeset
142
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
143 if commits_by_day_author_aggregate.has_key(author_key_cleaner(cs.author)):
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
144 try:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
145 l = [timegetter(x) for x in commits_by_day_author_aggregate\
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
146 [author_key_cleaner(cs.author)]['data']]
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
147 time_pos = l.index(k)
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
148 except ValueError:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
149 time_pos = False
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
150
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
151 if time_pos >= 0 and time_pos is not False:
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
152
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
153 datadict = commits_by_day_author_aggregate\
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
154 [author_key_cleaner(cs.author)]['data'][time_pos]
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
155
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
156 datadict["commits"] += 1
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
157 datadict["added"] += len(cs.added)
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
158 datadict["changed"] += len(cs.changed)
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
159 datadict["removed"] += len(cs.removed)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
160
467
3fc3ce53659b starting celery branch
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
161 else:
487
b12ea84fb906 Some fixes to summary, and total rewrite of summary graphs implemented more interactive graph.
Marcin Kuzminski <marcin@python-works.com>
parents: 486
diff changeset
162 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
163
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
164 datadict = {"time":k,
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
165 "commits":1,
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
166 "added":len(cs.added),
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
167 "changed":len(cs.changed),
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
168 "removed":len(cs.removed),
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
169 }
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
170 commits_by_day_author_aggregate\
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
171 [author_key_cleaner(cs.author)]['data'].append(datadict)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
172
467
3fc3ce53659b starting celery branch
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
173 else:
487
b12ea84fb906 Some fixes to summary, and total rewrite of summary graphs implemented more interactive graph.
Marcin Kuzminski <marcin@python-works.com>
parents: 486
diff changeset
174 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
175 commits_by_day_author_aggregate[author_key_cleaner(cs.author)] = {
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
176 "label":author_key_cleaner(cs.author),
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
177 "data":[{"time":k,
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
178 "commits":1,
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
179 "added":len(cs.added),
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
180 "changed":len(cs.changed),
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
181 "removed":len(cs.removed),
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
182 }],
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
183 "schema":["commits"],
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
184 }
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
185
562
1a77a0c350f5 extended size of some input fields
Marcin Kuzminski <marcin@python-works.com>
parents: 559
diff changeset
186 #gather all data by day
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
187 if commits_by_day_aggregate.has_key(k):
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
188 commits_by_day_aggregate[k] += 1
486
5c376ac2d4c9 rewrote graph plotting, added zooming and json dump insted of stupid string formating.
Marcin Kuzminski <marcin@python-works.com>
parents: 483
diff changeset
189 else:
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
190 commits_by_day_aggregate[k] = 1
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
191
804
b92d9a0321e0 code stats speed improvments
Marcin Kuzminski <marcin@python-works.com>
parents: 803
diff changeset
192 overview_data = sorted(commits_by_day_aggregate.items(), key=itemgetter(0))
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
193 if not commits_by_day_author_aggregate:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
194 commits_by_day_author_aggregate[author_key_cleaner(repo.contact)] = {
486
5c376ac2d4c9 rewrote graph plotting, added zooming and json dump insted of stupid string formating.
Marcin Kuzminski <marcin@python-works.com>
parents: 483
diff changeset
195 "label":author_key_cleaner(repo.contact),
5c376ac2d4c9 rewrote graph plotting, added zooming and json dump insted of stupid string formating.
Marcin Kuzminski <marcin@python-works.com>
parents: 483
diff changeset
196 "data":[0, 1],
5c376ac2d4c9 rewrote graph plotting, added zooming and json dump insted of stupid string formating.
Marcin Kuzminski <marcin@python-works.com>
parents: 483
diff changeset
197 "schema":["commits"],
5c376ac2d4c9 rewrote graph plotting, added zooming and json dump insted of stupid string formating.
Marcin Kuzminski <marcin@python-works.com>
parents: 483
diff changeset
198 }
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
199
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
200 stats = cur_stats if cur_stats else Statistics()
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
201 stats.commit_activity = json.dumps(commits_by_day_author_aggregate)
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
202 stats.commit_activity_combined = json.dumps(overview_data)
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
203
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
204 log.debug('last revison %s', last_rev)
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
205 leftovers = len(repo.revisions[last_rev:])
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
206 log.debug('revisions to parse %s', leftovers)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
207
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
208 if last_rev == 0 or leftovers < parse_limit:
854
25116ab9f0bc added more logs into code stats
Marcin Kuzminski <marcin@python-works.com>
parents: 804
diff changeset
209 log.debug('getting code trending stats')
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
210 stats.languages = json.dumps(__get_codes_stats(repo_name))
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
211
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
212 stats.repository = dbrepo
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
213 stats.stat_on_revision = last_cs.revision
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
214
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
215 try:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
216 sa.add(stats)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
217 sa.commit()
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
218 except:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
219 log.error(traceback.format_exc())
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
220 sa.rollback()
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
221 return False
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
222 if len(repo.revisions) > 1:
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
223 run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
224
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
225 return True
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
226
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
227 @task(ignore_result=True)
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
228 def reset_user_password(user_email):
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
229 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
230 log = reset_user_password.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
231 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
232 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
233
547
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
234 from rhodecode.lib import auth
1e757ac98988 renamed project to rhodecode
Marcin Kuzminski <marcin@python-works.com>
parents: 535
diff changeset
235 from rhodecode.model.db import User
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
236
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
237 try:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
238 try:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
239 sa = get_session()
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
240 user = sa.query(User).filter(User.email == user_email).scalar()
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
241 new_passwd = auth.PasswordGenerator().gen_password(8,
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
242 auth.PasswordGenerator.ALPHABETS_BIG_SMALL)
493
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
243 if user:
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
244 user.password = auth.get_crypt_password(new_passwd)
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
245 sa.add(user)
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
246 sa.commit()
2256c78afe53 implemented basic autoupdating statistics fetched from database
Marcin Kuzminski <marcin@python-works.com>
parents: 487
diff changeset
247 log.info('change password for %s', user_email)
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
248 if new_passwd is None:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
249 raise Exception('unable to generate new password')
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
250
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
251 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
252 log.error(traceback.format_exc())
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
253 sa.rollback()
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
254
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
255 run_task(send_email, user_email,
549
f99075170eb4 more renames for rhode code !!
Marcin Kuzminski <marcin@python-works.com>
parents: 548
diff changeset
256 "Your new rhodecode password",
f99075170eb4 more renames for rhode code !!
Marcin Kuzminski <marcin@python-works.com>
parents: 548
diff changeset
257 'Your new rhodecode password:%s' % (new_passwd))
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
258 log.info('send new password mail to %s', user_email)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
259
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
260
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
261 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
262 log.error('Failed to update user password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
263 log.error(traceback.format_exc())
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
264
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
265 return True
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
266
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
267 @task(ignore_result=True)
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
268 def send_email(recipients, subject, body):
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
269 """
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
270 Sends an email with defined parameters from the .ini files.
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
271
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
272
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
273 :param recipients: list of recipients, it this is empty the defined email
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
274 address from field 'email_to' is used instead
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
275 :param subject: subject of the mail
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
276 :param body: body of the mail
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
277 """
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
278 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
279 log = send_email.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
280 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
281 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
282
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
283 email_config = config
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
284
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
285 if not recipients:
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
286 recipients = [email_config.get('email_to')]
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
287
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
288 mail_from = email_config.get('app_email_from')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
289 user = email_config.get('smtp_username')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
290 passwd = email_config.get('smtp_password')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
291 mail_server = email_config.get('smtp_server')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
292 mail_port = email_config.get('smtp_port')
689
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
293 tls = str2bool(email_config.get('smtp_use_tls'))
ecc566f8b69f fixes #59, notifications for user registrations + some changes to mailer
Marcin Kuzminski <marcin@python-works.com>
parents: 685
diff changeset
294 ssl = str2bool(email_config.get('smtp_use_ssl'))
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
295
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
296 try:
483
a9e50dce3081 Removed config names from whoosh and celery,
Marcin Kuzminski <marcin@python-works.com>
parents: 474
diff changeset
297 m = SmtpMailer(mail_from, user, passwd, mail_server,
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
298 mail_port, ssl, tls)
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
299 m.send(recipients, subject, body)
474
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
300 except:
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
301 log.error('Mail sending failed')
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
302 log.error(traceback.format_exc())
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
303 return False
a3d9d24acbec Implemented password reset(forms/models/ tasks) and mailing tasks.
Marcin Kuzminski <marcin@python-works.com>
parents: 467
diff changeset
304 return True
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
305
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
306 @task(ignore_result=True)
530
a08f610e545e Implemented server side forks
Marcin Kuzminski <marcin@python-works.com>
parents: 509
diff changeset
307 def create_repo_fork(form_data, cur_user):
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
308 try:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
309 log = create_repo_fork.get_logger()
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
310 except:
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
311 log = logging.getLogger(__name__)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
312
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
313 from rhodecode.model.repo import RepoModel
659
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type.
Marcin Kuzminski <marcin@python-works.com>
parents: 635
diff changeset
314 from vcs import get_backend
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 903
diff changeset
315
752
89b9037d68b7 fixed Example celery config to ampq,
Marcin Kuzminski <marcin@python-works.com>
parents: 722
diff changeset
316 repo_model = RepoModel(get_session())
630
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
317 repo_model.create(form_data, cur_user, just_db=True, fork=True)
659
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type.
Marcin Kuzminski <marcin@python-works.com>
parents: 635
diff changeset
318 repo_name = form_data['repo_name']
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
319 repos_path = get_repos_path()
659
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type.
Marcin Kuzminski <marcin@python-works.com>
parents: 635
diff changeset
320 repo_path = os.path.join(repos_path, repo_name)
530
a08f610e545e Implemented server side forks
Marcin Kuzminski <marcin@python-works.com>
parents: 509
diff changeset
321 repo_fork_path = os.path.join(repos_path, form_data['fork_name'])
659
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type.
Marcin Kuzminski <marcin@python-works.com>
parents: 635
diff changeset
322 alias = form_data['repo_type']
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
323
659
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type.
Marcin Kuzminski <marcin@python-works.com>
parents: 635
diff changeset
324 log.info('creating repo fork %s as %s', repo_name, repo_path)
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type.
Marcin Kuzminski <marcin@python-works.com>
parents: 635
diff changeset
325 backend = get_backend(alias)
758f64f3fbda extended repo creation by repo type. fixed fork creation to maintain repo type.
Marcin Kuzminski <marcin@python-works.com>
parents: 635
diff changeset
326 backend(str(repo_fork_path), create=True, src_url=str(repo_path))
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
327
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
328 def __get_codes_stats(repo_name):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
329 LANGUAGES_EXTENSIONS_MAP = {'scm': 'Scheme', 'asmx': 'VbNetAspx', 'Rout':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
330 'RConsole', 'rest': 'Rst', 'abap': 'ABAP', 'go': 'Go', 'phtml': 'HtmlPhp',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
331 'ns2': 'Newspeak', 'xml': 'EvoqueXml', 'sh-session': 'BashSession', 'ads':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
332 'Ada', 'clj': 'Clojure', 'll': 'Llvm', 'ebuild': 'Bash', 'adb': 'Ada',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
333 'ada': 'Ada', 'c++-objdump': 'CppObjdump', 'aspx':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
334 'VbNetAspx', 'ksh': 'Bash', 'coffee': 'CoffeeScript', 'vert': 'GLShader',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
335 'Makefile.*': 'Makefile', 'di': 'D', 'dpatch': 'DarcsPatch', 'rake':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
336 'Ruby', 'moo': 'MOOCode', 'erl-sh': 'ErlangShell', 'geo': 'GLShader',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
337 'pov': 'Povray', 'bas': 'VbNet', 'bat': 'Batch', 'd': 'D', 'lisp':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
338 'CommonLisp', 'h': 'C', 'rbx': 'Ruby', 'tcl': 'Tcl', 'c++': 'Cpp', 'md':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
339 'MiniD', '.vimrc': 'Vim', 'xsd': 'Xml', 'ml': 'Ocaml', 'el': 'CommonLisp',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
340 'befunge': 'Befunge', 'xsl': 'Xslt', 'pyx': 'Cython', 'cfm':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
341 'ColdfusionHtml', 'evoque': 'Evoque', 'cfg': 'Ini', 'htm': 'Html',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
342 'Makefile': 'Makefile', 'cfc': 'ColdfusionHtml', 'tex': 'Tex', 'cs':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
343 'CSharp', 'mxml': 'Mxml', 'patch': 'Diff', 'apache.conf': 'ApacheConf',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
344 'scala': 'Scala', 'applescript': 'AppleScript', 'GNUmakefile': 'Makefile',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
345 'c-objdump': 'CObjdump', 'lua': 'Lua', 'apache2.conf': 'ApacheConf', 'rb':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
346 'Ruby', 'gemspec': 'Ruby', 'rl': 'RagelObjectiveC', 'vala': 'Vala', 'tmpl':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
347 'Cheetah', 'bf': 'Brainfuck', 'plt': 'Gnuplot', 'G': 'AntlrRuby', 'xslt':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
348 'Xslt', 'flxh': 'Felix', 'asax': 'VbNetAspx', 'Rakefile': 'Ruby', 'S': 'S',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
349 'wsdl': 'Xml', 'js': 'Javascript', 'autodelegate': 'Myghty', 'properties':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
350 'Ini', 'bash': 'Bash', 'c': 'C', 'g': 'AntlrRuby', 'r3': 'Rebol', 's':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
351 'Gas', 'ashx': 'VbNetAspx', 'cxx': 'Cpp', 'boo': 'Boo', 'prolog': 'Prolog',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
352 'sqlite3-console': 'SqliteConsole', 'cl': 'CommonLisp', 'cc': 'Cpp', 'pot':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
353 'Gettext', 'vim': 'Vim', 'pxi': 'Cython', 'yaml': 'Yaml', 'SConstruct':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
354 'Python', 'diff': 'Diff', 'txt': 'Text', 'cw': 'Redcode', 'pxd': 'Cython',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
355 'plot': 'Gnuplot', 'java': 'Java', 'hrl': 'Erlang', 'py': 'Python',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
356 'makefile': 'Makefile', 'squid.conf': 'SquidConf', 'asm': 'Nasm', 'toc':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
357 'Tex', 'kid': 'Genshi', 'rhtml': 'Rhtml', 'po': 'Gettext', 'pl': 'Prolog',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
358 'pm': 'Perl', 'hx': 'Haxe', 'ascx': 'VbNetAspx', 'ooc': 'Ooc', 'asy':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
359 'Asymptote', 'hs': 'Haskell', 'SConscript': 'Python', 'pytb':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
360 'PythonTraceback', 'myt': 'Myghty', 'hh': 'Cpp', 'R': 'S', 'aux': 'Tex',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
361 'rst': 'Rst', 'cpp-objdump': 'CppObjdump', 'lgt': 'Logtalk', 'rss': 'Xml',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
362 'flx': 'Felix', 'b': 'Brainfuck', 'f': 'Fortran', 'rbw': 'Ruby',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
363 '.htaccess': 'ApacheConf', 'cxx-objdump': 'CppObjdump', 'j': 'ObjectiveJ',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
364 'mll': 'Ocaml', 'yml': 'Yaml', 'mu': 'MuPAD', 'r': 'Rebol', 'ASM': 'Nasm',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
365 'erl': 'Erlang', 'mly': 'Ocaml', 'mo': 'Modelica', 'def': 'Modula2', 'ini':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
366 'Ini', 'control': 'DebianControl', 'vb': 'VbNet', 'vapi': 'Vala', 'pro':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
367 'Prolog', 'spt': 'Cheetah', 'mli': 'Ocaml', 'as': 'ActionScript3', 'cmd':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
368 'Batch', 'cpp': 'Cpp', 'io': 'Io', 'tac': 'Python', 'haml': 'Haml', 'rkt':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
369 'Racket', 'st':'Smalltalk', 'inc': 'Povray', 'pas': 'Delphi', 'cmake':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
370 'CMake', 'csh':'Tcsh', 'hpp': 'Cpp', 'feature': 'Gherkin', 'html': 'Html',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
371 'php':'Php', 'php3':'Php', 'php4':'Php', 'php5':'Php', 'xhtml': 'Html',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
372 'hxx': 'Cpp', 'eclass': 'Bash', 'css': 'Css',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
373 'frag': 'GLShader', 'd-objdump': 'DObjdump', 'weechatlog': 'IrcLogs',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
374 'tcsh': 'Tcsh', 'objdump': 'Objdump', 'pyw': 'Python', 'h++': 'Cpp',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
375 'py3tb': 'Python3Traceback', 'jsp': 'Jsp', 'sql': 'Sql', 'mak': 'Makefile',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
376 'php': 'Php', 'mao': 'Mako', 'man': 'Groff', 'dylan': 'Dylan', 'sass':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
377 'Sass', 'cfml': 'ColdfusionHtml', 'darcspatch': 'DarcsPatch', 'tpl':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
378 'Smarty', 'm': 'ObjectiveC', 'f90': 'Fortran', 'mod': 'Modula2', 'sh':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
379 'Bash', 'lhs': 'LiterateHaskell', 'sources.list': 'SourcesList', 'axd':
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
380 'VbNetAspx', 'sc': 'Python'}
630
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
381
692
cb0d9ce6ac5c #50 on point cache invalidation changes.
Marcin Kuzminski <marcin@python-works.com>
parents: 691
diff changeset
382 repos_path = get_repos_path()
631
05528ad948c4 Hacking for git support,and new faster repo scan
Marcin Kuzminski <marcin@python-works.com>
parents: 630
diff changeset
383 p = os.path.join(repos_path, repo_name)
635
fd63782c4426 Fixed age, for new vcs implementation. Removed all obsolete date formatters
Marcin Kuzminski <marcin@python-works.com>
parents: 631
diff changeset
384 repo = get_repo(p)
603
95a502d94860 removed soon deprecated walk method on repository instance
Marcin Kuzminski <marcin@python-works.com>
parents: 564
diff changeset
385 tip = repo.get_changeset()
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
386 code_stats = {}
630
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
387
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
388 def aggregate(cs):
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
389 for f in cs[2]:
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
390 ext = f.extension
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
391 key = LANGUAGES_EXTENSIONS_MAP.get(ext, ext)
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
392 key = key or ext
789
9fec2207f91c removed binary files from trending sources
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
393 if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary:
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
394 if code_stats.has_key(key):
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
395 code_stats[key] += 1
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
396 else:
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents: 752
diff changeset
397 code_stats[key] = 1
629
7e536d1af60d Code refactoring,models renames
Marcin Kuzminski <marcin@python-works.com>
parents: 603
diff changeset
398
630
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
399 map(aggregate, tip.walk('/'))
a9e72f7148c8 some small fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 629
diff changeset
400
506
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
401 return code_stats or {}
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
402
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
403
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
404
d5efb83590ef fixed lock decorator bug which didn't release the lock after func execution and rewrote the pidlock a little with Ask Solem suggestions!
Marcin Kuzminski <marcin@python-works.com>
parents: 504
diff changeset
405