Mercurial > kallithea
annotate rhodecode/lib/celerypylons/commands.py @ 1728:07e56179633e beta
- fixes celery sqlalchemy session issues for async forking
- summary page css fixes
- speed optimizations
author | Marcin Kuzminski <marcin@python-works.com> |
---|---|
date | Sat, 26 Nov 2011 17:51:03 +0200 |
parents | fe8c2e881403 |
children | 8ecfed1d8f8b |
rev | line source |
---|---|
1728
07e56179633e
- fixes celery sqlalchemy session issues for async forking
Marcin Kuzminski <marcin@python-works.com>
parents:
1726
diff
changeset
|
1 import rhodecode |
785
277427ac29a9
complete rewrite of paster commands,
Marcin Kuzminski <marcin@python-works.com>
parents:
776
diff
changeset
|
2 from rhodecode.lib.utils import BasePasterCommand, Command |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
3 from celery.app import app_or_default |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
4 from celery.bin import camqadm, celerybeat, celeryd, celeryev |
1726
fe8c2e881403
moved pylons config out of global scope for celerypylons
Marcin Kuzminski <marcin@python-works.com>
parents:
1672
diff
changeset
|
5 |
1672
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
6 from rhodecode.lib import str2bool |
776
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
7 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
8 __all__ = ['CeleryDaemonCommand', 'CeleryBeatCommand', |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
9 'CAMQPAdminCommand', 'CeleryEventCommand'] |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
10 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
11 |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
12 class CeleryCommand(BasePasterCommand): |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
13 """Abstract class implements run methods needed for celery |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
14 |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
15 Starts the celery worker that uses a paste.deploy configuration |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
16 file. |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
17 """ |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
18 |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
19 def update_parser(self): |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
20 """ |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
21 Abstract method. Allows for the class's parser to be updated |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
22 before the superclass's `run` method is called. Necessary to |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
23 allow options/arguments to be passed through to the underlying |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
24 celery command. |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
25 """ |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
26 |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
27 cmd = self.celery_command(app_or_default()) |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
28 for x in cmd.get_options(): |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
29 self.parser.add_option(x) |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
30 |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
31 def command(self): |
1726
fe8c2e881403
moved pylons config out of global scope for celerypylons
Marcin Kuzminski <marcin@python-works.com>
parents:
1672
diff
changeset
|
32 from pylons import config |
1672
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
33 try: |
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
34 CELERY_ON = str2bool(config['app_conf'].get('use_celery')) |
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
35 except KeyError: |
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
36 CELERY_ON = False |
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
37 |
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
38 if CELERY_ON == False: |
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
39 raise Exception('Please enable celery_on in .ini config ' |
13d6da57b0cf
Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents:
1002
diff
changeset
|
40 'file before running celeryd') |
1728
07e56179633e
- fixes celery sqlalchemy session issues for async forking
Marcin Kuzminski <marcin@python-works.com>
parents:
1726
diff
changeset
|
41 rhodecode.CELERY_ON = CELERY_ON |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
42 cmd = self.celery_command(app_or_default()) |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
43 return cmd.run(**vars(self.options)) |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
44 |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
45 class CeleryDaemonCommand(CeleryCommand): |
776
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
46 """Start the celery worker |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
47 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
48 Starts the celery worker that uses a paste.deploy configuration |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
49 file. |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
50 """ |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
51 usage = 'CONFIG_FILE [celeryd options...]' |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
52 summary = __doc__.splitlines()[0] |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
53 description = "".join(__doc__.splitlines()[2:]) |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
54 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
55 parser = Command.standard_parser(quiet=True) |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
56 celery_command = celeryd.WorkerCommand |
776
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
57 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
58 |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
59 class CeleryBeatCommand(CeleryCommand): |
776
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
60 """Start the celery beat server |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
61 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
62 Starts the celery beat server using a paste.deploy configuration |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
63 file. |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
64 """ |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
65 usage = 'CONFIG_FILE [celerybeat options...]' |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
66 summary = __doc__.splitlines()[0] |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
67 description = "".join(__doc__.splitlines()[2:]) |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
68 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
69 parser = Command.standard_parser(quiet=True) |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
70 celery_command = celerybeat.BeatCommand |
776
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
71 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
72 |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
73 class CAMQPAdminCommand(CeleryCommand): |
776
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
74 """CAMQP Admin |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
75 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
76 CAMQP celery admin tool. |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
77 """ |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
78 usage = 'CONFIG_FILE [camqadm options...]' |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
79 summary = __doc__.splitlines()[0] |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
80 description = "".join(__doc__.splitlines()[2:]) |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
81 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
82 parser = Command.standard_parser(quiet=True) |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
83 celery_command = camqadm.AMQPAdminCommand |
776
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
84 |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
85 class CeleryEventCommand(CeleryCommand): |
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
86 """Celery event command. |
776
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
87 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
88 Capture celery events. |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
89 """ |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
90 usage = 'CONFIG_FILE [celeryev options...]' |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
91 summary = __doc__.splitlines()[0] |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
92 description = "".join(__doc__.splitlines()[2:]) |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
93 |
f6c613fba757
Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
94 parser = Command.standard_parser(quiet=True) |
1002
3a7f5b1a19dd
made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents:
785
diff
changeset
|
95 celery_command = celeryev.EvCommand |