Mercurial > kallithea

annotate rhodecode/lib/celerypylons/commands.py @ 2105:926f55b038bc beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added initial rc-extension module - possible to store additional mappings for stats - possible to overwrite and add to whoosh index extensions issue #322 - post create repo hook callback - post push/pull hooks callback
author Marcin Kuzminski <marcin@python-works.com>
date Tue, 06 Mar 2012 23:03:10 +0200
parents 07e56179633e
children 8ecfed1d8f8b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1728
07e56179633e - fixes celery sqlalchemy session issues for async forking
Marcin Kuzminski <marcin@python-works.com>
parents: 1726
diff changeset
1 import rhodecode
785
277427ac29a9 complete rewrite of paster commands,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
2 from rhodecode.lib.utils import BasePasterCommand, Command
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
3 from celery.app import app_or_default
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
4 from celery.bin import camqadm, celerybeat, celeryd, celeryev
1726
fe8c2e881403 moved pylons config out of global scope for celerypylons
Marcin Kuzminski <marcin@python-works.com>
parents: 1672
diff changeset
5
1672
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
6 from rhodecode.lib import str2bool
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
7
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
8 __all__ = ['CeleryDaemonCommand', 'CeleryBeatCommand',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
9 'CAMQPAdminCommand', 'CeleryEventCommand']
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
10
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
11
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
12 class CeleryCommand(BasePasterCommand):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
13 """Abstract class implements run methods needed for celery
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
14
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
15 Starts the celery worker that uses a paste.deploy configuration
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
16 file.
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
17 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
18
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
19 def update_parser(self):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
20 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
21 Abstract method. Allows for the class's parser to be updated
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
22 before the superclass's `run` method is called. Necessary to
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
23 allow options/arguments to be passed through to the underlying
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
24 celery command.
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
25 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
26
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
27 cmd = self.celery_command(app_or_default())
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
28 for x in cmd.get_options():
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
29 self.parser.add_option(x)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
30
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
31 def command(self):
1726
fe8c2e881403 moved pylons config out of global scope for celerypylons
Marcin Kuzminski <marcin@python-works.com>
parents: 1672
diff changeset
32 from pylons import config
1672
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
33 try:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
34 CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
35 except KeyError:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
36 CELERY_ON = False
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
37
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
38 if CELERY_ON == False:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
39 raise Exception('Please enable celery_on in .ini config '
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
40 'file before running celeryd')
1728
07e56179633e - fixes celery sqlalchemy session issues for async forking
Marcin Kuzminski <marcin@python-works.com>
parents: 1726
diff changeset
41 rhodecode.CELERY_ON = CELERY_ON
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
42 cmd = self.celery_command(app_or_default())
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
43 return cmd.run(**vars(self.options))
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
44
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
45 class CeleryDaemonCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
46 """Start the celery worker
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
47
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
48 Starts the celery worker that uses a paste.deploy configuration
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
49 file.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
50 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
51 usage = 'CONFIG_FILE [celeryd options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
52 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
53 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
54
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
55 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
56 celery_command = celeryd.WorkerCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
57
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
58
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
59 class CeleryBeatCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
60 """Start the celery beat server
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
61
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
62 Starts the celery beat server using a paste.deploy configuration
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
63 file.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
64 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
65 usage = 'CONFIG_FILE [celerybeat options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
66 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
67 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
68
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
69 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
70 celery_command = celerybeat.BeatCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
71
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
72
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
73 class CAMQPAdminCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
74 """CAMQP Admin
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
75
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
76 CAMQP celery admin tool.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
77 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
78 usage = 'CONFIG_FILE [camqadm options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
79 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
80 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
81
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
82 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
83 celery_command = camqadm.AMQPAdminCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
84
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
85 class CeleryEventCommand(CeleryCommand):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
86 """Celery event command.
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
87
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
88 Capture celery events.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
89 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
90 usage = 'CONFIG_FILE [celeryev options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
91 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
92 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
93
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
94 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
95 celery_command = celeryev.EvCommand