Mercurial > kallithea

annotate rhodecode/lib/celerypylons/commands.py @ 1726:fe8c2e881403 beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
moved pylons config out of global scope for celerypylons
author Marcin Kuzminski <marcin@python-works.com>
date Sat, 26 Nov 2011 02:23:28 +0200
parents 13d6da57b0cf
children 07e56179633e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
785
277427ac29a9 complete rewrite of paster commands,
Marcin Kuzminski <marcin@python-works.com>
parents: 776
diff changeset
1 from rhodecode.lib.utils import BasePasterCommand, Command
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
2 from celery.app import app_or_default
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
3 from celery.bin import camqadm, celerybeat, celeryd, celeryev
1726
fe8c2e881403 moved pylons config out of global scope for celerypylons
Marcin Kuzminski <marcin@python-works.com>
parents: 1672
diff changeset
4
1672
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
5 from rhodecode.lib import str2bool
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
6
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
7 __all__ = ['CeleryDaemonCommand', 'CeleryBeatCommand',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
8 'CAMQPAdminCommand', 'CeleryEventCommand']
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
9
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
10
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
11 class CeleryCommand(BasePasterCommand):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
12 """Abstract class implements run methods needed for celery
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
13
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
14 Starts the celery worker that uses a paste.deploy configuration
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
15 file.
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
16 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
17
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
18 def update_parser(self):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
19 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
20 Abstract method. Allows for the class's parser to be updated
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
21 before the superclass's `run` method is called. Necessary to
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
22 allow options/arguments to be passed through to the underlying
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
23 celery command.
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
24 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
25
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
26 cmd = self.celery_command(app_or_default())
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
27 for x in cmd.get_options():
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
28 self.parser.add_option(x)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
29
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
30 def command(self):
1726
fe8c2e881403 moved pylons config out of global scope for celerypylons
Marcin Kuzminski <marcin@python-works.com>
parents: 1672
diff changeset
31 from pylons import config
1672
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
32 try:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
33 CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
34 except KeyError:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
35 CELERY_ON = False
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
36
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
37 if CELERY_ON == False:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
38 raise Exception('Please enable celery_on in .ini config '
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
39 'file before running celeryd')
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
40
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
41 cmd = self.celery_command(app_or_default())
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
42 return cmd.run(**vars(self.options))
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
43
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
44 class CeleryDaemonCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
45 """Start the celery worker
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
46
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
47 Starts the celery worker that uses a paste.deploy configuration
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
48 file.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
49 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
50 usage = 'CONFIG_FILE [celeryd options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
51 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
52 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
53
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
54 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
55 celery_command = celeryd.WorkerCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
56
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
57
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
58 class CeleryBeatCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
59 """Start the celery beat server
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
60
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
61 Starts the celery beat server using a paste.deploy configuration
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
62 file.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
63 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
64 usage = 'CONFIG_FILE [celerybeat options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
65 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
66 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
67
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
68 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
69 celery_command = celerybeat.BeatCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
70
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
71
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
72 class CAMQPAdminCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
73 """CAMQP Admin
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
74
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
75 CAMQP celery admin tool.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
76 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
77 usage = 'CONFIG_FILE [camqadm options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
78 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
79 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
80
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
81 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
82 celery_command = camqadm.AMQPAdminCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
83
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
84 class CeleryEventCommand(CeleryCommand):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
85 """Celery event command.
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
86
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
87 Capture celery events.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
88 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
89 usage = 'CONFIG_FILE [celeryev options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
90 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
91 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
92
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
93 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
94 celery_command = celeryev.EvCommand