Mercurial > kallithea

annotate rhodecode/lib/celerypylons/commands.py @ 3172:264d9c930c17 beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added some more logging into get_container_username function
author Marcin Kuzminski <marcin@python-works.com>
date Sat, 19 Jan 2013 16:28:18 +0100
parents 8ecfed1d8f8b
children 7aa0ff5b4e09
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1728
07e56179633e - fixes celery sqlalchemy session issues for async forking
Marcin Kuzminski <marcin@python-works.com>
parents: 1726
diff changeset
1 import rhodecode
2109
8ecfed1d8f8b utils/conf
Marcin Kuzminski <marcin@python-works.com>
parents: 1728
diff changeset
2 from rhodecode.lib.utils import BasePasterCommand, Command, load_rcextensions
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
3 from celery.app import app_or_default
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
4 from celery.bin import camqadm, celerybeat, celeryd, celeryev
1726
fe8c2e881403 moved pylons config out of global scope for celerypylons
Marcin Kuzminski <marcin@python-works.com>
parents: 1672
diff changeset
5
2109
8ecfed1d8f8b utils/conf
Marcin Kuzminski <marcin@python-works.com>
parents: 1728
diff changeset
6 from rhodecode.lib.utils2 import str2bool
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
7
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
8 __all__ = ['CeleryDaemonCommand', 'CeleryBeatCommand',
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
9 'CAMQPAdminCommand', 'CeleryEventCommand']
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
10
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
11
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
12 class CeleryCommand(BasePasterCommand):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
13 """Abstract class implements run methods needed for celery
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
14
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
15 Starts the celery worker that uses a paste.deploy configuration
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
16 file.
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
17 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
18
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
19 def update_parser(self):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
20 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
21 Abstract method. Allows for the class's parser to be updated
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
22 before the superclass's `run` method is called. Necessary to
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
23 allow options/arguments to be passed through to the underlying
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
24 celery command.
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
25 """
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
26
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
27 cmd = self.celery_command(app_or_default())
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
28 for x in cmd.get_options():
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
29 self.parser.add_option(x)
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
30
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
31 def command(self):
1726
fe8c2e881403 moved pylons config out of global scope for celerypylons
Marcin Kuzminski <marcin@python-works.com>
parents: 1672
diff changeset
32 from pylons import config
1672
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
33 try:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
34 CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
35 except KeyError:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
36 CELERY_ON = False
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
37
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
38 if CELERY_ON == False:
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
39 raise Exception('Please enable celery_on in .ini config '
13d6da57b0cf Added explicit error message about running celeryd without enable_celery flag set to true.
Marcin Kuzminski <marcin@python-works.com>
parents: 1002
diff changeset
40 'file before running celeryd')
1728
07e56179633e - fixes celery sqlalchemy session issues for async forking
Marcin Kuzminski <marcin@python-works.com>
parents: 1726
diff changeset
41 rhodecode.CELERY_ON = CELERY_ON
2109
8ecfed1d8f8b utils/conf
Marcin Kuzminski <marcin@python-works.com>
parents: 1728
diff changeset
42 load_rcextensions(config['here'])
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
43 cmd = self.celery_command(app_or_default())
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
44 return cmd.run(**vars(self.options))
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
45
2109
8ecfed1d8f8b utils/conf
Marcin Kuzminski <marcin@python-works.com>
parents: 1728
diff changeset
46
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
47 class CeleryDaemonCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
48 """Start the celery worker
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
49
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
50 Starts the celery worker that uses a paste.deploy configuration
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
51 file.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
52 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
53 usage = 'CONFIG_FILE [celeryd options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
54 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
55 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
56
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
57 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
58 celery_command = celeryd.WorkerCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
59
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
60
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
61 class CeleryBeatCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
62 """Start the celery beat server
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
63
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
64 Starts the celery beat server using a paste.deploy configuration
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
65 file.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
66 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
67 usage = 'CONFIG_FILE [celerybeat options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
68 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
69 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
70
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
71 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
72 celery_command = celerybeat.BeatCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
73
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
74
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
75 class CAMQPAdminCommand(CeleryCommand):
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
76 """CAMQP Admin
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
77
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
78 CAMQP celery admin tool.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
79 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
80 usage = 'CONFIG_FILE [camqadm options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
81 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
82 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
83
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
84 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
85 celery_command = camqadm.AMQPAdminCommand
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
86
2109
8ecfed1d8f8b utils/conf
Marcin Kuzminski <marcin@python-works.com>
parents: 1728
diff changeset
87
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
88 class CeleryEventCommand(CeleryCommand):
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
89 """Celery event command.
776
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
90
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
91 Capture celery events.
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
92 """
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
93 usage = 'CONFIG_FILE [celeryev options...]'
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
94 summary = __doc__.splitlines()[0]
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
95 description = "".join(__doc__.splitlines()[2:])
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
96
f6c613fba757 Celery is configured by the .ini files and run from paster now
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
97 parser = Command.standard_parser(quiet=True)
1002
3a7f5b1a19dd made rhodecode work with celery 2.2, made some tasks optimizations(forget results)
Marcin Kuzminski <marcin@python-works.com>
parents: 785
diff changeset
98 celery_command = celeryev.EvCommand