Mercurial > kallithea

annotate docs/usage/performance.rst @ 4955:4e6dfdb3fa01

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
docs: English and consistency corrections
author Michael V. DePalatis <mike@depalatis.net>
date Tue, 31 Mar 2015 22:15:38 +0200
parents e73a69cb98dc
children 778f7ae3b6eb
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
1 .. _performance:
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
2
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
3 ================================
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
4 Optimizing Kallithea Performance
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
5 ================================
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
6
4955
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
7 When serving a large amount of big repositories, Kallithea can start
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
8 performing slower than expected. Because of the demanding nature of handling large
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
9 amounts of data from version control systems, here are some tips on how to get
2775
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
10 the best performance.
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
11
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
12 * Kallithea will perform better on machines with faster disks (SSD/SAN). It's
4955
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
13 more important to have a faster disk than a faster CPU.
2775
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
14
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
15 * Slowness on initial page can be easily fixed by grouping repositories, and/or
4955
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
16 increasing cache size (see below). This includes using the lightweight dashboard
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
17 option and ``vcs_full_cache`` setting in .ini file
2775
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
18
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
19
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
20 Follow these few steps to improve performance of Kallithea system.
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
21
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
22
2680
3b179f1ac3a2 doc fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 2517
diff changeset
23 1. Increase cache
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
24
4955
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
25 In the .ini file::
3224
8b8edfc25856 whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 2775
diff changeset
26
2680
3b179f1ac3a2 doc fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 2517
diff changeset
27 beaker.cache.sql_cache_long.expire=3600 <-- set this to higher number
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
28
4955
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
29 This option affects the cache expiration time for the main
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
30 page. Having several hundreds of repositories on main page can
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
31 sometimes make the system behave slowly when the cache expires for
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
32 all of them. Increasing the ``expire`` option to a day (86400) or a
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
33 week (604800) will improve general response times for the main
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
34 page. Kallithea has an intelligent cache expiration system and it
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
35 will expire the cache for repositories that have been changed.
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
36
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
37 2. Switch from sqlite to postgres or mysql
3224
8b8edfc25856 whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 2775
diff changeset
38
4955
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
39 sqlite is a good option when having a small load on the system. But due to
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
40 locking issues with sqlite, it is not recommended to use it for larger
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
41 deployments. Switching to mysql or postgres will result in an immediate
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
42 performance increase.
3224
8b8edfc25856 whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 2775
diff changeset
43
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
44 3. Scale Kallithea horizontally
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
45
4955
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
46 Scaling horizontally can give huge performance increases when dealing with
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
47 large traffic (large amount of users, CI servers etc). Kallithea can be
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
48 scaled horizontally on one (recommended) or multiple machines. In order
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
49 to scale horizontally you need to do the following:
3413
d79f3505549e whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 3390
diff changeset
50
4955
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
51 - Each instance needs its own .ini file and unique ``instance_id`` set.
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
52 - Each instance's ``data`` storage needs to be configured to be stored on a
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
53 shared disk storage, preferably together with repositories. This ``data``
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
54 dir contains template caches, sessions, whoosh index and is used for
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
55 task locking (so it is safe across multiple instances). Set the
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
56 ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir``
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
57 variables in each .ini file to a shared location across Kallithea instances
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
58 - If celery is used each instance should run a separate Celery instance, but
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
59 the message broker should be common to all of them (e.g., one
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
60 shared RabbitMQ server)
4e6dfdb3fa01 docs: English and consistency corrections
Michael V. DePalatis <mike@depalatis.net>
parents: 4192
diff changeset
61 - Load balance using round robin or IP hash, recommended is writing LB rules
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
62 that will separate regular user traffic from automated processes like CI
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
63 servers or build bots.