Mercurial > kallithea

annotate docs/usage/performance.rst @ 4554:2dad9708c89f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
paster: add install-iis command to automate IIS handler generation A new paster command, install-iis, is added that automates generating the ISAPI-WSGI file that allows IIS to serve up Kallithea's WSGI application using IIS' ISAPI filters. The paster command's output also describes the final steps necessary to complete IIS installation.
author Henrik Stuart <hg@hstuart.dk>
date Sun, 14 Sep 2014 07:39:06 +0200
parents e73a69cb98dc
children 4e6dfdb3fa01
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
1 .. _performance:
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
2
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
3 ================================
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
4 Optimizing Kallithea Performance
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
5 ================================
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
6
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
7 When serving large amount of big repositories Kallithea can start
2775
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
8 performing slower than expected. Because of demanding nature of handling large
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
9 amount of data from version control systems here are some tips how to get
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
10 the best performance.
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
11
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
12 * Kallithea will perform better on machines with faster disks (SSD/SAN). It's
2775
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
13 more important to have faster disk than faster CPU.
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
14
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
15 * Slowness on initial page can be easily fixed by grouping repositories, and/or
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
16 increasing cache size (see below), that includes using lightweight dashboard
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
17 option and vcs_full_cache setting in .ini file
2775
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
18
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
19
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
20 Follow these few steps to improve performance of Kallithea system.
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
21
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
22
2680
3b179f1ac3a2 doc fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 2517
diff changeset
23 1. Increase cache
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
24
2680
3b179f1ac3a2 doc fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 2517
diff changeset
25 in the .ini file::
3224
8b8edfc25856 whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 2775
diff changeset
26
2680
3b179f1ac3a2 doc fixes
Marcin Kuzminski <marcin@python-works.com>
parents: 2517
diff changeset
27 beaker.cache.sql_cache_long.expire=3600 <-- set this to higher number
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
28
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
29 This option affects the cache expiration time for main page. Having
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
30 few hundreds of repositories on main page can sometimes make the system
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
31 to behave slow when cache expires for all of them. Increasing `expire`
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
32 option to day (86400) or a week (604800) will improve general response
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
33 times for the main page. Kallithea has an intelligent cache expiration
2775
5d12768a0aa1 docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents: 2680
diff changeset
34 system and it will expire cache for repositories that had been changed.
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
35
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
36 2. Switch from sqlite to postgres or mysql
3224
8b8edfc25856 whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 2775
diff changeset
37
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
38 sqlite is a good option when having small load on the system. But due to
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
39 locking issues with sqlite, it's not recommended to use it for larger
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
40 setup. Switching to mysql or postgres will result in a immediate
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
41 performance increase.
3224
8b8edfc25856 whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 2775
diff changeset
42
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
43 3. Scale Kallithea horizontally
2517
fa88997aa421 Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
44
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
45 Scaling horizontally can give huge performance increase when dealing with
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
46 large traffic (large amount of users, CI servers etc). Kallithea can be
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
47 scaled horizontally on one (recommended) or multiple machines. In order
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
48 to scale horizontally you need to do the following:
3413
d79f3505549e whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 3390
diff changeset
49
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
50 - each instance needs it's own .ini file and unique `instance_id` set in them
3413
d79f3505549e whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 3390
diff changeset
51 - each instance `data` storage needs to be configured to be stored on a
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
52 shared disk storage, preferably together with repositories. This `data`
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
53 dir contains template caches, sessions, whoosh index and it's used for
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
54 tasks locking (so it's safe across multiple instances). Set the
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
55 `cache_dir`, `index_dir`, `beaker.cache.data_dir`, `beaker.cache.lock_dir`
4192
e73a69cb98dc Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents: 3413
diff changeset
56 variables in each .ini file to shared location across Kallithea instances
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
57 - if celery is used each instance should run separate celery instance, but
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
58 the message broken should be common to all of them (ex one rabbitmq
3413
d79f3505549e whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 3390
diff changeset
59 shared server)
3390
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
60 - load balance using round robin or ip hash, recommended is writing LB rules
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
61 that will separate regular user traffic from automated processes like CI
4cd84f4f28fb performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents: 3224
diff changeset
62 servers or build bots.