Mercurial > kallithea
annotate docs/usage/performance.rst @ 4590:9ff235be8b48
cancel.png: replace icon-ban-circle with icon-block
author | Sean Farley <sean.michael.farley@gmail.com> |
---|---|
date | Mon, 14 Jul 2014 23:35:10 -0500 |
parents | e73a69cb98dc |
children | 4e6dfdb3fa01 |
rev | line source |
---|---|
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
1 .. _performance: |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
2 |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
3 ================================ |
4192
e73a69cb98dc
Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents:
3413
diff
changeset
|
4 Optimizing Kallithea Performance |
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
5 ================================ |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
6 |
4192
e73a69cb98dc
Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents:
3413
diff
changeset
|
7 When serving large amount of big repositories Kallithea can start |
2775
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
8 performing slower than expected. Because of demanding nature of handling large |
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
9 amount of data from version control systems here are some tips how to get |
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
10 the best performance. |
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
11 |
4192
e73a69cb98dc
Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents:
3413
diff
changeset
|
12 * Kallithea will perform better on machines with faster disks (SSD/SAN). It's |
2775
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
13 more important to have faster disk than faster CPU. |
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
14 |
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
15 * Slowness on initial page can be easily fixed by grouping repositories, and/or |
3390
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
16 increasing cache size (see below), that includes using lightweight dashboard |
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
17 option and vcs_full_cache setting in .ini file |
2775
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
18 |
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
19 |
4192
e73a69cb98dc
Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents:
3413
diff
changeset
|
20 Follow these few steps to improve performance of Kallithea system. |
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
21 |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
22 |
2680 | 23 1. Increase cache |
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
24 |
2680 | 25 in the .ini file:: |
3224
8b8edfc25856
whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents:
2775
diff
changeset
|
26 |
2680 | 27 beaker.cache.sql_cache_long.expire=3600 <-- set this to higher number |
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
28 |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
29 This option affects the cache expiration time for main page. Having |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
30 few hundreds of repositories on main page can sometimes make the system |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
31 to behave slow when cache expires for all of them. Increasing `expire` |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
32 option to day (86400) or a week (604800) will improve general response |
4192
e73a69cb98dc
Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents:
3413
diff
changeset
|
33 times for the main page. Kallithea has an intelligent cache expiration |
2775
5d12768a0aa1
docs improvements
Marcin Kuzminski <marcin@python-works.com>
parents:
2680
diff
changeset
|
34 system and it will expire cache for repositories that had been changed. |
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
35 |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
36 2. Switch from sqlite to postgres or mysql |
3224
8b8edfc25856
whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents:
2775
diff
changeset
|
37 |
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
38 sqlite is a good option when having small load on the system. But due to |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
39 locking issues with sqlite, it's not recommended to use it for larger |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
40 setup. Switching to mysql or postgres will result in a immediate |
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
41 performance increase. |
3224
8b8edfc25856
whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents:
2775
diff
changeset
|
42 |
4192
e73a69cb98dc
Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents:
3413
diff
changeset
|
43 3. Scale Kallithea horizontally |
2517
fa88997aa421
Added simple docs for optimizing RhodeCode performance
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
44 |
3390
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
45 Scaling horizontally can give huge performance increase when dealing with |
4192
e73a69cb98dc
Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents:
3413
diff
changeset
|
46 large traffic (large amount of users, CI servers etc). Kallithea can be |
3390
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
47 scaled horizontally on one (recommended) or multiple machines. In order |
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
48 to scale horizontally you need to do the following: |
3413
d79f3505549e
whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents:
3390
diff
changeset
|
49 |
3390
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
50 - each instance needs it's own .ini file and unique `instance_id` set in them |
3413
d79f3505549e
whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents:
3390
diff
changeset
|
51 - each instance `data` storage needs to be configured to be stored on a |
3390
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
52 shared disk storage, preferably together with repositories. This `data` |
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
53 dir contains template caches, sessions, whoosh index and it's used for |
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
54 tasks locking (so it's safe across multiple instances). Set the |
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
55 `cache_dir`, `index_dir`, `beaker.cache.data_dir`, `beaker.cache.lock_dir` |
4192
e73a69cb98dc
Rename some strings examples and commands in documentation
Bradley M. Kuhn <bkuhn@sfconservancy.org>
parents:
3413
diff
changeset
|
56 variables in each .ini file to shared location across Kallithea instances |
3390
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
57 - if celery is used each instance should run separate celery instance, but |
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
58 the message broken should be common to all of them (ex one rabbitmq |
3413
d79f3505549e
whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents:
3390
diff
changeset
|
59 shared server) |
3390
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
60 - load balance using round robin or ip hash, recommended is writing LB rules |
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
61 that will separate regular user traffic from automated processes like CI |
4cd84f4f28fb
performance section docs update
Marcin Kuzminski <marcin@python-works.com>
parents:
3224
diff
changeset
|
62 servers or build bots. |