view docs/usage/performance.rst @ 4815:64b1a2320bcb

docs: update Windows installation documentation for Windows 7/Server 2008 R2 and newer Update to the Windows installation documentation following my setup experience on our production server. Changes : * Use of Microsoft Visual C++ Compiler for Python 2.7 which simplifies matter versus installing Visual Studio 2008 Express, because it removes the need to use the Visual Studio 2008 command prompt. It is also a lot smaller to download and install. Unfortunately, this means the instruction will only work on the same platforms as those which are supported by the compiler. * For that reason, I split the documentation into newer and older Windows. * Added more explanations where I feel it was necessary based on my experience * Added explanation on Git (Warning : I did not try this part) * Instructions assumes x64 instead of Win32. * Clarified titles * Grammar Potential issues: * I have the user install pip system wide so that virtualenv installation is easier (especially if using Python 2.7.9 which already includes pip). One may prefer to install virtualenv and the pip in the virtual environment (I know of no good reason, but it could happen). * Removed some line feeds that I found useless. I do not know the .rst format, they might be needed. It makes no difference when the documentation is generated using make.bat, so I am not sure Potential improvements: * Instructions on using srvany.exe to install as a Windows service * Instructions to make a reverse proxy using Apache * Instructions to make a reverse proxy using IIS
author Denis Blanchette <dblanchette@coveo.com>
date Mon, 02 Feb 2015 17:20:08 -0500
parents e73a69cb98dc
children 4e6dfdb3fa01
line wrap: on
line source

.. _performance:

================================
Optimizing Kallithea Performance
================================

When serving large amount of big repositories Kallithea can start
performing slower than expected. Because of demanding nature of handling large
amount of data from version control systems here are some tips how to get
the best performance.

* Kallithea will perform better on machines with faster disks (SSD/SAN). It's
  more important to have faster disk than faster CPU.

* Slowness on initial page can be easily fixed by grouping repositories, and/or
  increasing cache size (see below), that includes using lightweight dashboard
  option and vcs_full_cache setting in .ini file


Follow these few steps to improve performance of Kallithea system.


1. Increase cache

    in the .ini file::

     beaker.cache.sql_cache_long.expire=3600 <-- set this to higher number

    This option affects the cache expiration time for main page. Having
    few hundreds of repositories on main page can sometimes make the system
    to behave slow when cache expires for all of them. Increasing `expire`
    option to day (86400) or a week (604800) will improve general response
    times for the main page. Kallithea has an intelligent cache expiration
    system and it will expire cache for repositories that had been changed.

2. Switch from sqlite to postgres or mysql

    sqlite is a good option when having small load on the system. But due to
    locking issues with sqlite, it's not recommended to use it for larger
    setup. Switching to mysql or postgres will result in a immediate
    performance increase.

3. Scale Kallithea horizontally

    Scaling horizontally can give huge performance increase when dealing with
    large traffic (large amount of users, CI servers etc). Kallithea can be
    scaled horizontally on one (recommended) or multiple machines. In order
    to scale horizontally you need to do the following:

    - each instance needs it's own .ini file and unique `instance_id` set in them
    - each instance `data` storage needs to be configured to be stored on a
      shared disk storage, preferably together with repositories. This `data`
      dir contains template caches, sessions, whoosh index and it's used for
      tasks locking (so it's safe across multiple instances). Set the
      `cache_dir`, `index_dir`, `beaker.cache.data_dir`, `beaker.cache.lock_dir`
      variables in each .ini file to shared location across Kallithea instances
    - if celery is used each instance should run separate celery instance, but
      the message broken should be common to all of them (ex one rabbitmq
      shared server)
    - load balance using round robin or ip hash, recommended is writing LB rules
      that will separate regular user traffic from automated processes like CI
      servers or build bots.