Mercurial > kallithea
changeset 6554:2c3d30095d5e
gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack
This is a step towards moving away from the Pylons stack to TurboGears2, but
still independent of it.
Some notes from the porting - it could perhaps be the missing(?) documentation
for migrating from paster to gearbox:
Note: 'gearbox' without parameters will crash - specify '-h' to get started
testing.
Replace paster
summary = 'yada yada'
with the first line of the docstring of the Command class ... or override
get_description.
Note: All newlines in the docstring will be collapsed and mangle the long help
text.
Grouping of commands is not possible. Standard commands (for development) can't
be customized under the same name or hidden. (Like for paster, the conceptual
model also assumes that the sub-command naming is namespaced so commands from
other packages won't conflict.)
The usage help is fully automated from the declared options.
For all deprecated Commands, replace paster
hidden = True
with gearbox
deprecated = True
Note: config_file, takes_config_file, min_args and max_args are not available /
relevant.
The gearbox parser is customized by overriding get_parser - there is nothing
like paster update_parser.
Gearbox is using argparse instead of optparse ... but argparse add_argument is
mostly backwards compatible with optparse add_option.
Instead of overriding command or run as in paster, override take_action in
gearbox. The parsed arguments are passed to take_action, not available on the
command instance.
Paster BadCommand is not available and must be handled manually, terminating
with sys.exit(1).
There is no standard make-config command in gearbox.
Paster appinstall has been replaced by the somewhat different setup_app module
in gearbox. There is still no clean way to pass parameters to SetupAppCommand
and it relies on websetup and other apparently unnecessary complexity. Instead,
implement setup-db from scratch.
Minor change by Thomas De Schampheleire: add gearbox logging configuration.
Because we use logging.config.fileConfig(.inifile) during gearbox command
execution, the logging settings need to be correct and contain a block for
gearbox logging itself. Otherwise, errors in command processing are not even
visible and the command exits silently.
line wrap: on
line diff
--- a/development.ini Sat Dec 24 01:27:47 2016 +0100 +++ b/development.ini Tue Oct 25 21:32:24 2016 +0200 @@ -67,8 +67,8 @@ #smtp_use_tls = false [server:main] -## PASTE ## -#use = egg:Paste#http +## Gearbox default web server ## +#use = egg:gearbox#wsgiref ## nr of worker threads to spawn #threadpool_workers = 1 ## max request before thread respawn @@ -76,6 +76,9 @@ ## option to use threads of process #use_threadpool = true +## Gearbox gevent web server ## +#use = egg:gearbox#gevent + ## WAITRESS ## use = egg:waitress#main ## number of worker threads @@ -509,7 +512,7 @@ ################################ [loggers] -keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer +keys = root, routes, kallithea, sqlalchemy, gearbox, beaker, templates, whoosh_indexer [handlers] keys = console, console_sql @@ -550,6 +553,12 @@ qualname = kallithea propagate = 1 +[logger_gearbox] +level = DEBUG +handlers = +qualname = gearbox +propagate = 1 + [logger_sqlalchemy] level = INFO handlers = console_sql
--- a/docs/contributing.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/contributing.rst Tue Oct 25 21:32:24 2016 +0200 @@ -36,9 +36,9 @@ source ../kallithea-venv/bin/activate pip install --upgrade pip setuptools pip install -e . - paster make-config Kallithea my.ini - paster setup-db my.ini --user=user --email=user@example.com --password=password --repos=/tmp - paster serve my.ini --reload & + TODO make-config Kallithea my.ini + gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp + gearbox serve -c my.ini --reload & firefox http://127.0.0.1:5000/ You can also start out by forking https://bitbucket.org/conservancy/kallithea @@ -66,7 +66,7 @@ It is possible to avoid recreating the full test database on each invocation of the tests, thus eliminating the initial delay. To achieve this, run the tests as:: - paster serve kallithea/tests/test.ini --pid-file=test.pid --daemon + gearbox serve -c kallithea/tests/test.ini --pid-file=test.pid --daemon KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test kill -9 $(cat test.pid)
--- a/docs/dev/dbmigrations.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/dev/dbmigrations.rst Tue Oct 25 21:32:24 2016 +0200 @@ -18,7 +18,7 @@ 2. Create a separate throwaway configuration for iterating on the actual database changes:: - paster make-config Kallithea temp.ini + TODO make-config Kallithea temp.ini Edit the file to change database settings. SQLite is typically fine, but make sure to change the path to e.g. ``temp.db``, to avoid @@ -30,8 +30,8 @@ to test the changes:: rm temp.db - paster setup-db temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes - paster repo-scan temp.ini + gearbox setup-db -c temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes + gearbox repo-scan -c temp.ini 5. Once satisfied with the schema changes, auto-generate a draft Alembic script using the development database that has *not* been upgraded.
--- a/docs/installation_iis.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/installation_iis.rst Tue Oct 25 21:32:24 2016 +0200 @@ -59,7 +59,7 @@ The ISAPI handler can be generated using:: - paster install-iis my.ini --virtualdir=/ + gearbox install-iis -c my.ini --virtualdir=/ This will generate a ``dispatch.py`` file in the current directory that contains the necessary components to finalize an installation into IIS. Once this file @@ -74,7 +74,7 @@ The ISAPI handler is registered to all file extensions, so it will automatically be the one handling all requests to the specified virtual directory. When the website starts -the ISAPI handler, it will start a thread pool managed wrapper around the paster +the ISAPI handler, it will start a thread pool managed wrapper around the middleware WSGI handler that Kallithea runs within and each HTTP request to the site will be processed through this logic henceforth. @@ -111,7 +111,7 @@ --------------- Typically, any issues in this setup will either be entirely in IIS or entirely -in Kallithea (or Kallithea's WSGI/paster middleware). Consequently, two +in Kallithea (or Kallithea's WSGI middleware). Consequently, two different options for finding issues exist: IIS' failed request tracking which is great at finding issues until they exist inside Kallithea, at which point the ISAPI-WSGI wrapper above uses ``win32traceutil``, which is part of ``pywin32``.
--- a/docs/installation_win.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/installation_win.rst Tue Oct 25 21:32:24 2016 +0200 @@ -166,7 +166,7 @@ one). When ready, type:: cd C:\Kallithea\Bin - paster make-config Kallithea production.ini + TODO make-config Kallithea production.ini Then you must edit production.ini to fit your needs (IP address, IP port, mail settings, database, etc.). `NotePad++`__ or a similar text @@ -177,7 +177,7 @@ For the sake of simplicity, run it with the default settings. After your edits (if any) in the previous command prompt, type:: - paster setup-db production.ini + gearbox setup-db -c production.ini .. warning:: This time a *new* database will be installed. You must follow a different process to later :ref:`upgrade <upgrade>` @@ -200,7 +200,7 @@ In the previous command prompt, being in the C:\\Kallithea\\Bin folder, type:: - paster serve production.ini + gearbox serve -c production.ini Open your web server, and go to http://127.0.0.1:5000
--- a/docs/installation_win_old.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/installation_win_old.rst Tue Oct 25 21:32:24 2016 +0200 @@ -204,7 +204,7 @@ "activate" one). When ready, just type:: cd C:\Kallithea\Bin - paster make-config Kallithea production.ini + TODO make-config Kallithea production.ini Then, you must edit production.ini to fit your needs (network address and port, mail settings, database, whatever). I recommend using NotePad++ @@ -215,7 +215,7 @@ For the sake of simplicity lets run it with the default settings. After your edits (if any), in the previous Command Prompt, type:: - paster setup-db production.ini + gearbox setup-db -c production.ini .. warning:: This time a *new* database will be installed. You must follow a different process to later :ref:`upgrade <upgrade>` @@ -239,7 +239,7 @@ In the previous command prompt, being in the C:\\Kallithea\\Bin folder, just type:: - paster serve production.ini + gearbox serve -c production.ini Open yout web server, and go to http://127.0.0.1:5000
--- a/docs/overview.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/overview.rst Tue Oct 25 21:32:24 2016 +0200 @@ -84,17 +84,17 @@ There are several web server options: -- Kallithea uses the Paste_ tool as command line interface. Paste provides - ``paster serve`` as a convenient way to launch a Python WSGI / web server +- Kallithea uses the Gearbox_ tool as command line interface. Gearbox provides + ``gearbox serve`` as a convenient way to launch a Python WSGI / web server from the command line. That is perfect for development and evaluation. Actual use in production might have different requirements and need extra work to make it manageable as a scalable system service. - Paste comes with its own built-in web server but Kallithea defaults to use + Gearbox comes with its own built-in web server but Kallithea defaults to use Waitress_. Gunicorn_ is also an option. These web servers have different limited feature sets. - The web server used by ``paster`` is configured in the ``.ini`` file passed + The web server used by ``gearbox`` is configured in the ``.ini`` file passed to it. The entry point for the WSGI application is configured in ``setup.py`` as ``kallithea.config.middleware:make_app``. @@ -113,7 +113,7 @@ encryption or special authentication or for other security reasons, to provide caching of static files, or to provide load balancing or fail-over. Nginx_, Varnish_ and HAProxy_ are often used for this purpose, often in front - of a ``paster`` server that somehow is wrapped as a service. + of a ``gearbox serve`` that somehow is wrapped as a service. The best option depends on what you are familiar with and the requirements for performance and stability. Also, keep in mind that Kallithea mainly is serving @@ -126,7 +126,7 @@ .. _Gunicorn: http://gunicorn.org/ .. _Waitress: http://waitress.readthedocs.org/en/latest/ .. _virtualenv: http://pypi.python.org/pypi/virtualenv -.. _Paste: http://pythonpaste.org/ +.. _Gearbox: http://turbogears.readthedocs.io/en/latest/turbogears/gearbox.html .. _PyPI: https://pypi.python.org/pypi .. _Apache httpd: http://httpd.apache.org/ .. _mod_wsgi: https://code.google.com/p/modwsgi/
--- a/docs/setup.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/setup.rst Tue Oct 25 21:32:24 2016 +0200 @@ -11,7 +11,7 @@ First, you will need to create a Kallithea configuration file. Run the following command to do so:: - paster make-config Kallithea my.ini + TODO make-config Kallithea my.ini This will create the file ``my.ini`` in the current directory. This configuration file contains the various settings for Kallithea, e.g. @@ -25,7 +25,7 @@ PostgreSQL, SQLite and MySQL databases. Create the database by running the following command:: - paster setup-db my.ini + gearbox setup-db -c my.ini This will prompt you for a "root" path. This "root" path is the location where Kallithea will store all of its repositories on the current machine. After @@ -36,7 +36,7 @@ The ``setup-db`` values can also be given on the command line. Example:: - paster setup-db my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos + gearbox setup-db -c my.ini --user=nn --password=secret --email=nn@example.com --repos=/srv/repos The ``setup-db`` command will create all needed tables and an admin account. When choosing a root path you can either use a new @@ -54,7 +54,7 @@ You are now ready to use Kallithea. To run it simply execute:: - paster serve my.ini + gearbox serve -c my.ini - This command runs the Kallithea server. The web app should be available at http://127.0.0.1:5000. The IP address and port is configurable via the @@ -111,23 +111,23 @@ For an incremental index build, run:: - paster make-index my.ini + gearbox make-index -c my.ini For a full index rebuild, run:: - paster make-index my.ini -f + gearbox make-index -c my.ini -f The ``--repo-location`` option allows the location of the repositories to be overridden; usually, the location is retrieved from the Kallithea database. The ``--index-only`` option can be used to limit the indexed repositories to a comma-separated list:: - paster make-index my.ini --index-only=vcs,kallithea + gearbox make-index -c my.ini --index-only=vcs,kallithea To keep your index up-to-date it is necessary to do periodic index builds; for this, it is recommended to use a crontab entry. Example:: - 0 3 * * * /path/to/virtualenv/bin/paster make-index /path/to/kallithea/my.ini + 0 3 * * * /path/to/virtualenv/bin/gearbox make-index -c /path/to/kallithea/my.ini When using incremental mode (the default), Whoosh will check the last modification date of each file and add it to be reindexed if a newer file is @@ -592,7 +592,10 @@ To start the Celery process, run:: - paster celeryd <configfile.ini> + gearbox celeryd -c <configfile.ini> + +Extra options to the Celery worker can be passed after ``--`` - see ``-- -h`` +for more info. .. note:: Make sure you run this command from the same virtualenv, and with the same
--- a/docs/upgrade.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/upgrade.rst Tue Oct 25 21:32:24 2016 +0200 @@ -93,7 +93,7 @@ Run the following command to upgrade your configuration (``.ini``) file:: - paster make-config Kallithea my.ini + TODO make-config Kallithea my.ini This will display any changes made by the new version of Kallithea to your current configuration, and attempt an automatic merge. It is recommended
--- a/docs/usage/customization.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/usage/customization.rst Tue Oct 25 21:32:24 2016 +0200 @@ -32,7 +32,7 @@ To generate a skeleton extensions package, run:: - paster make-rcext my.ini + gearbox make-rcext -c my.ini This will create an ``rcextensions`` package next to the specified ``ini`` file. See the ``__init__.py`` file inside the generated ``rcextensions`` package
--- a/docs/usage/general.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/usage/general.rst Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ There is a special command for cleaning up such archived repositories:: - paster cleanup-repos --older-than=30d my.ini + gearbox cleanup-repos --older-than=30d -c my.ini This command scans for archived repositories that are older than 30 days, displays them, and asks if you want to delete them (unless given
--- a/docs/usage/vcs_support.rst Sat Dec 24 01:27:47 2016 +0100 +++ b/docs/usage/vcs_support.rst Tue Oct 25 21:32:24 2016 +0200 @@ -28,10 +28,10 @@ Large Git pushes require an HTTP server with support for chunked encoding for POST. The Python web servers waitress_ and gunicorn_ (Linux only) can be used. By default, Kallithea uses -waitress_ for `paster serve` instead of the built-in `paste` WSGI +waitress_ for `gearbox serve` instead of the built-in `paste` WSGI server. -The paster server is controlled in the .ini file:: +The web server used by gearbox is controlled in the .ini file:: use = egg:waitress#main
--- a/init.d/celeryd-upstart.conf Sat Dec 24 01:27:47 2016 +0100 +++ b/init.d/celeryd-upstart.conf Tue Oct 25 21:32:24 2016 +0200 @@ -21,7 +21,7 @@ # env GROUP=hg script - COMMAND="/var/hg/.virtualenvs/kallithea/bin/paster celeryd $APPINI --pidfile=$PIDFILE" + COMMAND="/var/hg/.virtualenvs/kallithea/bin/gearbox celeryd -c $APPINI -- --pidfile=$PIDFILE" if [ -z "$GROUP" ]; then exec sudo -u $USER $COMMAND else
--- a/init.d/kallithea-daemon-arch Sat Dec 24 01:27:47 2016 +0100 +++ b/init.d/kallithea-daemon-arch Tue Oct 25 21:32:24 2016 +0200 @@ -12,7 +12,7 @@ CONF_NAME="production.ini" LOG_FILE="/var/log/$DAEMON.log" PID_FILE="/run/daemons/$DAEMON" -APPL=/usr/bin/paster +APPL=/usr/bin/gearbox RUN_AS="*****" ARGS="serve --daemon \ @@ -20,7 +20,7 @@ --group=$RUN_AS \ --pid-file=$PID_FILE \ --log-file=$LOG_FILE \ -$APP_PATH/$CONF_NAME" +-c $APP_PATH/$CONF_NAME" [ -r /etc/conf.d/$DAEMON ] && . /etc/conf.d/$DAEMON
--- a/init.d/kallithea-daemon-debian Sat Dec 24 01:27:47 2016 +0100 +++ b/init.d/kallithea-daemon-debian Tue Oct 25 21:32:24 2016 +0200 @@ -26,13 +26,13 @@ RUN_AS="root" -DAEMON="$PYTHON_PATH/bin/paster" +DAEMON="$PYTHON_PATH/bin/gearbox" DAEMON_OPTS="serve --daemon \ --user=$RUN_AS \ --group=$RUN_AS \ --pid-file=$PID_PATH \ - --log-file=$LOG_PATH $APP_PATH/$CONF_NAME" + --log-file=$LOG_PATH -c $APP_PATH/$CONF_NAME" start() {
--- a/init.d/kallithea-daemon-gentoo Sat Dec 24 01:27:47 2016 +0100 +++ b/init.d/kallithea-daemon-gentoo Tue Oct 25 21:32:24 2016 +0200 @@ -16,13 +16,13 @@ RUN_AS="username" -DAEMON="$PYTHON_PATH/bin/paster" +DAEMON="$PYTHON_PATH/bin/gearbox" DAEMON_OPTS="serve --daemon \ --user=$RUN_AS \ --group=$RUN_AS \ --pid-file=$PID_PATH \ ---log-file=$LOG_PATH $APP_PATH/$CONF_NAME" +--log-file=$LOG_PATH -c $APP_PATH/$CONF_NAME" #extra options opts="${opts} restartdelay"
--- a/init.d/kallithea-daemon-redhat Sat Dec 24 01:27:47 2016 +0100 +++ b/init.d/kallithea-daemon-redhat Tue Oct 25 21:32:24 2016 +0200 @@ -20,7 +20,7 @@ CONF_NAME="production.ini" # write to wherever the PID should be stored, just ensure -# that the user you run paster as has the appropriate permissions +# that the user you run gearbox as has the appropriate permissions # same goes for the log file PID_PATH="/var/run/kallithea/pid" LOG_PATH="/var/log/kallithea/kallithea.log" @@ -31,13 +31,13 @@ RUN_AS="kallithea" -DAEMON="$PYTHON_PATH/bin/paster" +DAEMON="$PYTHON_PATH/bin/gearbox" DAEMON_OPTS="serve --daemon \ --user=$RUN_AS \ --group=$RUN_AS \ --pid-file=$PID_PATH \ - --log-file=$LOG_PATH $APP_PATH/$CONF_NAME" + --log-file=$LOG_PATH -c $APP_PATH/$CONF_NAME" DESC="kallithea-server" LOCK_FILE="/var/lock/subsys/$APP_NAME"
--- a/init.d/kallithea-upstart.conf Sat Dec 24 01:27:47 2016 +0100 +++ b/init.d/kallithea-upstart.conf Tue Oct 25 21:32:24 2016 +0200 @@ -19,7 +19,7 @@ env USER=hg env GROUP=hg -exec /var/hg/.virtualenvs/kallithea/bin/paster serve --user=$USER --group=$GROUP --pid-file=$PIDFILE --log-file=$LOGFILE $APPINI +exec /var/hg/.virtualenvs/kallithea/bin/gearbox serve --user=$USER --group=$GROUP --pid-file=$PIDFILE --log-file=$LOGFILE -c $APPINI post-stop script rm -f $PIDFILE
--- a/init.d/supervisord.conf Sat Dec 24 01:27:47 2016 +0100 +++ b/init.d/supervisord.conf Tue Oct 25 21:32:24 2016 +0200 @@ -45,7 +45,7 @@ numprocs = 1 numprocs_start = 5000 # possible should match ports directory=/srv/kallithea -command = /srv/kallithea/venv/bin/paster serve my.ini +command = /srv/kallithea/venv/bin/gearbox serve -c my.ini process_name = %(program_name)s_%(process_num)04d redirect_stderr=true stdout_logfile=/%(here)s/kallithea.log
--- a/kallithea/alembic/env.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/alembic/env.py Tue Oct 25 21:32:24 2016 +0200 @@ -40,7 +40,7 @@ # Setup Python loggers based on the config file provided to the alembic # command. If we're being invoked via the Alembic API (presumably for -# stamping during "paster setup-db"), config_file_name is not available, +# stamping during "gearbox setup-db"), config_file_name is not available, # and loggers are assumed to already have been configured. if config.config_file_name: fileConfig(config.config_file_name, disable_existing_loggers=False)
--- a/kallithea/bin/template.ini.mako Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/bin/template.ini.mako Tue Oct 25 21:32:24 2016 +0200 @@ -61,9 +61,9 @@ #smtp_use_tls = false [server:main] -%if http_server == 'paste': -<%text>## PASTE ##</%text> -use = egg:Paste#http +%if http_server == 'gearbox': +<%text>## Gearbox default web server ##</%text> +use = egg:gearbox#wsgiref <%text>## nr of worker threads to spawn</%text> threadpool_workers = 1 <%text>## max request before thread respawn</%text> @@ -71,6 +71,10 @@ <%text>## option to use threads of process</%text> use_threadpool = true +%elif http_server == 'gevent': +<%text>## Gearbox gevent web server ##</%text> +use = egg:gearbox#gevent + %elif http_server == 'waitress': <%text>## WAITRESS ##</%text> use = egg:waitress#main @@ -512,7 +516,7 @@ <%text>################################</%text> [loggers] -keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer +keys = root, routes, kallithea, sqlalchemy, gearbox, beaker, templates, whoosh_indexer [handlers] keys = console, console_sql @@ -553,6 +557,12 @@ qualname = kallithea propagate = 1 +[logger_gearbox] +level = DEBUG +handlers = +qualname = gearbox +propagate = 1 + [logger_sqlalchemy] level = INFO handlers = console_sql
--- a/kallithea/config/deployment.ini_tmpl Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/config/deployment.ini_tmpl Tue Oct 25 21:32:24 2016 +0200 @@ -62,8 +62,8 @@ #smtp_use_tls = false [server:main] -## PASTE ## -#use = egg:Paste#http +## Gearbox default web server ## +#use = egg:gearbox#wsgiref ## nr of worker threads to spawn #threadpool_workers = 1 ## max request before thread respawn @@ -71,6 +71,9 @@ ## option to use threads of process #use_threadpool = true +## Gearbox gevent web server ## +#use = egg:gearbox#gevent + ## WAITRESS ## use = egg:waitress#main ## number of worker threads @@ -501,7 +504,7 @@ ################################ [loggers] -keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer +keys = root, routes, kallithea, sqlalchemy, gearbox, beaker, templates, whoosh_indexer [handlers] keys = console, console_sql @@ -542,6 +545,12 @@ qualname = kallithea propagate = 1 +[logger_gearbox] +level = DEBUG +handlers = +qualname = gearbox +propagate = 1 + [logger_sqlalchemy] level = INFO handlers = console_sql
--- a/kallithea/lib/dbmigrate/__init__.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/dbmigrate/__init__.py Tue Oct 25 21:32:24 2016 +0200 @@ -1,8 +1,9 @@ -from paste.script.command import Command +from gearbox.command import Command class UpgradeDb(Command): - hidden = True - summary = '(removed)' + '''(removed)''' + + deprecated = True def run(self, args): raise SystemExit(
--- a/kallithea/lib/paster_commands/cache_keys.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/cache_keys.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ kallithea.lib.paster_commands.cache_keys ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -cleanup-keys paster command for Kallithea +cleanup-keys gearbox command for Kallithea This file was forked by the Kallithea project in July 2014. @@ -37,25 +37,14 @@ class Command(BasePasterCommand): - - max_args = 1 - min_args = 1 + "Kallithea: Utilities for managing caching of database content" - usage = "CONFIG_FILE" - group_name = "Kallithea" - takes_config_file = -1 - parser = BasePasterCommand.standard_parser(verbose=True) - summary = "Cache keys utils" - - def command(self): - #get SqlAlchemy session - self._init_session() - + def take_action(self, args): _caches = CacheInvalidation.query().order_by(CacheInvalidation.cache_key).all() - if self.options.show: + if args.show: for c_obj in _caches: print 'key:%s active:%s' % (safe_str(c_obj.cache_key), c_obj.cache_active) - elif self.options.cleanup: + elif args.cleanup: for c_obj in _caches: Session().delete(c_obj) print 'Removing key: %s' % (safe_str(c_obj.cache_key)) @@ -63,17 +52,21 @@ else: print 'Nothing done, exiting...' - def update_parser(self): - self.parser.add_option( + def get_parser(self, prog_name): + parser = super(Command, self).get_parser(prog_name) + + parser.add_argument( '--show', action='store_true', dest='show', - help=("show existing cache keys with together with status") + help="show existing cache keys with together with status", ) - self.parser.add_option( + parser.add_argument( '--cleanup', action="store_true", dest="cleanup", - help="cleanup existing cache keys" + help="cleanup existing cache keys", ) + + return parser
--- a/kallithea/lib/paster_commands/celeryd.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/celeryd.py Tue Oct 25 21:32:24 2016 +0200 @@ -1,5 +1,7 @@ # -*- coding: utf-8 -*- +import argparse + import kallithea from kallithea.lib.paster_commands.common import BasePasterCommand from kallithea.lib.utils import load_rcextensions @@ -9,26 +11,14 @@ class Command(BasePasterCommand): - """Start the celery worker - - Starts the celery worker that uses a paste.deploy configuration - file. - """ + """Kallithea: Celery worker for asynchronous tasks""" - usage = 'CONFIG_FILE [celeryd options...]' - summary = __doc__.splitlines()[0] - description = "".join(__doc__.splitlines()[2:]) - group_name = "Kallithea" - - parser = BasePasterCommand.standard_parser(quiet=True) + # Starts the celery worker using configuration from a paste.deploy + # configuration file. - def update_parser(self): - from kallithea.lib import celerypylons - cmd = celerypylons.worker.worker(celerypylons.app.app_or_default()) - for x in cmd.get_options(): - self.parser.add_option(x) + requires_db_session = False # will start session on demand - def command(self): + def take_action(self, args): from kallithea.lib import celerypylons from tg import config try: @@ -43,4 +33,18 @@ load_rcextensions(config['here']) cmd = celerypylons.worker.worker(celerypylons.app.app_or_default()) - return cmd.run(**vars(self.options)) + + celery_args = args.celery_args + if '--' in celery_args: + celery_args.remove('--') + + return cmd.run_from_argv('kallithea celery worker', celery_args) + + def get_parser(self, prog_name): + parser = super(Command, self).get_parser(prog_name) + + parser.add_argument('celery_args', nargs=argparse.REMAINDER, + help="Pass extra options to Celery after a '--' separator", + ) + + return parser
--- a/kallithea/lib/paster_commands/cleanup.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/cleanup.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ kallithea.lib.paster_commands.cleanup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -cleanup-repos paster command for Kallithea +cleanup-repos gearbox command for Kallithea This file was forked by the Kallithea project in July 2014. @@ -40,15 +40,7 @@ class Command(BasePasterCommand): - - max_args = 1 - min_args = 1 - - usage = "CONFIG_FILE" - group_name = "Kallithea" - takes_config_file = -1 - parser = BasePasterCommand.standard_parser(verbose=True) - summary = "Cleanup deleted repos" + """Kallithea: Cleanup of backup files of deleted repositories""" def _parse_older_than(self, val): regex = re.compile(r'((?P<days>\d+?)d)?((?P<hours>\d+?)h)?((?P<minutes>\d+?)m)?((?P<seconds>\d+?)s)?') @@ -72,10 +64,7 @@ date_part = name[4:19] # 4:19 since we don't parse milliseconds return datetime.datetime.strptime(date_part, '%Y%m%d_%H%M%S') - def command(self): - #get SqlAlchemy session - self._init_session() - + def take_action(self, args): repos_location = Ui.get_repos_location() to_remove = [] for dn_, dirs, f in os.walk(safe_str(repos_location)): @@ -97,7 +86,7 @@ #filter older than (if present)! now = datetime.datetime.now() - older_than = self.options.older_than + older_than = args.older_than if older_than: to_remove_filtered = [] older_than_date = self._parse_older_than(older_than) @@ -111,7 +100,7 @@ % (len(to_remove), older_than, older_than_date) else: print 'Removing all %s deleted repos' % len(to_remove) - if self.options.dont_ask or not to_remove: + if args.dont_ask or not to_remove: # don't ask just remove ! remove = True else: @@ -127,8 +116,10 @@ else: print 'Nothing done, exiting...' - def update_parser(self): - self.parser.add_option( + def get_parser(self, prog_name): + parser = super(Command, self).get_parser(prog_name) + + parser.add_argument( '--older-than', action='store', dest='older_than', @@ -141,9 +132,11 @@ "removed more than 30 days ago.") ) - self.parser.add_option( + parser.add_argument( '--dont-ask', action="store_true", dest="dont_ask", help="remove repositories without asking for confirmation." ) + + return parser
--- a/kallithea/lib/paster_commands/common.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/common.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ kallithea.lib.paster_commands.common ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Common code for Paster commands. +Common code for gearbox commands. This file was forked by the Kallithea project in July 2014. Original author and date, and relevant copyright and licensing information is below: @@ -26,12 +26,11 @@ """ import os -import logging +import sys +import logging.config -import paste -from paste.script.command import Command, BadCommand - -from kallithea.lib.utils import setup_cache_regions +import paste.deploy +import gearbox.command def ask_ok(prompt, retries=4, complaint='Yes or no please!'): @@ -47,14 +46,14 @@ print complaint -class BasePasterCommand(Command): +class BasePasterCommand(gearbox.command.Command): """ - Abstract Base Class for paster commands. + Abstract Base Class for gearbox commands. """ - min_args = 1 - min_args_error = "Please provide a paster config file as an argument." - takes_config_file = 1 - requires_config_file = True + + # override to control how much get_parser and run should do: + takes_config_file = True + requires_db_session = True def run(self, args): """ @@ -62,46 +61,48 @@ Checks for a config file argument and loads it. """ - if len(args) < self.min_args: - raise BadCommand( - self.min_args_error % {'min_args': self.min_args, - 'actual_args': len(args)}) + if self.takes_config_file: + self._bootstrap_config(args.config_file) + if self.requires_db_session: + self._init_session() - # Decrement because we're going to lob off the first argument. - # @@ This is hacky - self.min_args -= 1 - self.bootstrap_config(args[0]) - self.update_parser() - return super(BasePasterCommand, self).run(args[1:]) + return super(BasePasterCommand, self).run(args) + + def get_parser(self, prog_name): + parser = super(BasePasterCommand, self).get_parser(prog_name) - def update_parser(self): + if self.takes_config_file: + parser.add_argument("-c", "--config", + help='Kallithea .ini file with configuration of database etc', + dest='config_file', required=True) + + return parser + + def _bootstrap_config(self, config_file): """ - Abstract method. Allows for the class's parser to be updated - before the superclass's `run` method is called. Necessary to - allow options/arguments to be passed through to the underlying - celery command. - """ - raise NotImplementedError("Abstract Method.") - - def bootstrap_config(self, conf): - """ - Loads the app configuration. + Read the config file and initialize logging and the application. """ from tg import config as pylonsconfig - self.path_to_ini_file = os.path.realpath(conf) - conf = paste.deploy.appconfig('config:' + self.path_to_ini_file) + path_to_ini_file = os.path.realpath(config_file) + conf = paste.deploy.appconfig('config:' + path_to_ini_file) + logging.config.fileConfig(path_to_ini_file) pylonsconfig.init_app(conf.global_conf, conf.local_conf) def _init_session(self): """ - Inits SqlAlchemy Session + Initialize SqlAlchemy Session from global config. """ - logging.config.fileConfig(self.path_to_ini_file) from tg import config from kallithea.model.base import init_model from kallithea.lib.utils2 import engine_from_config + from kallithea.lib.utils import setup_cache_regions setup_cache_regions(config) engine = engine_from_config(config, 'sqlalchemy.') init_model(engine) + + def error(self, msg, exitcode=1): + """Write error message and exit""" + sys.stderr.write('%s\n' % msg) + raise SystemExit(exitcode)
--- a/kallithea/lib/paster_commands/install_iis.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/install_iis.py Tue Oct 25 21:32:24 2016 +0200 @@ -22,37 +22,21 @@ import os import sys from paste.script.appinstall import AbstractInstallCommand -from paste.script.command import BadCommand + +from kallithea.lib.paster_commands.common import BasePasterCommand -class Command(AbstractInstallCommand): - default_verbosity = 1 - max_args = 1 - min_args = 1 - summary = 'Setup IIS given a config file' - usage = 'CONFIG_FILE' +class Command(BasePasterCommand): + '''Kallithea: Install into IIS using isapi-wsgi''' - description = ''' - Script for installing into IIS using isapi-wsgi. - ''' - parser = AbstractInstallCommand.standard_parser( - simulate=True, quiet=True, interactive=True) - parser.add_option('--virtualdir', - action='store', - dest='virtualdir', - default='/', - help='The virtual folder to install into on IIS') + requires_db_session = False - def command(self): - config_spec = self.args[0] - if not config_spec.startswith('config:'): - config_spec = 'config:' + config_spec - config_file = config_spec[len('config:'):].split('#', 1)[0] - config_file = os.path.join(os.getcwd(), config_file) + def take_action(self, args): + config_file = os.path.abspath(args.config_file) try: import isapi_wsgi except ImportError: - raise BadCommand('missing requirement: isapi-wsgi not installed') + self.error('missing requirement: isapi-wsgi not installed') file = '''\ # Created by Kallithea install_iis @@ -92,7 +76,7 @@ outdata = file % { 'inifile': config_file.replace('\\', '\\\\'), - 'virtualdir': self.options.virtualdir + 'virtualdir': args.virtualdir, } dispatchfile = os.path.join(os.getcwd(), 'dispatch.py') @@ -102,3 +86,14 @@ print ('Run \'python "%s" install\' with administrative privileges ' 'to generate the _dispatch.dll file and install it into the ' 'default web site') % (dispatchfile,) + + def get_parser(self, prog_name): + parser = super(Command, self).get_parser(prog_name) + + parser.add_argument('--virtualdir', + action='store', + dest='virtualdir', + default='/', + help='The virtual folder to install into on IIS') + + return parser
--- a/kallithea/lib/paster_commands/ishell.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/ishell.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ kallithea.lib.paster_commands.ishell ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -interactive shell paster command for Kallithea +interactive shell gearbox command for Kallithea This file was forked by the Kallithea project in July 2014. Original author and date, and relevant copyright and licensing information is below: @@ -39,20 +39,9 @@ class Command(BasePasterCommand): - - max_args = 1 - min_args = 1 + "Kallithea: Interactive Python shell" - usage = "CONFIG_FILE" - group_name = "Kallithea" - takes_config_file = -1 - parser = BasePasterCommand.standard_parser(verbose=True) - summary = "Interactive shell" - - def command(self): - #get SqlAlchemy session - self._init_session() - + def take_action(self, args): try: from IPython import embed from IPython.config.loader import Config @@ -62,6 +51,3 @@ except ImportError: print 'Kallithea ishell requires the IPython Python package' sys.exit(-1) - - def update_parser(self): - pass
--- a/kallithea/lib/paster_commands/make_index.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/make_index.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ kallithea.lib.paster_commands.make_index ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -make-index paster command for Kallithea +make-index gearbox command for Kallithea This file was forked by the Kallithea project in July 2014. Original author and date, and relevant copyright and licensing information is below: @@ -37,30 +37,20 @@ class Command(BasePasterCommand): - - max_args = 1 - min_args = 1 + "Kallithea: Create or update full text search index" - usage = "CONFIG_FILE" - group_name = "Kallithea" - takes_config_file = -1 - parser = BasePasterCommand.standard_parser(verbose=True) - summary = "Creates or updates full text search index" - - def command(self): - #get SqlAlchemy session - self._init_session() - from tg import config + def take_action(self, args): + from pylons import config index_location = config['index_dir'] load_rcextensions(config['here']) - repo_location = self.options.repo_location \ - if self.options.repo_location else RepoModel().repos_path - repo_list = map(strip, self.options.repo_list.split(',')) \ - if self.options.repo_list else None + repo_location = args.repo_location \ + if args.repo_location else RepoModel().repos_path + repo_list = map(strip, args.repo_list.split(',')) \ + if args.repo_list else None - repo_update_list = map(strip, self.options.repo_update_list.split(',')) \ - if self.options.repo_update_list else None + repo_update_list = map(strip, args.repo_update_list.split(',')) \ + if args.repo_update_list else None #====================================================================== # WHOOSH DAEMON @@ -74,33 +64,37 @@ repo_location=repo_location, repo_list=repo_list, repo_update_list=repo_update_list) \ - .run(full_index=self.options.full_index) + .run(full_index=args.full_index) l.release() except LockHeld: sys.exit(1) - def update_parser(self): - self.parser.add_option('--repo-location', + def get_parser(self, prog_name): + parser = super(Command, self).get_parser(prog_name) + + parser.add_argument('--repo-location', action='store', dest='repo_location', help="Specifies repositories location to index OPTIONAL", ) - self.parser.add_option('--index-only', + parser.add_argument('--index-only', action='store', dest='repo_list', help="Specifies a comma separated list of repositories " "to build index on. If not given all repositories " "are scanned for indexing. OPTIONAL", ) - self.parser.add_option('--update-only', + parser.add_argument('--update-only', action='store', dest='repo_update_list', help="Specifies a comma separated list of repositories " "to re-build index on. OPTIONAL", ) - self.parser.add_option('-f', + parser.add_argument('-f', action='store_true', dest='full_index', help="Specifies that index should be made full i.e" " destroy old and build from scratch", default=False) + + return parser
--- a/kallithea/lib/paster_commands/make_rcextensions.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/make_rcextensions.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ kallithea.lib.paster_commands.make_rcextensions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -make-rcext paster command for Kallithea +make-rcext gearbox command for Kallithea This file was forked by the Kallithea project in July 2014. Original author and date, and relevant copyright and licensing information is below: @@ -34,23 +34,17 @@ class Command(BasePasterCommand): - - max_args = 1 - min_args = 1 + """Kallithea: Write template file for extending Kallithea in Python - group_name = "Kallithea" - takes_config_file = -1 - parser = BasePasterCommand.standard_parser(verbose=True) - summary = "Write template file for extending Kallithea in Python." - usage = "CONFIG_FILE" - description = '''\ - A rcextensions directory with a __init__.py file will be created next to - the ini file. Local customizations in that file will survive upgrades. - The file contains instructions on how it can be customized. - ''' + A rcextensions directory with a __init__.py file will be created next to + the ini file. Local customizations in that file will survive upgrades. + The file contains instructions on how it can be customized. + """ - def command(self): - from tg import config + takes_config_file = False + + def take_action(self, args): + from pylons import config here = config['here'] content = pkg_resources.resource_string( @@ -70,6 +64,3 @@ with open(ext_file, 'wb') as f: f.write(content) print 'Wrote new extensions file to %s' % ext_file - - def update_parser(self): - pass
--- a/kallithea/lib/paster_commands/repo_scan.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/repo_scan.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ kallithea.lib.paster_commands.repo_scan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -repo-scan paster command for Kallithea +repo-scan gearbox command for Kallithea This file was forked by the Kallithea project in July 2014. Original author and date, and relevant copyright and licensing information is below: @@ -35,20 +35,14 @@ class Command(BasePasterCommand): - - max_args = 1 - min_args = 1 + """Kallithea: Scan file system for repositories - usage = "CONFIG_FILE" - group_name = "Kallithea" - takes_config_file = -1 - parser = BasePasterCommand.standard_parser(verbose=True) - summary = "Rescan default location for new repositories" + Search under the repository root configured in the database, + all new repositories, and report missing ones with an option of removing them. + """ - def command(self): - #get SqlAlchemy session - self._init_session() - rm_obsolete = self.options.delete_obsolete + def take_action(self, args): + rm_obsolete = args.delete_obsolete print 'Now scanning root location for new repos ...' added, removed = repo2db_mapper(ScmModel().repo_scan(), remove_obsolete=rm_obsolete) @@ -61,10 +55,14 @@ else: print 'Missing: %s' % removed - def update_parser(self): - self.parser.add_option( + def get_parser(self, prog_name): + parser = super(Command, self).get_parser(prog_name) + + parser.add_argument( '--delete-obsolete', action='store_true', help="Use this flag do delete repositories that are " "present in Kallithea database but not on the filesystem", ) + + return parser
--- a/kallithea/lib/paster_commands/setup_db.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/setup_db.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,112 +15,93 @@ kallithea.lib.paster_commands.setup_db ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Databaset setup paster command for Kallithea +Databaset setup gearbox command for Kallithea """ import os import sys -from paste.script.appinstall import AbstractInstallCommand -from paste.script.command import BadCommand -from paste.deploy import appconfig +import paste.deploy + +from kallithea.lib.db_manage import DbManage +from kallithea.lib.paster_commands.common import BasePasterCommand +from kallithea.model.meta import Session -class Command(AbstractInstallCommand): +# This is almost like SetupAppCommand ... but we have to pass options and it is +# thus simpler to drop websetup and reimplement everything +class Command(BasePasterCommand): + """Kallithea: Configure the database specified in the .ini file - default_verbosity = 1 - max_args = 1 - min_args = 1 - summary = "Setup an application, given a config file" - usage = "CONFIG_FILE" - group_name = "Kallithea" - - description = """\ - Setup Kallithea according to its configuration file. This is - the second part of a two-phase web application installation - process (the first phase is prepare-app). The setup process - consist of things like setting up databases, creating super user + Setup Kallithea according to its configuration file. This is + the second part of a two-phase web application installation + process (the first phase is prepare-app). The setup process + consist of things like setting up databases and creating the admin user """ - parser = AbstractInstallCommand.standard_parser( - simulate=True, quiet=True, interactive=True) - parser.add_option('--user', - action='store', - dest='username', - default=None, - help='Admin Username') - parser.add_option('--email', - action='store', - dest='email', - default=None, - help='Admin Email') - parser.add_option('--password', - action='store', - dest='password', - default=None, - help='Admin password min 6 chars') - parser.add_option('--repos', - action='store', - dest='repos_location', - default=None, - help='Absolute path to repositories location') - parser.add_option('--name', - action='store', - dest='section_name', - default=None, - help='The name of the section to set up (default: app:main)') - parser.add_option('--force-yes', - action='store_true', - dest='force_ask', - default=None, - help='Force yes to every question') - parser.add_option('--force-no', - action='store_false', - dest='force_ask', - default=None, - help='Force no to every question') - parser.add_option('--public-access', - action='store_true', - dest='public_access', - default=None, - help='Enable public access on this installation (default)') - parser.add_option('--no-public-access', - action='store_false', - dest='public_access', - default=None, - help='Disable public access on this installation ') + def get_description(self): + return self.__doc__.splitlines()[0] + + requires_db_session = False # only available after this command has been run + + def get_parser(self, prog_name): + parser = super(Command, self).get_parser(prog_name) - def command(self): - config_spec = self.args[0] - section = self.options.section_name - if section is None: - if '#' in config_spec: - config_spec, section = config_spec.split('#', 1) - else: - section = 'main' - if not ':' in section: - plain_section = section - section = 'app:' + section - else: - plain_section = section.split(':', 1)[0] - if not config_spec.startswith('config:'): - config_spec = 'config:' + config_spec - if plain_section != 'main': - config_spec += '#' + plain_section - config_file = config_spec[len('config:'):].split('#', 1)[0] - config_file = os.path.join(os.getcwd(), config_file) - self.logging_file_config(config_file) - conf = appconfig(config_spec, relative_to=os.getcwd()) - ep_name = conf.context.entry_point_name - ep_group = conf.context.protocol - dist = conf.context.distribution - if dist is None: - raise BadCommand( - "The section %r is not the application (probably a filter). " - "You should add #section_name, where section_name is the " - "section that configures your application" % plain_section) - installer = self.get_installer(dist, ep_group, ep_name) - installer.setup_config( - self, config_file, section, self.sysconfig_install_vars(installer)) - self.call_sysconfig_functions( - 'post_setup_hook', installer, config_file) + parser.add_argument('--user', + action='store', + dest='username', + default=None, + help='Admin Username') + parser.add_argument('--email', + action='store', + dest='email', + default=None, + help='Admin Email') + parser.add_argument('--password', + action='store', + dest='password', + default=None, + help='Admin password min 6 chars') + parser.add_argument('--repos', + action='store', + dest='repos_location', + default=None, + help='Absolute path to repositories location') + parser.add_argument('--force-yes', + action='store_true', + dest='force_ask', + default=None, + help='Force yes to every question') + parser.add_argument('--force-no', + action='store_false', + dest='force_ask', + default=None, + help='Force no to every question') + parser.add_argument('--public-access', + action='store_true', + dest='public_access', + default=None, + help='Enable public access on this installation (default)') + parser.add_argument('--no-public-access', + action='store_false', + dest='public_access', + default=None, + help='Disable public access on this installation ') + + return parser + + def take_action(self, opts): + path_to_ini_file = os.path.realpath(opts.config_file) + conf = paste.deploy.appconfig('config:' + path_to_ini_file) + + dbconf = conf['sqlalchemy.url'] + dbmanage = DbManage(log_sql=True, dbconf=dbconf, root=conf['here'], + tests=False, cli_args=vars(opts)) + dbmanage.create_tables(override=True) + opts = dbmanage.config_prompt(None) + dbmanage.create_settings(opts) + dbmanage.create_default_user() + dbmanage.admin_prompt() + dbmanage.create_permissions() + dbmanage.populate_default_permissions() + Session().commit()
--- a/kallithea/lib/paster_commands/update_repoinfo.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/lib/paster_commands/update_repoinfo.py Tue Oct 25 21:32:24 2016 +0200 @@ -15,7 +15,7 @@ kallithea.lib.paster_commands.update_repoinfo ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -update-repoinfo paster command for Kallithea +update-repoinfo gearbox command for Kallithea This file was forked by the Kallithea project in July 2014. Original author and date, and relevant copyright and licensing information is below: @@ -38,47 +38,39 @@ class Command(BasePasterCommand): - - max_args = 1 - min_args = 1 + "Kallithea: Update database cache of repository data" - usage = "CONFIG_FILE" - group_name = "Kallithea" - takes_config_file = -1 - parser = BasePasterCommand.standard_parser(verbose=True) - summary = "Updates repositories caches for last changeset" - - def command(self): - #get SqlAlchemy session - self._init_session() - - - if self.options.repo_update_list is None: + def take_action(self, args): + if args.repo_update_list is None: repo_list = Repository.query().all() else: repo_names = [safe_unicode(n.strip()) - for n in self.options.repo_update_list.split(',')] + for n in args.repo_update_list.split(',')] repo_list = list(Repository.query() .filter(Repository.repo_name.in_(repo_names))) for repo in repo_list: repo.update_changeset_cache() Session().commit() - if self.options.invalidate_cache: + if args.invalidate_cache: for r in repo_list: r.set_invalidate() print 'Updated repo info and invalidated cache for %s repositories' % (len(repo_list)) else: print 'Updated repo info for %s repositories' % (len(repo_list)) - def update_parser(self): - self.parser.add_option('--update-only', + def get_parser(self, prog_name): + parser = super(Command, self).get_parser(prog_name) + + parser.add_argument('--update-only', action='store', dest='repo_update_list', help="Specifies a comma separated list of repositories " "to update last commit info for. OPTIONAL") - self.parser.add_option('--invalidate-cache', + parser.add_argument('--invalidate-cache', action='store_true', dest='invalidate_cache', help="Trigger cache invalidation event for repos. " "OPTIONAL") + + return parser
--- a/kallithea/tests/other/manual_test_vcs_operations.py Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/tests/other/manual_test_vcs_operations.py Tue Oct 25 21:32:24 2016 +0200 @@ -18,7 +18,7 @@ Test suite for making push/pull operations. Run it in two terminals:: - paster serve kallithea/tests/test.ini + gearbox serve -c kallithea/tests/test.ini KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test kallithea/tests/other/manual_test_vcs_operations.py You must have git > 1.8.1 for tests to work fine
--- a/kallithea/tests/scripts/create_rc.sh Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/tests/scripts/create_rc.sh Tue Oct 25 21:32:24 2016 +0200 @@ -1,10 +1,10 @@ #!/bin/sh psql -U postgres -h localhost -c 'drop database if exists kallithea;' psql -U postgres -h localhost -c 'create database kallithea;' -paster setup-db server.ini --force-yes --user=username --password=qweqwe --email=username@example.com --repos=/home/username/repos --no-public-access +gearbox setup-db -c server.ini --force-yes --user=username --password=qweqwe --email=username@example.com --repos=/home/username/repos --no-public-access API_KEY=`psql -R " " -A -U postgres -h localhost -c "select api_key from users where admin=TRUE" -d kallithea | awk '{print $2}'` echo "run those after running server" -paster serve server.ini --pid-file=server.pid --daemon +gearbox serve -c server.ini --pid-file=server.pid --daemon sleep 3 kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo1 password:qweqwe email:demo1@example.com kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo2 password:qweqwe email:demo2@example.com
--- a/kallithea/tests/test.ini Sat Dec 24 01:27:47 2016 +0100 +++ b/kallithea/tests/test.ini Tue Oct 25 21:32:24 2016 +0200 @@ -65,8 +65,8 @@ #smtp_use_tls = false [server:main] -## PASTE ## -#use = egg:Paste#http +## Gearbox default web server ## +#use = egg:gearbox#wsgiref ## nr of worker threads to spawn #threadpool_workers = 1 ## max request before thread respawn @@ -74,6 +74,9 @@ ## option to use threads of process #use_threadpool = true +## Gearbox gevent web server ## +#use = egg:gearbox#gevent + ## WAITRESS ## use = egg:waitress#main ## number of worker threads @@ -514,7 +517,7 @@ ################################ [loggers] -keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer +keys = root, routes, kallithea, sqlalchemy, gearbox, beaker, templates, whoosh_indexer [handlers] keys = console, console_sql @@ -555,6 +558,12 @@ qualname = kallithea propagate = 1 +[logger_gearbox] +level = DEBUG +handlers = +qualname = gearbox +propagate = 1 + [logger_sqlalchemy] level = INFO handlers = console_sql
--- a/kallithea/websetup.py Sat Dec 24 01:27:47 2016 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,51 +0,0 @@ -# -*- coding: utf-8 -*- -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see <http://www.gnu.org/licenses/>. -""" -kallithea.websetup -~~~~~~~~~~~~~~~~~~ - -Weboperations and setup for kallithea - -This file was forked by the Kallithea project in July 2014. -Original author and date, and relevant copyright and licensing information is below: -:created_on: Dec 11, 2010 -:author: marcink -:copyright: (c) 2013 RhodeCode GmbH, and others. -:license: GPLv3, see LICENSE.md for more details. -""" - -import logging - -from kallithea.config.environment import load_environment -from kallithea.lib.db_manage import DbManage -from kallithea.model.meta import Session - - -log = logging.getLogger(__name__) - - -def setup_app(command, conf, vars): - """Place any commands to setup kallithea here""" - dbconf = conf['sqlalchemy.url'] - dbmanage = DbManage(log_sql=True, dbconf=dbconf, root=conf['here'], - tests=False, cli_args=command.options.__dict__) - dbmanage.create_tables(override=True) - opts = dbmanage.config_prompt(None) - dbmanage.create_settings(opts) - dbmanage.create_default_user() - dbmanage.admin_prompt() - dbmanage.create_permissions() - dbmanage.populate_default_permissions() - Session().commit() - load_environment(conf.global_conf, conf.local_conf)
--- a/scripts/dbmigrate-test Sat Dec 24 01:27:47 2016 +0100 +++ b/scripts/dbmigrate-test Tue Oct 25 21:32:24 2016 +0200 @@ -104,7 +104,7 @@ cd "$temp/from" . "$temp/from-env/bin/activate" announce "Initializing database..." - quiet_if_ok paster setup-db "$config_file" --repos="$temp/repos" --user=doe --email=doe@example.com --password=123456 --no-public-access --force-yes + quiet_if_ok gearbox setup-db -c "$config_file" --repos="$temp/repos" --user=doe --email=doe@example.com --password=123456 --no-public-access --force-yes alembic -c "$config_file" current -v )
--- a/setup.py Sat Dec 24 01:27:47 2016 +0100 +++ b/setup.py Tue Oct 25 21:32:24 2016 +0200 @@ -36,6 +36,7 @@ requirements = [ "alembic>=0.8.0,<0.9", + "GearBox<1", "waitress>=0.8.8,<1.0", "webob>=1.7,<2", "Pylons>=1.0.0,<=1.0.2", @@ -132,7 +133,6 @@ url=__url__, install_requires=requirements, classifiers=classifiers, - setup_requires=['PasteScript>=1.6.3'], data_files=data_files, packages=packages, include_package_data=True, @@ -142,7 +142,6 @@ ('templates/**.html', 'mako', {'input_encoding': 'utf-8'}), ('public/**', 'ignore', None)]}, zip_safe=False, - paster_plugins=['PasteScript', 'Pylons'], entry_points=""" [console_scripts] kallithea-api = kallithea.bin.kallithea_api:main @@ -155,7 +154,7 @@ [paste.app_install] main = pylons.util:PylonsInstaller - [paste.global_paster_command] + [gearbox.commands] setup-db=kallithea.lib.paster_commands.setup_db:Command cleanup-repos=kallithea.lib.paster_commands.cleanup:Command update-repoinfo=kallithea.lib.paster_commands.update_repoinfo:Command