Mercurial > kallithea
changeset 6023:9fd64dd2617d
docs: document how to use Alembic for database migrations
author | Søren Løvborg <sorenl@unity3d.com> |
---|---|
date | Mon, 25 Jul 2016 15:33:21 +0200 |
parents | e36bc85e3ecb |
children | 1a080d4e926e |
files | docs/dev/dbmigrations.rst docs/index.rst docs/upgrade.rst kallithea/lib/dbmigrate/__init__.py |
diffstat | 4 files changed, 133 insertions(+), 2 deletions(-) [+] |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/dev/dbmigrations.rst Mon Jul 25 15:33:21 2016 +0200 @@ -0,0 +1,74 @@ +======================= +Database schema changes +======================= + +Kallithea uses Alembic for :ref:`database migrations <upgrade_db>` +(upgrades and downgrades). + +If you are developing a Kallithea feature that requires database schema +changes, you should make a matching Alembic database migration script: + +1. :ref:`Create a Kallithea configuration and database <setup>` for testing + the migration script, or use existing ``development.ini`` setup. + + Ensure that this database is up to date with the latest database + schema *before* the changes you're currently developing. (Do not + create the database while your new schema changes are applied.) + +2. Create a separate throwaway configuration for iterating on the actual + database changes:: + + paster make-config Kallithea temp.ini + + Edit the file to change database settings. SQLite is typically fine, + but make sure to change the path to e.g. ``temp.db``, to avoid + clobbering any existing database file. + +3. Make your code changes (including database schema changes in ``db.py``). + +4. After every database schema change, recreate the throwaway database + to test the changes:: + + rm temp.db + paster setup-db temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes + paster repo-scan temp.ini + +5. Once satisfied with the schema changes, auto-generate a draft Alembic + script using the development database that has *not* been upgraded. + (The generated script will upgrade the database to match the code.) + + :: + + alembic -c development.ini revision -m "area: add cool feature" --autogenerate + +6. Edit the script to clean it up and fix any problems. + + Note that for changes that simply add columns, it may be appropriate + to not remove them in the downgrade script (and instead do nothing), + to avoid the loss of data. Unknown columns will simply be ignored by + Kallithea versions predating your changes. + +7. Run ``alembic -c development.ini upgrade head`` to apply changes to + the (non-throwaway) database, and test the upgrade script. Also test + downgrades. + + The included ``development.ini`` has full SQL logging enabled. If + you're using another configuration file, you may want to enable it + by setting ``level = DEBUG`` in section ``[handler_console_sql]``. + +The Alembic migration script should be committed in the same revision as +the database schema (``db.py``) changes. + +See the `Alembic documentation`__ for more information, in particular +the tutorial and the section about auto-generating migration scripts. + +.. __: http://alembic.zzzcomputing.com/en/latest/ + + +Troubleshooting +--------------- + +* If ``alembic --autogenerate`` responds "Target database is not up to + date", you need to either first use Alembic to upgrade the database + to the most recent version (before your changes), or recreate the + database from scratch (without your schema changes applied).
--- a/docs/index.rst Tue May 24 12:02:23 2016 +0200 +++ b/docs/index.rst Mon Jul 25 15:33:21 2016 +0200 @@ -54,6 +54,7 @@ contributing dev/translation + dev/dbmigrations changelog **API**
--- a/docs/upgrade.rst Tue May 24 12:02:23 2016 +0200 +++ b/docs/upgrade.rst Mon Jul 25 15:33:21 2016 +0200 @@ -103,11 +103,64 @@ Please always make sure your ``.ini`` files are up to date. Errors can often be caused by missing parameters added in new versions. +.. _upgrade_db: + 6. Upgrade your database ------------------------ -Not required. +.. note:: + If you are *downgrading* Kallithea, you should perform the database + migration step *before* installing the older version. (That is, + always perform migrations using the most recent of the two versions + you're migrating between.) + +First, run the following command to see your current database version:: + + alembic -c my.ini current + +Typical output will be something like "9358dc3d6828 (head)", which is +the current Alembic database "revision ID". Write down the entire output +for troubleshooting purposes. + +The output will be empty if you're upgrading from Kallithea 0.3.x or +older. That's expected. If you get an error that the config file was not +found or has no ``[alembic]`` section, see the next section. + +Next, if you are performing an *upgrade*: Run the following command to +upgrade your database to the current Kallithea version:: + + alembic -c my.ini upgrade head + +If you are performing a *downgrade*: Run the following command to +downgrade your database to the given version:: + + alembic -c my.ini downgrade 0.4 + +Alembic will show the necessary migrations (if any) as it executes them. +If no "ERROR" is displayed, the command was successful. + +Should an error occur, the database may be "stranded" half-way +through the migration, and you should restore it from backup. + +Enabling old Kallithea config files for Alembic use +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Kallithea configuration files created before the introduction of Alembic +(i.e. predating Kallithea 0.4) need to be updated for use with Alembic. +Without this, Alembic will fail with an error like this:: + + FAILED: No config file 'my.ini' found, or file has no '[alembic]' section + +If Alembic complains specifically about a missing ``alembic.ini``, it is +likely because you did not specify a config file using the ``-c`` option. +On the other hand, if the mentioned config file actually exists, you +need to append the following lines to it:: + + [alembic] + script_location = kallithea:alembic + +Your config file should now work with Alembic. 7. Rebuild the Whoosh full-text index
--- a/kallithea/lib/dbmigrate/__init__.py Tue May 24 12:02:23 2016 +0200 +++ b/kallithea/lib/dbmigrate/__init__.py Mon Jul 25 15:33:21 2016 +0200 @@ -5,4 +5,7 @@ summary = '(removed)' def run(self, args): - raise SystemExit('The "paster upgrade-db" command has been removed.') + raise SystemExit( + 'The "paster upgrade-db" command has been removed; please see the docs:\n' + ' https://kallithea.readthedocs.io/en/default/upgrade.html' + )