Mercurial > kallithea

annotate docs/dev/dbmigrations.rst @ 7572:dcececb1f53e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
i18n: regenerate translations (according to instructions from kallithea/i18n/how_to)
author Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
date Mon, 11 Mar 2019 20:51:31 +0100
parents 3158cf0dafb7
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6023
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
1 =======================
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
2 Database schema changes
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
3 =======================
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
4
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
5 Kallithea uses Alembic for :ref:`database migrations <upgrade_db>`
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
6 (upgrades and downgrades).
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
7
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
8 If you are developing a Kallithea feature that requires database schema
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
9 changes, you should make a matching Alembic database migration script:
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
10
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
11 1. :ref:`Create a Kallithea configuration and database <setup>` for testing
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
12 the migration script, or use existing ``development.ini`` setup.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
13
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
14 Ensure that this database is up to date with the latest database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
15 schema *before* the changes you're currently developing. (Do not
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
16 create the database while your new schema changes are applied.)
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
17
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
18 2. Create a separate throwaway configuration for iterating on the actual
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
19 database changes::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
20
7406
7784a1212471 cli: convert 'gearbox make-config' into 'kallithea-cli config-create'
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6555
diff changeset
21 kallithea-cli config-create temp.ini
6023
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
22
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
23 Edit the file to change database settings. SQLite is typically fine,
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
24 but make sure to change the path to e.g. ``temp.db``, to avoid
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
25 clobbering any existing database file.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
26
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
27 3. Make your code changes (including database schema changes in ``db.py``).
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
28
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
29 4. After every database schema change, recreate the throwaway database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
30 to test the changes::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
31
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
32 rm temp.db
7414
3158cf0dafb7 cli: convert 'gearbox setup-db' into 'kallithea-cli db-create'
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 7408
diff changeset
33 kallithea-cli db-create -c temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes
7408
0080ffd8aea0 cli: convert 'gearbox repo-scan' into 'kallithea-cli repo-scan'
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 7406
diff changeset
34 kallithea-cli repo-scan -c temp.ini
6023
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
35
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
36 5. Once satisfied with the schema changes, auto-generate a draft Alembic
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
37 script using the development database that has *not* been upgraded.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
38 (The generated script will upgrade the database to match the code.)
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
39
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
40 ::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
41
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
42 alembic -c development.ini revision -m "area: add cool feature" --autogenerate
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
43
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
44 6. Edit the script to clean it up and fix any problems.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
45
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
46 Note that for changes that simply add columns, it may be appropriate
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
47 to not remove them in the downgrade script (and instead do nothing),
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
48 to avoid the loss of data. Unknown columns will simply be ignored by
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
49 Kallithea versions predating your changes.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
50
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
51 7. Run ``alembic -c development.ini upgrade head`` to apply changes to
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
52 the (non-throwaway) database, and test the upgrade script. Also test
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
53 downgrades.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
54
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
55 The included ``development.ini`` has full SQL logging enabled. If
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
56 you're using another configuration file, you may want to enable it
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
57 by setting ``level = DEBUG`` in section ``[handler_console_sql]``.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
58
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
59 The Alembic migration script should be committed in the same revision as
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
60 the database schema (``db.py``) changes.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
61
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
62 See the `Alembic documentation`__ for more information, in particular
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
63 the tutorial and the section about auto-generating migration scripts.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
64
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
65 .. __: http://alembic.zzzcomputing.com/en/latest/
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
66
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
67
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
68 Troubleshooting
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
69 ---------------
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
70
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
71 * If ``alembic --autogenerate`` responds "Target database is not up to
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
72 date", you need to either first use Alembic to upgrade the database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
73 to the most recent version (before your changes), or recreate the
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
74 database from scratch (without your schema changes applied).