Mercurial > kallithea

annotate docs/dev/dbmigrations.rst @ 7450:1c4007ec86e8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
hg: fix URL cloning with Mercurial 4.6 and later In 03dfcbe52906, I didn't notice that the API also changed ... and it was not sufficiently tested. Now, instead of using the peer classes directly, use the instance wrapper. There is no automated testing of this, but it was tested manually to also work in the oldest supported Mercurial version.
author Mads Kiilerich <mads@kiilerich.com>
date Sun, 23 Dec 2018 21:16:06 +0100
parents 3158cf0dafb7
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6023
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
1 =======================
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
2 Database schema changes
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
3 =======================
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
4
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
5 Kallithea uses Alembic for :ref:`database migrations <upgrade_db>`
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
6 (upgrades and downgrades).
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
7
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
8 If you are developing a Kallithea feature that requires database schema
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
9 changes, you should make a matching Alembic database migration script:
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
10
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
11 1. :ref:`Create a Kallithea configuration and database <setup>` for testing
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
12 the migration script, or use existing ``development.ini`` setup.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
13
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
14 Ensure that this database is up to date with the latest database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
15 schema *before* the changes you're currently developing. (Do not
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
16 create the database while your new schema changes are applied.)
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
17
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
18 2. Create a separate throwaway configuration for iterating on the actual
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
19 database changes::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
20
7406
7784a1212471 cli: convert 'gearbox make-config' into 'kallithea-cli config-create'
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6555
diff changeset
21 kallithea-cli config-create temp.ini
6023
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
22
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
23 Edit the file to change database settings. SQLite is typically fine,
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
24 but make sure to change the path to e.g. ``temp.db``, to avoid
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
25 clobbering any existing database file.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
26
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
27 3. Make your code changes (including database schema changes in ``db.py``).
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
28
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
29 4. After every database schema change, recreate the throwaway database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
30 to test the changes::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
31
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
32 rm temp.db
7414
3158cf0dafb7 cli: convert 'gearbox setup-db' into 'kallithea-cli db-create'
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 7408
diff changeset
33 kallithea-cli db-create -c temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes
7408
0080ffd8aea0 cli: convert 'gearbox repo-scan' into 'kallithea-cli repo-scan'
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 7406
diff changeset
34 kallithea-cli repo-scan -c temp.ini
6023
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
35
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
36 5. Once satisfied with the schema changes, auto-generate a draft Alembic
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
37 script using the development database that has *not* been upgraded.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
38 (The generated script will upgrade the database to match the code.)
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
39
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
40 ::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
41
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
42 alembic -c development.ini revision -m "area: add cool feature" --autogenerate
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
43
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
44 6. Edit the script to clean it up and fix any problems.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
45
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
46 Note that for changes that simply add columns, it may be appropriate
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
47 to not remove them in the downgrade script (and instead do nothing),
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
48 to avoid the loss of data. Unknown columns will simply be ignored by
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
49 Kallithea versions predating your changes.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
50
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
51 7. Run ``alembic -c development.ini upgrade head`` to apply changes to
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
52 the (non-throwaway) database, and test the upgrade script. Also test
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
53 downgrades.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
54
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
55 The included ``development.ini`` has full SQL logging enabled. If
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
56 you're using another configuration file, you may want to enable it
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
57 by setting ``level = DEBUG`` in section ``[handler_console_sql]``.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
58
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
59 The Alembic migration script should be committed in the same revision as
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
60 the database schema (``db.py``) changes.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
61
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
62 See the `Alembic documentation`__ for more information, in particular
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
63 the tutorial and the section about auto-generating migration scripts.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
64
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
65 .. __: http://alembic.zzzcomputing.com/en/latest/
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
66
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
67
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
68 Troubleshooting
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
69 ---------------
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
70
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
71 * If ``alembic --autogenerate`` responds "Target database is not up to
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
72 date", you need to either first use Alembic to upgrade the database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
73 to the most recent version (before your changes), or recreate the
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
74 database from scratch (without your schema changes applied).