Mercurial > kallithea
annotate docs/dev/dbmigrations.rst @ 6401:785a9770e8e0
templates: textarea doesn't have a size attribute - drop it!
We could use the rows attribute ... but so far it has worked without ...
author | Mads Kiilerich <mads@kiilerich.com> |
---|---|
date | Fri, 06 Jan 2017 01:43:50 +0100 |
parents | 9fd64dd2617d |
children | 2c3d30095d5e |
rev | line source |
---|---|
6023
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
1 ======================= |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
2 Database schema changes |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
3 ======================= |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
4 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
5 Kallithea uses Alembic for :ref:`database migrations <upgrade_db>` |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
6 (upgrades and downgrades). |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
7 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
8 If you are developing a Kallithea feature that requires database schema |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
9 changes, you should make a matching Alembic database migration script: |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
10 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
11 1. :ref:`Create a Kallithea configuration and database <setup>` for testing |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
12 the migration script, or use existing ``development.ini`` setup. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
13 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
14 Ensure that this database is up to date with the latest database |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
15 schema *before* the changes you're currently developing. (Do not |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
16 create the database while your new schema changes are applied.) |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
17 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
18 2. Create a separate throwaway configuration for iterating on the actual |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
19 database changes:: |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
20 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
21 paster make-config Kallithea temp.ini |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
22 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
23 Edit the file to change database settings. SQLite is typically fine, |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
24 but make sure to change the path to e.g. ``temp.db``, to avoid |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
25 clobbering any existing database file. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
26 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
27 3. Make your code changes (including database schema changes in ``db.py``). |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
28 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
29 4. After every database schema change, recreate the throwaway database |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
30 to test the changes:: |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
31 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
32 rm temp.db |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
33 paster setup-db temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
34 paster repo-scan temp.ini |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
35 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
36 5. Once satisfied with the schema changes, auto-generate a draft Alembic |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
37 script using the development database that has *not* been upgraded. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
38 (The generated script will upgrade the database to match the code.) |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
39 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
40 :: |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
41 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
42 alembic -c development.ini revision -m "area: add cool feature" --autogenerate |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
43 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
44 6. Edit the script to clean it up and fix any problems. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
45 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
46 Note that for changes that simply add columns, it may be appropriate |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
47 to not remove them in the downgrade script (and instead do nothing), |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
48 to avoid the loss of data. Unknown columns will simply be ignored by |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
49 Kallithea versions predating your changes. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
50 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
51 7. Run ``alembic -c development.ini upgrade head`` to apply changes to |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
52 the (non-throwaway) database, and test the upgrade script. Also test |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
53 downgrades. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
54 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
55 The included ``development.ini`` has full SQL logging enabled. If |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
56 you're using another configuration file, you may want to enable it |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
57 by setting ``level = DEBUG`` in section ``[handler_console_sql]``. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
58 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
59 The Alembic migration script should be committed in the same revision as |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
60 the database schema (``db.py``) changes. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
61 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
62 See the `Alembic documentation`__ for more information, in particular |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
63 the tutorial and the section about auto-generating migration scripts. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
64 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
65 .. __: http://alembic.zzzcomputing.com/en/latest/ |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
66 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
67 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
68 Troubleshooting |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
69 --------------- |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
70 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
71 * If ``alembic --autogenerate`` responds "Target database is not up to |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
72 date", you need to either first use Alembic to upgrade the database |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
73 to the most recent version (before your changes), or recreate the |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
74 database from scratch (without your schema changes applied). |