Mercurial > kallithea

annotate docs/dev/dbmigrations.rst @ 6026:dd676fdeda0f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
setup: move test dependencies to dev_requirements.txt to make them optional Remove the need for having test tools on production systems. Installing test dependencies is made an extra explicit step. pip is the future, but doesn't have the same tests_require features as setuptools kind of has. I don't like this way of handling it without setup.py support and with explicit naming of the ugly dev_requirements.txt ... but that seems to be the way to do it.
author Mads Kiilerich <madski@unity3d.com>
date Thu, 28 Jul 2016 16:28:34 +0200
parents 9fd64dd2617d
children 2c3d30095d5e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6023
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
1 =======================
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
2 Database schema changes
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
3 =======================
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
4
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
5 Kallithea uses Alembic for :ref:`database migrations <upgrade_db>`
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
6 (upgrades and downgrades).
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
7
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
8 If you are developing a Kallithea feature that requires database schema
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
9 changes, you should make a matching Alembic database migration script:
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
10
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
11 1. :ref:`Create a Kallithea configuration and database <setup>` for testing
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
12 the migration script, or use existing ``development.ini`` setup.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
13
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
14 Ensure that this database is up to date with the latest database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
15 schema *before* the changes you're currently developing. (Do not
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
16 create the database while your new schema changes are applied.)
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
17
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
18 2. Create a separate throwaway configuration for iterating on the actual
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
19 database changes::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
20
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
21 paster make-config Kallithea temp.ini
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
22
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
23 Edit the file to change database settings. SQLite is typically fine,
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
24 but make sure to change the path to e.g. ``temp.db``, to avoid
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
25 clobbering any existing database file.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
26
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
27 3. Make your code changes (including database schema changes in ``db.py``).
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
28
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
29 4. After every database schema change, recreate the throwaway database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
30 to test the changes::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
31
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
32 rm temp.db
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
33 paster setup-db temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
34 paster repo-scan temp.ini
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
35
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
36 5. Once satisfied with the schema changes, auto-generate a draft Alembic
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
37 script using the development database that has *not* been upgraded.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
38 (The generated script will upgrade the database to match the code.)
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
39
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
40 ::
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
41
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
42 alembic -c development.ini revision -m "area: add cool feature" --autogenerate
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
43
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
44 6. Edit the script to clean it up and fix any problems.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
45
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
46 Note that for changes that simply add columns, it may be appropriate
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
47 to not remove them in the downgrade script (and instead do nothing),
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
48 to avoid the loss of data. Unknown columns will simply be ignored by
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
49 Kallithea versions predating your changes.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
50
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
51 7. Run ``alembic -c development.ini upgrade head`` to apply changes to
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
52 the (non-throwaway) database, and test the upgrade script. Also test
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
53 downgrades.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
54
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
55 The included ``development.ini`` has full SQL logging enabled. If
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
56 you're using another configuration file, you may want to enable it
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
57 by setting ``level = DEBUG`` in section ``[handler_console_sql]``.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
58
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
59 The Alembic migration script should be committed in the same revision as
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
60 the database schema (``db.py``) changes.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
61
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
62 See the `Alembic documentation`__ for more information, in particular
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
63 the tutorial and the section about auto-generating migration scripts.
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
64
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
65 .. __: http://alembic.zzzcomputing.com/en/latest/
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
66
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
67
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
68 Troubleshooting
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
69 ---------------
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
70
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
71 * If ``alembic --autogenerate`` responds "Target database is not up to
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
72 date", you need to either first use Alembic to upgrade the database
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
73 to the most recent version (before your changes), or recreate the
9fd64dd2617d docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff changeset
74 database from scratch (without your schema changes applied).