Mercurial > kallithea
annotate docs/dev/dbmigrations.rst @ 6554:2c3d30095d5e
gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack
This is a step towards moving away from the Pylons stack to TurboGears2, but
still independent of it.
Some notes from the porting - it could perhaps be the missing(?) documentation
for migrating from paster to gearbox:
Note: 'gearbox' without parameters will crash - specify '-h' to get started
testing.
Replace paster
summary = 'yada yada'
with the first line of the docstring of the Command class ... or override
get_description.
Note: All newlines in the docstring will be collapsed and mangle the long help
text.
Grouping of commands is not possible. Standard commands (for development) can't
be customized under the same name or hidden. (Like for paster, the conceptual
model also assumes that the sub-command naming is namespaced so commands from
other packages won't conflict.)
The usage help is fully automated from the declared options.
For all deprecated Commands, replace paster
hidden = True
with gearbox
deprecated = True
Note: config_file, takes_config_file, min_args and max_args are not available /
relevant.
The gearbox parser is customized by overriding get_parser - there is nothing
like paster update_parser.
Gearbox is using argparse instead of optparse ... but argparse add_argument is
mostly backwards compatible with optparse add_option.
Instead of overriding command or run as in paster, override take_action in
gearbox. The parsed arguments are passed to take_action, not available on the
command instance.
Paster BadCommand is not available and must be handled manually, terminating
with sys.exit(1).
There is no standard make-config command in gearbox.
Paster appinstall has been replaced by the somewhat different setup_app module
in gearbox. There is still no clean way to pass parameters to SetupAppCommand
and it relies on websetup and other apparently unnecessary complexity. Instead,
implement setup-db from scratch.
Minor change by Thomas De Schampheleire: add gearbox logging configuration.
Because we use logging.config.fileConfig(.inifile) during gearbox command
execution, the logging settings need to be correct and contain a block for
gearbox logging itself. Otherwise, errors in command processing are not even
visible and the command exits silently.
author | Mads Kiilerich <madski@unity3d.com> |
---|---|
date | Tue, 25 Oct 2016 21:32:24 +0200 |
parents | 9fd64dd2617d |
children | 213085032127 |
rev | line source |
---|---|
6023
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
1 ======================= |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
2 Database schema changes |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
3 ======================= |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
4 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
5 Kallithea uses Alembic for :ref:`database migrations <upgrade_db>` |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
6 (upgrades and downgrades). |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
7 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
8 If you are developing a Kallithea feature that requires database schema |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
9 changes, you should make a matching Alembic database migration script: |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
10 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
11 1. :ref:`Create a Kallithea configuration and database <setup>` for testing |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
12 the migration script, or use existing ``development.ini`` setup. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
13 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
14 Ensure that this database is up to date with the latest database |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
15 schema *before* the changes you're currently developing. (Do not |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
16 create the database while your new schema changes are applied.) |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
17 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
18 2. Create a separate throwaway configuration for iterating on the actual |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
19 database changes:: |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
20 |
6554
2c3d30095d5e
gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack
Mads Kiilerich <madski@unity3d.com>
parents:
6023
diff
changeset
|
21 TODO make-config Kallithea temp.ini |
6023
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
22 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
23 Edit the file to change database settings. SQLite is typically fine, |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
24 but make sure to change the path to e.g. ``temp.db``, to avoid |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
25 clobbering any existing database file. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
26 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
27 3. Make your code changes (including database schema changes in ``db.py``). |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
28 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
29 4. After every database schema change, recreate the throwaway database |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
30 to test the changes:: |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
31 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
32 rm temp.db |
6554
2c3d30095d5e
gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack
Mads Kiilerich <madski@unity3d.com>
parents:
6023
diff
changeset
|
33 gearbox setup-db -c temp.ini --repos=/var/repos --user=doe --email doe@example.com --password=123456 --no-public-access --force-yes |
2c3d30095d5e
gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack
Mads Kiilerich <madski@unity3d.com>
parents:
6023
diff
changeset
|
34 gearbox repo-scan -c temp.ini |
6023
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
35 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
36 5. Once satisfied with the schema changes, auto-generate a draft Alembic |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
37 script using the development database that has *not* been upgraded. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
38 (The generated script will upgrade the database to match the code.) |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
39 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
40 :: |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
41 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
42 alembic -c development.ini revision -m "area: add cool feature" --autogenerate |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
43 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
44 6. Edit the script to clean it up and fix any problems. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
45 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
46 Note that for changes that simply add columns, it may be appropriate |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
47 to not remove them in the downgrade script (and instead do nothing), |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
48 to avoid the loss of data. Unknown columns will simply be ignored by |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
49 Kallithea versions predating your changes. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
50 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
51 7. Run ``alembic -c development.ini upgrade head`` to apply changes to |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
52 the (non-throwaway) database, and test the upgrade script. Also test |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
53 downgrades. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
54 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
55 The included ``development.ini`` has full SQL logging enabled. If |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
56 you're using another configuration file, you may want to enable it |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
57 by setting ``level = DEBUG`` in section ``[handler_console_sql]``. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
58 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
59 The Alembic migration script should be committed in the same revision as |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
60 the database schema (``db.py``) changes. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
61 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
62 See the `Alembic documentation`__ for more information, in particular |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
63 the tutorial and the section about auto-generating migration scripts. |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
64 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
65 .. __: http://alembic.zzzcomputing.com/en/latest/ |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
66 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
67 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
68 Troubleshooting |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
69 --------------- |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
70 |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
71 * If ``alembic --autogenerate`` responds "Target database is not up to |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
72 date", you need to either first use Alembic to upgrade the database |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
73 to the most recent version (before your changes), or recreate the |
9fd64dd2617d
docs: document how to use Alembic for database migrations
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
74 database from scratch (without your schema changes applied). |