Mercurial > kallithea
annotate scripts/dbmigrate-test @ 6554:2c3d30095d5e
gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack
This is a step towards moving away from the Pylons stack to TurboGears2, but
still independent of it.
Some notes from the porting - it could perhaps be the missing(?) documentation
for migrating from paster to gearbox:
Note: 'gearbox' without parameters will crash - specify '-h' to get started
testing.
Replace paster
summary = 'yada yada'
with the first line of the docstring of the Command class ... or override
get_description.
Note: All newlines in the docstring will be collapsed and mangle the long help
text.
Grouping of commands is not possible. Standard commands (for development) can't
be customized under the same name or hidden. (Like for paster, the conceptual
model also assumes that the sub-command naming is namespaced so commands from
other packages won't conflict.)
The usage help is fully automated from the declared options.
For all deprecated Commands, replace paster
hidden = True
with gearbox
deprecated = True
Note: config_file, takes_config_file, min_args and max_args are not available /
relevant.
The gearbox parser is customized by overriding get_parser - there is nothing
like paster update_parser.
Gearbox is using argparse instead of optparse ... but argparse add_argument is
mostly backwards compatible with optparse add_option.
Instead of overriding command or run as in paster, override take_action in
gearbox. The parsed arguments are passed to take_action, not available on the
command instance.
Paster BadCommand is not available and must be handled manually, terminating
with sys.exit(1).
There is no standard make-config command in gearbox.
Paster appinstall has been replaced by the somewhat different setup_app module
in gearbox. There is still no clean way to pass parameters to SetupAppCommand
and it relies on websetup and other apparently unnecessary complexity. Instead,
implement setup-db from scratch.
Minor change by Thomas De Schampheleire: add gearbox logging configuration.
Because we use logging.config.fileConfig(.inifile) during gearbox command
execution, the logging settings need to be correct and contain a block for
gearbox logging itself. Otherwise, errors in command processing are not even
visible and the command exits silently.
author | Mads Kiilerich <madski@unity3d.com> |
---|---|
date | Tue, 25 Oct 2016 21:32:24 +0200 |
parents | e285bb7abb28 |
children | 3158cf0dafb7 |
rev | line source |
---|---|
6025
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
1 #!/bin/sh -e |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
2 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
3 if [ $# -lt 2 ] || [ $# -gt 3 ]; then |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
4 cat >&2 <<EOD |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
5 usage: $0 CONFIG_FILE FROM_REV [TO_REV] |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
6 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
7 Runs a database migration from FROM_REV to TO_REV (default: current |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
8 working directory parent), using the specified CONFIG_FILE (.ini file). |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
9 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
10 Test is run using a clean Kallithea install, in a temporary virtual |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
11 environment. FROM_REV and (optional) TO_REV should be Mercurial revision |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
12 identifiers (e.g. changeset hash or a version number tag). The working |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
13 directory is not touched, but the database referenced in the config file |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
14 will be (re)created. |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
15 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
16 Only SQLite is available out of the box; for MySQL or PostgreSQL, set |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
17 the EXTRA environment variable to the required package(s), and it'll |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
18 be installed in the virtual environment. (E.g. EXTRA=MySQL-python or |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
19 EXTRA=psycopg2.) |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
20 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
21 The temporary directory is not removed, allowing follow-up examination |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
22 of the upgrade results. It is, however, created in /tmp by default, |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
23 which many Linux distributions automatically clean at regular intervals. |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
24 EOD |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
25 exit 1 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
26 fi |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
27 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
28 config_file=$(readlink -f "$1") |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
29 from_rev=$2 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
30 to_rev=$3 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
31 source_repo=$(dirname "$(dirname "$(readlink -f "$0")")") |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
32 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
33 announce() { |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
34 echo |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
35 echo "$1" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
36 echo |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
37 } |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
38 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
39 quiet_if_ok() ( |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
40 local output |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
41 local st |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
42 set +e |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
43 output=$("$@" < /dev/null 2>&1) |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
44 st=$? |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
45 if [ $st -ne 0 ]; then |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
46 echo "$output" >&2 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
47 echo "Command $@ returned exit status $st." >&2 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
48 exit 1 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
49 fi |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
50 ) |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
51 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
52 HG() { |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
53 "${HG:-hg}" --repository "$source_repo" "$@" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
54 } |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
55 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
56 # If upgrading to "current revision", warn if working directory is dirty. |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
57 if [ ! "$to_rev" ] && [ "$(HG status -mard)" ]; then |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
58 announce "Warning: Uncommitted changes in working directory will be ignored!" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
59 fi |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
60 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
61 from_rev_hash=$(HG id --id --rev "${from_rev:-.}") |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
62 to_rev_hash=$(HG id --id --rev "${to_rev:-.}") |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
63 temp=$(readlink -f "$(mktemp --tmpdir -d 'dbmigrate-test.XXXXXX')") |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
64 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
65 cat <<EOD |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
66 Config file: $config_file |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
67 EOD |
6214
f973b866fffc
Turbogears2 migration: use sqlalchemy.url iso sqlalchemy.db1.url
Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
parents:
6025
diff
changeset
|
68 sed -n -e 's/^sqlalchemy\.url *= */Database URL: /p' "$config_file" |
6025
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
69 cat <<EOD |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
70 Working dir: $temp |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
71 Repository: $source_repo |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
72 Upgrade from: $from_rev_hash (${from_rev:-current}) |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
73 Upgrade to: $to_rev_hash (${to_rev:-current}) |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
74 Extra packages: ${EXTRA:-(none)} |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
75 EOD |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
76 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
77 mkdir "$temp/repos" # empty |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
78 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
79 # Enable caching for old pip versions (this will cache the pip upgrade) |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
80 # Newer pip versions cache automatically, and don't use this variable. |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
81 if [ ! "$PIP_DOWNLOAD_CACHE" ]; then |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
82 export PIP_DOWNLOAD_CACHE=$HOME/.cache/pip/legacy |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
83 fi |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
84 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
85 install_kallithea() { |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
86 local prefix=$1 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
87 local rev=$2 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
88 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
89 announce "Installing Kallithea $rev in $prefix..." |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
90 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
91 "${VIRTUALENV:-virtualenv}" --quiet "$prefix-env" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
92 HG archive --rev "$rev" "$prefix" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
93 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
94 ( |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
95 cd "$prefix" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
96 . "$prefix-env/bin/activate" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
97 pip install --quiet --upgrade pip setuptools mercurial $EXTRA |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
98 pip install --quiet -e . |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
99 ) |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
100 } |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
101 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
102 install_kallithea "$temp/from" "$from_rev_hash" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
103 ( |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
104 cd "$temp/from" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
105 . "$temp/from-env/bin/activate" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
106 announce "Initializing database..." |
6554
2c3d30095d5e
gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack
Mads Kiilerich <madski@unity3d.com>
parents:
6370
diff
changeset
|
107 quiet_if_ok gearbox setup-db -c "$config_file" --repos="$temp/repos" --user=doe --email=doe@example.com --password=123456 --no-public-access --force-yes |
6025
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
108 alembic -c "$config_file" current -v |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
109 ) |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
110 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
111 install_kallithea "$temp/to" "$to_rev_hash" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
112 ( |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
113 cd "$temp/to" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
114 . "$temp/to-env/bin/activate" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
115 |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
116 announce "Commencing database upgrade from shown Alembic revision to head..." |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
117 alembic -c "$config_file" current -v |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
118 alembic -c "$config_file" upgrade head |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
119 announce "Upgrade complete, now at the shown Alembic revision:" |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
120 alembic -c "$config_file" current -v |
51b1af58589b
db: script to test database upgrades
Søren Løvborg <sorenl@unity3d.com>
parents:
diff
changeset
|
121 ) |