Mercurial > kallithea

annotate rhodecode/lib/dbmigrate/migrate/versioning/migrate_repository.py @ 833:9753e0907827 beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added dbmigrate package, added model changes moved out upgrade db command to that package
author Marcin Kuzminski <marcin@python-works.com>
date Sat, 11 Dec 2010 01:54:12 +0100
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
833
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
1 """
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
2 Script to migrate repository from sqlalchemy <= 0.4.4 to the new
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
3 repository schema. This shouldn't use any other migrate modules, so
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
4 that it can work in any version.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
5 """
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
6
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
7 import os
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
8 import sys
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
9 import logging
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
10
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
11 log = logging.getLogger(__name__)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
12
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
13
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
14 def usage():
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
15 """Gives usage information."""
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
16 print """Usage: %(prog)s repository-to-migrate
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
17
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
18 Upgrade your repository to the new flat format.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
19
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
20 NOTE: You should probably make a backup before running this.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
21 """ % {'prog': sys.argv[0]}
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
22
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
23 sys.exit(1)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
24
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
25
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
26 def delete_file(filepath):
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
27 """Deletes a file and prints a message."""
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
28 log.info('Deleting file: %s' % filepath)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
29 os.remove(filepath)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
30
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
31
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
32 def move_file(src, tgt):
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
33 """Moves a file and prints a message."""
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
34 log.info('Moving file %s to %s' % (src, tgt))
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
35 if os.path.exists(tgt):
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
36 raise Exception(
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
37 'Cannot move file %s because target %s already exists' % \
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
38 (src, tgt))
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
39 os.rename(src, tgt)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
40
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
41
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
42 def delete_directory(dirpath):
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
43 """Delete a directory and print a message."""
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
44 log.info('Deleting directory: %s' % dirpath)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
45 os.rmdir(dirpath)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
46
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
47
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
48 def migrate_repository(repos):
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
49 """Does the actual migration to the new repository format."""
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
50 log.info('Migrating repository at: %s to new format' % repos)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
51 versions = '%s/versions' % repos
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
52 dirs = os.listdir(versions)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
53 # Only use int's in list.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
54 numdirs = [int(dirname) for dirname in dirs if dirname.isdigit()]
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
55 numdirs.sort() # Sort list.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
56 for dirname in numdirs:
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
57 origdir = '%s/%s' % (versions, dirname)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
58 log.info('Working on directory: %s' % origdir)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
59 files = os.listdir(origdir)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
60 files.sort()
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
61 for filename in files:
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
62 # Delete compiled Python files.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
63 if filename.endswith('.pyc') or filename.endswith('.pyo'):
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
64 delete_file('%s/%s' % (origdir, filename))
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
65
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
66 # Delete empty __init__.py files.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
67 origfile = '%s/__init__.py' % origdir
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
68 if os.path.exists(origfile) and len(open(origfile).read()) == 0:
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
69 delete_file(origfile)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
70
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
71 # Move sql upgrade scripts.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
72 if filename.endswith('.sql'):
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
73 version, dbms, operation = filename.split('.', 3)[0:3]
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
74 origfile = '%s/%s' % (origdir, filename)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
75 # For instance: 2.postgres.upgrade.sql ->
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
76 # 002_postgres_upgrade.sql
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
77 tgtfile = '%s/%03d_%s_%s.sql' % (
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
78 versions, int(version), dbms, operation)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
79 move_file(origfile, tgtfile)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
80
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
81 # Move Python upgrade script.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
82 pyfile = '%s.py' % dirname
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
83 pyfilepath = '%s/%s' % (origdir, pyfile)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
84 if os.path.exists(pyfilepath):
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
85 tgtfile = '%s/%03d.py' % (versions, int(dirname))
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
86 move_file(pyfilepath, tgtfile)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
87
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
88 # Try to remove directory. Will fail if it's not empty.
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
89 delete_directory(origdir)
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
90
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
91
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
92 def main():
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
93 """Main function to be called when using this script."""
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
94 if len(sys.argv) != 2:
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
95 usage()
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
96 migrate_repository(sys.argv[1])
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
97
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
98
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
99 if __name__ == '__main__':
9753e0907827 added dbmigrate package, added model changes
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
100 main()