Mercurial > kallithea
annotate pylons_app/lib/backup_manager.py @ 421:cbfb853a0a4c
fixed bug when author had " in name, thus breaking the javascript data format
small fix to css for commit messages, and fixed a bug when message had some special chars that caused unicodeDecode error
author | Marcin Kuzminski <marcin@python-works.com> |
---|---|
date | Mon, 23 Aug 2010 21:39:17 +0200 |
parents | 3782a6d698af |
children | bd3f341cad45 |
rev | line source |
---|---|
252
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
1 #!/usr/bin/env python |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
2 # encoding: utf-8 |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
3 # mercurial repository backup manager |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
4 # Copyright (C) 2009-2010 Marcin Kuzminski <marcin@python-works.com> |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
5 |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
6 # This program is free software; you can redistribute it and/or |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
7 # modify it under the terms of the GNU General Public License |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
8 # as published by the Free Software Foundation; version 2 |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
9 # of the License or (at your opinion) any later version of the license. |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
10 # |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
11 # This program is distributed in the hope that it will be useful, |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
14 # GNU General Public License for more details. |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
15 # |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
16 # You should have received a copy of the GNU General Public License |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
17 # along with this program; if not, write to the Free Software |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
18 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
19 # MA 02110-1301, USA. |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
20 |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
21 """ |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
22 Created on Feb 28, 2010 |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
23 Mercurial repositories backup manager |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
24 @author: marcink |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
25 """ |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
26 |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
27 |
25 | 28 import logging |
29 from mercurial import config | |
30 import tarfile | |
31 import os | |
32 import datetime | |
33 import sys | |
28 | 34 import subprocess |
33
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
35 logging.basicConfig(level=logging.DEBUG, |
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
36 format="%(asctime)s %(levelname)-5.5s %(message)s") |
25 | 37 |
38 class BackupManager(object): | |
241
48727add84c9
Made repos path config configurable from pylons app configs. update Readme
Marcin Kuzminski <marcin@python-works.com>
parents:
39
diff
changeset
|
39 def __init__(self, id_rsa_path, repo_conf): |
34
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
40 self.repos_path = None |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
41 self.backup_file_name = None |
241
48727add84c9
Made repos path config configurable from pylons app configs. update Readme
Marcin Kuzminski <marcin@python-works.com>
parents:
39
diff
changeset
|
42 self.id_rsa_path = id_rsa_path |
28 | 43 self.check_id_rsa() |
27 | 44 cur_dir = os.path.realpath(__file__) |
25 | 45 dn = os.path.dirname |
27 | 46 self.backup_file_path = os.path.join(dn(dn(dn(cur_dir))), 'data') |
25 | 47 cfg = config.config() |
48 try: | |
241
48727add84c9
Made repos path config configurable from pylons app configs. update Readme
Marcin Kuzminski <marcin@python-works.com>
parents:
39
diff
changeset
|
49 cfg.read(os.path.join(dn(dn(dn(cur_dir))), repo_conf)) |
25 | 50 except IOError: |
241
48727add84c9
Made repos path config configurable from pylons app configs. update Readme
Marcin Kuzminski <marcin@python-works.com>
parents:
39
diff
changeset
|
51 logging.error('Could not read %s', repo_conf) |
25 | 52 sys.exit() |
53 self.set_repos_path(cfg.items('paths')) | |
54 logging.info('starting backup for %s', self.repos_path) | |
55 logging.info('backup target %s', self.backup_file_path) | |
56 | |
57 if not os.path.isdir(self.repos_path): | |
58 raise Exception('Not a valid directory in %s' % self.repos_path) | |
59 | |
28 | 60 def check_id_rsa(self): |
61 if not os.path.isfile(self.id_rsa_path): | |
34
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
62 logging.error('Could not load id_rsa key file in %s', |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
63 self.id_rsa_path) |
28 | 64 sys.exit() |
65 | |
25 | 66 def set_repos_path(self, paths): |
67 repos_path = paths[0][1].split('/') | |
68 if repos_path[-1] in ['*', '**']: | |
69 repos_path = repos_path[:-1] | |
70 if repos_path[0] != '/': | |
71 repos_path[0] = '/' | |
72 self.repos_path = os.path.join(*repos_path) | |
73 | |
74 def backup_repos(self): | |
75 today = datetime.datetime.now().weekday() + 1 | |
28 | 76 self.backup_file_name = "mercurial_repos.%s.tar.gz" % today |
77 bckp_file = os.path.join(self.backup_file_path, self.backup_file_name) | |
25 | 78 tar = tarfile.open(bckp_file, "w:gz") |
79 | |
34
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
80 for dir_name in os.listdir(self.repos_path): |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
81 logging.info('backing up %s', dir_name) |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
82 tar.add(os.path.join(self.repos_path, dir_name), dir_name) |
25 | 83 tar.close() |
26 | 84 logging.info('finished backup of mercurial repositories') |
25 | 85 |
86 | |
28 | 87 |
88 def transfer_files(self): | |
89 params = { | |
90 'id_rsa_key': self.id_rsa_path, | |
91 'backup_file_path':self.backup_file_path, | |
92 'backup_file_name':self.backup_file_name, | |
93 } | |
39 | 94 cmd = ['scp', '-l', '40000', '-i', '%(id_rsa_key)s' % params, |
28 | 95 '%(backup_file_path)s/%(backup_file_name)s' % params, |
96 'root@192.168.2.102:/backups/mercurial' % params] | |
97 | |
35
c92f827280a3
Changed to blocking operation
Marcin Kuzminski <marcin@python-blog.com>
parents:
34
diff
changeset
|
98 subprocess.call(cmd) |
28 | 99 logging.info('Transfered file %s to %s', self.backup_file_name, cmd[4]) |
33
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
100 |
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
101 |
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
102 def rm_file(self): |
36
846e2d8672c4
Added message when file removing
Marcin Kuzminski <marcin@python-blog.com>
parents:
35
diff
changeset
|
103 logging.info('Removing file %s', self.backup_file_name) |
34
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
104 os.remove(os.path.join(self.backup_file_path, self.backup_file_name)) |
33
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
105 |
28 | 106 |
107 | |
25 | 108 if __name__ == "__main__": |
241
48727add84c9
Made repos path config configurable from pylons app configs. update Readme
Marcin Kuzminski <marcin@python-works.com>
parents:
39
diff
changeset
|
109 B_MANAGER = BackupManager('/home/pylons/id_rsa', 'repositories.config') |
34
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
110 B_MANAGER.backup_repos() |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
111 B_MANAGER.transfer_files() |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
112 B_MANAGER.rm_file() |
25 | 113 |
114 |