Mercurial > kallithea
annotate rhodecode/bin/rhodecode_backup.py @ 3574:000653f7cdf9 beta
avoid displaying repr of internal classes in user facing messages
The context of the message will tell where the problem was and there is no reason to show
... does not exist for this repository <MercurialRepository at /home/marcink/repos/rhodecode>
author | Mads Kiilerich <madski@unity3d.com> |
---|---|
date | Thu, 21 Mar 2013 11:17:01 +0100 |
parents | 7ac09514a178 |
children | 63e58ef80ef1 |
rev | line source |
---|---|
1307 | 1 # -*- coding: utf-8 -*- |
2 """ | |
2379
7ac09514a178
created rhodecode-api binary script for working with api via cli
Marcin Kuzminski <marcin@python-works.com>
parents:
1824
diff
changeset
|
3 rhodecode.bin.backup_manager |
1307 | 4 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
5 |
2379
7ac09514a178
created rhodecode-api binary script for working with api via cli
Marcin Kuzminski <marcin@python-works.com>
parents:
1824
diff
changeset
|
6 Repositories backup manager, it allows to backups all |
1307 | 7 repositories and send it to backup server using RSA key via ssh. |
8 | |
9 :created_on: Feb 28, 2010 | |
1824
89efedac4e6c
2012 copyrights
Marcin Kuzminski <marcin@python-works.com>
parents:
1818
diff
changeset
|
10 :author: marcink |
89efedac4e6c
2012 copyrights
Marcin Kuzminski <marcin@python-works.com>
parents:
1818
diff
changeset
|
11 :copyright: (C) 2010-2012 Marcin Kuzminski <marcin@python-works.com> |
1532
2afe9320d5e6
updated docstrings
Marcin Kuzminski <marcin@python-works.com>
parents:
1307
diff
changeset
|
12 :license: GPLv3, see COPYING for more details. |
1307 | 13 """ |
1206
a671db5bdd58
fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents:
1203
diff
changeset
|
14 # This program is free software: you can redistribute it and/or modify |
a671db5bdd58
fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents:
1203
diff
changeset
|
15 # it under the terms of the GNU General Public License as published by |
a671db5bdd58
fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents:
1203
diff
changeset
|
16 # the Free Software Foundation, either version 3 of the License, or |
a671db5bdd58
fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents:
1203
diff
changeset
|
17 # (at your option) any later version. |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
18 # |
252
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
19 # This program is distributed in the hope that it will be useful, |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
20 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
21 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
22 # GNU General Public License for more details. |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
23 # |
252
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
24 # You should have received a copy of the GNU General Public License |
1206
a671db5bdd58
fixed license issue #149
Marcin Kuzminski <marcin@python-works.com>
parents:
1203
diff
changeset
|
25 # along with this program. If not, see <http://www.gnu.org/licenses/>. |
252
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
26 |
1307 | 27 import os |
28 import sys | |
252
3782a6d698af
licensing updates, code cleanups
Marcin Kuzminski <marcin@python-works.com>
parents:
241
diff
changeset
|
29 |
25 | 30 import logging |
31 import tarfile | |
32 import datetime | |
28 | 33 import subprocess |
1307 | 34 |
33
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
35 logging.basicConfig(level=logging.DEBUG, |
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
36 format="%(asctime)s %(levelname)-5.5s %(message)s") |
25 | 37 |
1307 | 38 |
25 | 39 class BackupManager(object): |
434
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
40 def __init__(self, repos_location, rsa_key, backup_server): |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
41 today = datetime.datetime.now().weekday() + 1 |
2379
7ac09514a178
created rhodecode-api binary script for working with api via cli
Marcin Kuzminski <marcin@python-works.com>
parents:
1824
diff
changeset
|
42 self.backup_file_name = "rhodecode_repos.%s.tar.gz" % today |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
43 |
434
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
44 self.id_rsa_path = self.get_id_rsa(rsa_key) |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
45 self.repos_path = self.get_repos_path(repos_location) |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
46 self.backup_server = backup_server |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
47 |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
48 self.backup_file_path = '/tmp' |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
49 |
25 | 50 logging.info('starting backup for %s', self.repos_path) |
51 logging.info('backup target %s', self.backup_file_path) | |
52 | |
434
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
53 def get_id_rsa(self, rsa_key): |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
54 if not os.path.isfile(rsa_key): |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
55 logging.error('Could not load id_rsa key file in %s', rsa_key) |
28 | 56 sys.exit() |
440
9fe5ca6d72f6
fixed stupid typo in backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
434
diff
changeset
|
57 return rsa_key |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
58 |
434
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
59 def get_repos_path(self, path): |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
60 if not os.path.isdir(path): |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
61 logging.error('Wrong location for repositories in %s', path) |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
62 sys.exit() |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
63 return path |
25 | 64 |
65 def backup_repos(self): | |
28 | 66 bckp_file = os.path.join(self.backup_file_path, self.backup_file_name) |
25 | 67 tar = tarfile.open(bckp_file, "w:gz") |
68 | |
34
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
69 for dir_name in os.listdir(self.repos_path): |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
70 logging.info('backing up %s', dir_name) |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
71 tar.add(os.path.join(self.repos_path, dir_name), dir_name) |
25 | 72 tar.close() |
26 | 73 logging.info('finished backup of mercurial repositories') |
25 | 74 |
28 | 75 def transfer_files(self): |
76 params = { | |
77 'id_rsa_key': self.id_rsa_path, | |
1307 | 78 'backup_file': os.path.join(self.backup_file_path, |
434
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
79 self.backup_file_name), |
1307 | 80 'backup_server': self.backup_server |
28 | 81 } |
39 | 82 cmd = ['scp', '-l', '40000', '-i', '%(id_rsa_key)s' % params, |
434
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
83 '%(backup_file)s' % params, |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
84 '%(backup_server)s' % params] |
28 | 85 |
35
c92f827280a3
Changed to blocking operation
Marcin Kuzminski <marcin@python-blog.com>
parents:
34
diff
changeset
|
86 subprocess.call(cmd) |
28 | 87 logging.info('Transfered file %s to %s', self.backup_file_name, cmd[4]) |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
88 |
33
22b2234e51cd
Added removing of files from data
Marcin Kuzminski <marcin@python-blog.com>
parents:
29
diff
changeset
|
89 def rm_file(self): |
36
846e2d8672c4
Added message when file removing
Marcin Kuzminski <marcin@python-blog.com>
parents:
35
diff
changeset
|
90 logging.info('Removing file %s', self.backup_file_name) |
34
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
91 os.remove(os.path.join(self.backup_file_path, self.backup_file_name)) |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
92 |
25 | 93 if __name__ == "__main__": |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
94 |
434
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
95 repo_location = '/home/repo_path' |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
96 backup_server = 'root@192.168.1.100:/backups/mercurial' |
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
97 rsa_key = '/home/id_rsa' |
1203
6832ef664673
source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents:
902
diff
changeset
|
98 |
434
bd3f341cad45
simplified and updated backup manager
Marcin Kuzminski <marcin@python-works.com>
parents:
252
diff
changeset
|
99 B_MANAGER = BackupManager(repo_location, rsa_key, backup_server) |
34
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
100 B_MANAGER.backup_repos() |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
101 B_MANAGER.transfer_files() |
b4b25ece7797
Code refactoring, and changed proper way of removing files
Marcin Kuzminski <marcin@python-blog.com>
parents:
33
diff
changeset
|
102 B_MANAGER.rm_file() |