annotate docs/usage/vcs_notes.rst @ 8455:d727e81e0097 stable

vcs: fix cloning remote repository with HTTP authentication (Issue #379) Using a remote clone URI of http://user:pass@host/... triggered an exception: ... E File ".../kallithea/lib/utils.py", line 256, in is_valid_repo_uri E GitRepository._check_url(url) E File ".../kallithea/lib/vcs/backends/git/repository.py", line 183, in _check_url E passmgr.add_password(*authinfo) E File "/usr/lib/python3.7/urllib/request.py", line 848, in add_password E self.reduce_uri(u, default_port) for u in uri) E File "/usr/lib/python3.7/urllib/request.py", line 848, in <genexpr> E self.reduce_uri(u, default_port) for u in uri) E File "/usr/lib/python3.7/urllib/request.py", line 875, in reduce_uri E host, port = splitport(authority) E File "/usr/lib/python3.7/urllib/parse.py", line 1022, in splitport E match = _portprog.fullmatch(host) E TypeError: cannot use a string pattern on a bytes-like object The authinfo tuple is obtained via mercurial.util.url, which unfortunately returns a tuple of bytes whereas urllib expects strings. It seems that mercurial internally has some more hacking around urllib as urllibcompat.py, which we don't use. Therefore, transform the bytes into strings before passing authinfo to urllib. As the realm can be None, we need to check it specifically otherwise safe_str would return a string 'None'. A basic test that catches the mentioned problem is added, even though it does not actually test that cloning with auth info will actually work (it only tests that it fails cleanly if the URI is not reachable). Additionally, one use of 'test_uri' in hg/repository.py still needed to be transformed from bytes to string. For git this was already ok.
author Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
date Wed, 22 Jul 2020 21:55:57 +0200
parents 3158cf0dafb7
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7338
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
1 .. _vcs_notes:
3224
8b8edfc25856 whitespace cleanup
Marcin Kuzminski <marcin@python-works.com>
parents: 2992
diff changeset
2
7338
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
3 ===================================
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
4 Version control systems usage notes
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
5 ===================================
811
bb35ad076e2f docs updates
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
6
6734
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
7 .. _importing:
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
8
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
9
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
10 Importing existing repositories
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
11 -------------------------------
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
12
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
13 There are two main methods to import repositories in Kallithea: via the web
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
14 interface or via the filesystem. If you have a large number of repositories to
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
15 import, importing them via the filesystem is more convenient.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
16
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
17 Importing via web interface
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
18 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
19
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
20 For a small number of repositories, it may be easier to create the target
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
21 repositories through the Kallithea web interface, via *Admin > Repositories* or
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
22 via the *Add Repository* button on the entry page of the web interface.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
23
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
24 Repositories can be nested in repository groups by first creating the group (via
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
25 *Admin > Repository Groups* or via the *Add Repository Group* button on the
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
26 entry page of the web interface) and then selecting the appropriate group when
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
27 adding the repository.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
28
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
29 After creation of the (empty) repository, push the existing commits to the
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
30 *Clone URL* displayed on the repository summary page. For Git repositories,
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
31 first add the *Clone URL* as remote, then push the commits to that remote. The
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
32 specific commands to execute are shown under the *Existing repository?* section
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
33 of the new repository's summary page.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
34
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
35 A benefit of this method particular for Git repositories, is that the
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
36 Kallithea-specific Git hooks are installed automatically. For Mercurial, no
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
37 hooks are required anyway.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
38
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
39 Importing via the filesystem
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
40 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
41
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
42 The alternative method of importing repositories consists of creating the
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
43 repositories in the desired hierarchy on the filesystem and letting Kallithea
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
44 scan that location.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
45
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
46 All repositories are stored in a central location on the filesystem. This
7414
3158cf0dafb7 cli: convert 'gearbox setup-db' into 'kallithea-cli db-create'
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 7338
diff changeset
47 location is specified during installation (via ``db-create``) and can be reviewed
6734
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
48 at *Admin > Settings > VCS > Location of repositories*. Repository groups
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
49 (defined in *Admin > Repository Groups*) are represented by a directory in that
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
50 repository location. Repositories of the repository group are nested under that
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
51 directory.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
52
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
53 To import a set of repositories and organize them in a certain repository group
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
54 structure, first place clones in the desired hierarchy at the configured
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
55 repository location.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
56 These clones should be created without working directory. For Mercurial, this is
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
57 done with ``hg clone -U``, for Git with ``git clone --bare``.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
58
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
59 When the repositories are added correctly on the filesystem:
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
60
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
61 * go to *Admin > Settings > Remap and Rescan* in the Kallithea web interface
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
62 * select the *Install Git hooks* checkbox when importing Git repositories
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
63 * click *Rescan Repositories*
6789
4d04ac08fff7 docs: fix some sphinx warnings
Mads Kiilerich <mads@kiilerich.com>
parents: 6734
diff changeset
64
6734
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
65 This step will scan the filesystem and create the appropriate repository groups
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
66 and repositories in Kallithea.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
67
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
68 *Note*: Once repository groups have been created this way, manage their access
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
69 permissions through the Kallithea web interface.
c5512c9d2118 docs/vcs_support: add information on importing repositories
Karl Goetz <karl@kgoetz.id.au>
parents: 6554
diff changeset
70
5433
fbbe80e3322b docs: consistent spacing around headings
Mads Kiilerich <madski@unity3d.com>
parents: 5417
diff changeset
71
7338
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
72 Mercurial-specific notes
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
73 ------------------------
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
74
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
75
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
76 Working with subrepositories
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
77 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
78
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
79 This section explains how to use Mercurial subrepositories_ in Kallithea.
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
80
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
81 Example usage::
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
82
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
83 ## init a simple repo
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
84 hg init mainrepo
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
85 cd mainrepo
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
86 echo "file" > file
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
87 hg add file
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
88 hg ci --message "initial file"
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
89
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
90 # clone subrepo we want to add from Kallithea
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
91 hg clone http://kallithea.local/subrepo
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
92
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
93 ## specify URL to existing repo in Kallithea as subrepository path
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
94 echo "subrepo = http://kallithea.local/subrepo" > .hgsub
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
95 hg add .hgsub
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
96 hg ci --message "added remote subrepo"
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
97
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
98 In the file list of a clone of ``mainrepo`` you will see a connected
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
99 subrepository at the revision it was cloned with. Clicking on the
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
100 subrepository link sends you to the proper repository in Kallithea.
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
101
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
102 Cloning ``mainrepo`` will also clone the attached subrepository.
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
103
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
104 Next we can edit the subrepository data, and push back to Kallithea. This will
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
105 update both repositories.
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
106
2bb5e9ee49fe docs: split vcs_support into admin/vcs_setup and usage/vcs_notes
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents: 6789
diff changeset
107
4972
b52a1ccee927 docs/usage: reword information on Mercurial subrepository support
Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
parents: 4971
diff changeset
108 .. _subrepositories: http://mercurial.aragost.com/kick-start/en/subrepositories/