Mercurial > kallithea
annotate scripts/validate-minimum-dependency-versions @ 8455:d727e81e0097 stable
vcs: fix cloning remote repository with HTTP authentication (Issue #379)
Using a remote clone URI of
http://user:pass@host/...
triggered an exception:
...
E File ".../kallithea/lib/utils.py", line 256, in is_valid_repo_uri
E GitRepository._check_url(url)
E File ".../kallithea/lib/vcs/backends/git/repository.py", line 183, in _check_url
E passmgr.add_password(*authinfo)
E File "/usr/lib/python3.7/urllib/request.py", line 848, in add_password
E self.reduce_uri(u, default_port) for u in uri)
E File "/usr/lib/python3.7/urllib/request.py", line 848, in <genexpr>
E self.reduce_uri(u, default_port) for u in uri)
E File "/usr/lib/python3.7/urllib/request.py", line 875, in reduce_uri
E host, port = splitport(authority)
E File "/usr/lib/python3.7/urllib/parse.py", line 1022, in splitport
E match = _portprog.fullmatch(host)
E TypeError: cannot use a string pattern on a bytes-like object
The authinfo tuple is obtained via mercurial.util.url, which unfortunately
returns a tuple of bytes whereas urllib expects strings.
It seems that mercurial internally has some more hacking around urllib as
urllibcompat.py, which we don't use.
Therefore, transform the bytes into strings before passing authinfo to
urllib. As the realm can be None, we need to check it specifically otherwise
safe_str would return a string 'None'.
A basic test that catches the mentioned problem is added, even though it
does not actually test that cloning with auth info will actually work (it
only tests that it fails cleanly if the URI is not reachable).
Additionally, one use of 'test_uri' in hg/repository.py still needed to be
transformed from bytes to string. For git this was already ok.
author | Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> |
---|---|
date | Wed, 22 Jul 2020 21:55:57 +0200 |
parents | 01aca0a4f876 |
children | 0a9ddb8cd8c1 |
rev | line source |
---|---|
7901
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
1 #!/bin/bash |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
2 # Test that installation of all dependencies works fine if versions are set to |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
3 # the minimum ones. |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
4 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
5 set -e |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
6 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
7 if [ -n "$VIRTUAL_ENV" ]; then |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
8 echo "This script will create its own virtualenv - please don't run it inside an existing one." >&2 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
9 exit 1 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
10 fi |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
11 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
12 cd "$(hg root)" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
13 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
14 venv=build/minimum-dependency-versions-venv |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
15 log=build/minimum-dependency-versions.log |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
16 min_requirements=build/minimum-dependency-versions-requirements.txt |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
17 echo "virtualenv: $venv" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
18 echo "log: $log" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
19 echo "minimum requirements file: $min_requirements" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
20 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
21 # clean up previous runs |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
22 rm -rf "$venv" "$log" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
23 mkdir -p "$venv" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
24 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
25 # Make a light weight parsing of setup.py and dev_requirements.txt, |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
26 # finding all >= requirements and dumping into a custom requirements.txt |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
27 # while fixating the requirement at the lower bound. |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
28 sed -n 's/.*"\(.*\)>=\(.*\)".*/\1==\2/p' setup.py > "$min_requirements" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
29 sed 's/>=/==/p' dev_requirements.txt >> "$min_requirements" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
30 |
8193
89e9aef9b983
py3: use "python3 -m venv" instead of virtualenv package
Mads Kiilerich <mads@kiilerich.com>
parents:
8173
diff
changeset
|
31 python3 -m venv "$venv" |
7901
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
32 source "$venv/bin/activate" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
33 pip install --upgrade pip setuptools |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
34 pip install -e . -r "$min_requirements" python-ldap python-pam 2> >(tee "$log" >&2) |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
35 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
36 # Treat any message on stderr as a problem, for the caller to interpret. |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
37 if [ -s "$log" ]; then |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
38 echo |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
39 echo "Error: pip detected following problems:" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
40 cat "$log" |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
41 echo |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
42 exit 1 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
43 fi |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
44 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
45 freeze_txt=build/minimum-dependency-versions.txt |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
46 pip freeze > $freeze_txt |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
47 echo "Installation of minimum packages was successful, providing a set of packages as in $freeze_txt . Now running test suite..." |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
48 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
49 pytest |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
50 |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
51 echo "Test suite execution was successful." |
ac6cc1b8a07e
scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff
changeset
|
52 echo "You can now do additional validation using virtual env '$venv'." |