annotate scripts/validate-minimum-dependency-versions @ 8455:d727e81e0097 stable

vcs: fix cloning remote repository with HTTP authentication (Issue #379) Using a remote clone URI of http://user:pass@host/... triggered an exception: ... E File ".../kallithea/lib/utils.py", line 256, in is_valid_repo_uri E GitRepository._check_url(url) E File ".../kallithea/lib/vcs/backends/git/repository.py", line 183, in _check_url E passmgr.add_password(*authinfo) E File "/usr/lib/python3.7/urllib/request.py", line 848, in add_password E self.reduce_uri(u, default_port) for u in uri) E File "/usr/lib/python3.7/urllib/request.py", line 848, in <genexpr> E self.reduce_uri(u, default_port) for u in uri) E File "/usr/lib/python3.7/urllib/request.py", line 875, in reduce_uri E host, port = splitport(authority) E File "/usr/lib/python3.7/urllib/parse.py", line 1022, in splitport E match = _portprog.fullmatch(host) E TypeError: cannot use a string pattern on a bytes-like object The authinfo tuple is obtained via mercurial.util.url, which unfortunately returns a tuple of bytes whereas urllib expects strings. It seems that mercurial internally has some more hacking around urllib as urllibcompat.py, which we don't use. Therefore, transform the bytes into strings before passing authinfo to urllib. As the realm can be None, we need to check it specifically otherwise safe_str would return a string 'None'. A basic test that catches the mentioned problem is added, even though it does not actually test that cloning with auth info will actually work (it only tests that it fails cleanly if the URI is not reachable). Additionally, one use of 'test_uri' in hg/repository.py still needed to be transformed from bytes to string. For git this was already ok.
author Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
date Wed, 22 Jul 2020 21:55:57 +0200
parents 01aca0a4f876
children 0a9ddb8cd8c1
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7901
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
1 #!/bin/bash
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
2 # Test that installation of all dependencies works fine if versions are set to
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
3 # the minimum ones.
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
4
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
5 set -e
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
6
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
7 if [ -n "$VIRTUAL_ENV" ]; then
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
8 echo "This script will create its own virtualenv - please don't run it inside an existing one." >&2
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
9 exit 1
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
10 fi
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
11
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
12 cd "$(hg root)"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
13
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
14 venv=build/minimum-dependency-versions-venv
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
15 log=build/minimum-dependency-versions.log
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
16 min_requirements=build/minimum-dependency-versions-requirements.txt
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
17 echo "virtualenv: $venv"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
18 echo "log: $log"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
19 echo "minimum requirements file: $min_requirements"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
20
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
21 # clean up previous runs
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
22 rm -rf "$venv" "$log"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
23 mkdir -p "$venv"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
24
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
25 # Make a light weight parsing of setup.py and dev_requirements.txt,
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
26 # finding all >= requirements and dumping into a custom requirements.txt
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
27 # while fixating the requirement at the lower bound.
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
28 sed -n 's/.*"\(.*\)>=\(.*\)".*/\1==\2/p' setup.py > "$min_requirements"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
29 sed 's/>=/==/p' dev_requirements.txt >> "$min_requirements"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
30
8193
89e9aef9b983 py3: use "python3 -m venv" instead of virtualenv package
Mads Kiilerich <mads@kiilerich.com>
parents: 8173
diff changeset
31 python3 -m venv "$venv"
7901
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
32 source "$venv/bin/activate"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
33 pip install --upgrade pip setuptools
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
34 pip install -e . -r "$min_requirements" python-ldap python-pam 2> >(tee "$log" >&2)
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
35
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
36 # Treat any message on stderr as a problem, for the caller to interpret.
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
37 if [ -s "$log" ]; then
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
38 echo
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
39 echo "Error: pip detected following problems:"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
40 cat "$log"
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
41 echo
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
42 exit 1
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
43 fi
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
44
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
45 freeze_txt=build/minimum-dependency-versions.txt
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
46 pip freeze > $freeze_txt
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
47 echo "Installation of minimum packages was successful, providing a set of packages as in $freeze_txt . Now running test suite..."
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
48
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
49 pytest
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
50
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
51 echo "Test suite execution was successful."
ac6cc1b8a07e scripts: new maintainer script validate-minimum-dependency-versions
Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
parents:
diff changeset
52 echo "You can now do additional validation using virtual env '$venv'."