Mercurial > kallithea
annotate scripts/generate-ini.py @ 8455:d727e81e0097 stable
vcs: fix cloning remote repository with HTTP authentication (Issue #379)
Using a remote clone URI of
http://user:pass@host/...
triggered an exception:
...
E File ".../kallithea/lib/utils.py", line 256, in is_valid_repo_uri
E GitRepository._check_url(url)
E File ".../kallithea/lib/vcs/backends/git/repository.py", line 183, in _check_url
E passmgr.add_password(*authinfo)
E File "/usr/lib/python3.7/urllib/request.py", line 848, in add_password
E self.reduce_uri(u, default_port) for u in uri)
E File "/usr/lib/python3.7/urllib/request.py", line 848, in <genexpr>
E self.reduce_uri(u, default_port) for u in uri)
E File "/usr/lib/python3.7/urllib/request.py", line 875, in reduce_uri
E host, port = splitport(authority)
E File "/usr/lib/python3.7/urllib/parse.py", line 1022, in splitport
E match = _portprog.fullmatch(host)
E TypeError: cannot use a string pattern on a bytes-like object
The authinfo tuple is obtained via mercurial.util.url, which unfortunately
returns a tuple of bytes whereas urllib expects strings.
It seems that mercurial internally has some more hacking around urllib as
urllibcompat.py, which we don't use.
Therefore, transform the bytes into strings before passing authinfo to
urllib. As the realm can be None, we need to check it specifically otherwise
safe_str would return a string 'None'.
A basic test that catches the mentioned problem is added, even though it
does not actually test that cloning with auth info will actually work (it
only tests that it fails cleanly if the URI is not reachable).
Additionally, one use of 'test_uri' in hg/repository.py still needed to be
transformed from bytes to string. For git this was already ok.
author | Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> |
---|---|
date | Wed, 22 Jul 2020 21:55:57 +0200 |
parents | 6eb1f66ac23f |
children | 495dea7c2a13 |
rev | line source |
---|---|
8173
aa6f17a53b49
py3: switch to use Python 3 interpreter, temporarily leaving many things very broken until they have been migrated/fixed in a reviewable way
Mads Kiilerich <mads@kiilerich.com>
parents:
7844
diff
changeset
|
1 #!/usr/bin/env python3 |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
2 """ |
7390
451b3f9d814e
docs: update i18n doc after TG migration changed lang to i18n.lang and test.ini is generated
Mads Kiilerich <mads@kiilerich.com>
parents:
7260
diff
changeset
|
3 Based on kallithea/lib/paster_commands/template.ini.mako, generate development.ini |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
4 """ |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
5 |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
6 import re |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
7 |
6884
e3cce237d77c
ini: extract .ini handling from scripts/generate-ini.py to kallithea/lib/inifile.py
Mads Kiilerich <mads@kiilerich.com>
parents:
6883
diff
changeset
|
8 from kallithea.lib import inifile |
e3cce237d77c
ini: extract .ini handling from scripts/generate-ini.py to kallithea/lib/inifile.py
Mads Kiilerich <mads@kiilerich.com>
parents:
6883
diff
changeset
|
9 |
7811
0a277465fddf
scripts: initial run of import cleanup using isort
Mads Kiilerich <mads@kiilerich.com>
parents:
7768
diff
changeset
|
10 |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
11 # files to be generated from the mako template |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
12 ini_files = [ |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
13 ('development.ini', |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
14 { |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
15 '[server:main]': { |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
16 'host': '0.0.0.0', |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
17 }, |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
18 '[app:main]': { |
6573
fc6b1b0e1096
ini: set 'debug' only in one place
Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
parents:
6555
diff
changeset
|
19 'debug': 'true', |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
20 'app_instance_uuid': 'development-not-secret', |
7738
bbf7be28a11e
config: rename .ini settings to avoid beaker-session deprecation warnings
Mads Kiilerich <mads@kiilerich.com>
parents:
7390
diff
changeset
|
21 'session.secret': 'development-not-secret', |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
22 }, |
7768
609d52bbf917
ini: create separate log handlers for color and null, and add comments hinting how they can be used
Mads Kiilerich <mads@kiilerich.com>
parents:
7738
diff
changeset
|
23 '[logger_root]': { |
609d52bbf917
ini: create separate log handlers for color and null, and add comments hinting how they can be used
Mads Kiilerich <mads@kiilerich.com>
parents:
7738
diff
changeset
|
24 'handlers': 'console_color', |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
25 }, |
7193
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
26 '[logger_routes]': { |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
27 'level': 'DEBUG', |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
28 }, |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
29 '[logger_beaker]': { |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
30 'level': 'DEBUG', |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
31 }, |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
32 '[logger_templates]': { |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
33 'level': 'INFO', |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
34 }, |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
35 '[logger_kallithea]': { |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
36 'level': 'DEBUG', |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
37 }, |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
38 '[logger_tg]': { |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
39 'level': 'DEBUG', |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
40 }, |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
41 '[logger_gearbox]': { |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
42 'level': 'DEBUG', |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
43 }, |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
44 '[logger_whoosh_indexer]': { |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
45 'level': 'DEBUG', |
150173a027ee
ini: set default log level to WARN
domruf <dominikruf@gmail.com>
parents:
6895
diff
changeset
|
46 }, |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
47 }, |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
48 ), |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
49 ] |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
50 |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
51 |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
52 def main(): |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
53 # make sure all mako lines starting with '#' (the '##' comments) are marked up as <text> |
6894
94f6b23e52d0
ini: move high level functionality and defaults to inifiles library
Mads Kiilerich <mads@kiilerich.com>
parents:
6893
diff
changeset
|
54 makofile = inifile.template_file |
7844
a8e6bb9ee9ea
future: use Python print function
Mads Kiilerich <mads@kiilerich.com>
parents:
7811
diff
changeset
|
55 print('reading:', makofile) |
6860
665dfa112f2c
py3: replace "file" with "open"
Lars Kruse <devel@sumpfralle.de>
parents:
6857
diff
changeset
|
56 mako_org = open(makofile).read() |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
57 mako_no_text_markup = re.sub(r'</?%text>', '', mako_org) |
8428
6eb1f66ac23f
ini: tweak mako comment markup - prepare for variable expansion in ## lines by only "quoting" ##
Mads Kiilerich <mads@kiilerich.com>
parents:
8354
diff
changeset
|
58 mako_marked_up = re.sub(r'\n##(.*)', r'\n<%text>##</%text>\1', mako_no_text_markup, flags=re.MULTILINE) |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
59 if mako_marked_up != mako_org: |
7844
a8e6bb9ee9ea
future: use Python print function
Mads Kiilerich <mads@kiilerich.com>
parents:
7811
diff
changeset
|
60 print('writing:', makofile) |
6860
665dfa112f2c
py3: replace "file" with "open"
Lars Kruse <devel@sumpfralle.de>
parents:
6857
diff
changeset
|
61 open(makofile, 'w').write(mako_marked_up) |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
62 |
8354
ef9fd1434270
ini: be more consistent about how # is used in the config template
Mads Kiilerich <mads@kiilerich.com>
parents:
8225
diff
changeset
|
63 lines = re.findall(r'\n(# [^ ].*)', mako_marked_up) |
ef9fd1434270
ini: be more consistent about how # is used in the config template
Mads Kiilerich <mads@kiilerich.com>
parents:
8225
diff
changeset
|
64 if lines: |
ef9fd1434270
ini: be more consistent about how # is used in the config template
Mads Kiilerich <mads@kiilerich.com>
parents:
8225
diff
changeset
|
65 print('ERROR: the template .ini file convention is to use "## Foo Bar" for text comments and "#foo = bar" for disabled settings') |
ef9fd1434270
ini: be more consistent about how # is used in the config template
Mads Kiilerich <mads@kiilerich.com>
parents:
8225
diff
changeset
|
66 for line in lines: |
ef9fd1434270
ini: be more consistent about how # is used in the config template
Mads Kiilerich <mads@kiilerich.com>
parents:
8225
diff
changeset
|
67 print(line) |
ef9fd1434270
ini: be more consistent about how # is used in the config template
Mads Kiilerich <mads@kiilerich.com>
parents:
8225
diff
changeset
|
68 raise SystemExit(1) |
ef9fd1434270
ini: be more consistent about how # is used in the config template
Mads Kiilerich <mads@kiilerich.com>
parents:
8225
diff
changeset
|
69 |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
70 # create ini files |
6893
d06039dc4ca2
ini: drop insertion of header comments in generated ini files
Mads Kiilerich <mads@kiilerich.com>
parents:
6891
diff
changeset
|
71 for fn, settings in ini_files: |
7844
a8e6bb9ee9ea
future: use Python print function
Mads Kiilerich <mads@kiilerich.com>
parents:
7811
diff
changeset
|
72 print('updating:', fn) |
6894
94f6b23e52d0
ini: move high level functionality and defaults to inifiles library
Mads Kiilerich <mads@kiilerich.com>
parents:
6893
diff
changeset
|
73 inifile.create(fn, None, settings) |
94f6b23e52d0
ini: move high level functionality and defaults to inifiles library
Mads Kiilerich <mads@kiilerich.com>
parents:
6893
diff
changeset
|
74 |
5536
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
75 |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
76 if __name__ == '__main__': |
06d5c043e989
ini: add scripts/generate-ini.py for generating all .ini files from template.ini.mako
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
77 main() |