Mercurial > kallithea

changeset 8769:d35d14b05b82

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
diff: handle some escaped characters in Git diffs There are some odd characters (like \r and \n) that the Kallithea UI doesn't allow in filenames in repos. Kallithea (through the routes module) will fail to generate URLs when browsing Files. That is a known limitation with minimal real-world impact, non-trivial to work around or fix. There are very few relevant use cases for tracking files with odd filenames. \t is valid but is hard to render in a meaningful way in the UI. And ASCII characters like \ and " are not usable on Windows and should just be avoided. Kallithea would parse Git diffs with odd characers incorrectly or fail, even before hitting the known limitation. With this change, Kallithea will parse diffs with odd filenames correctly (and then hit the limitation). Git will quote odd filenames and escape the odd characters when emitting diffs. (Mercurial does by design not allow \r and \n , and Mercurial will thus never have to quote file names in diffs.) Quotes are already handled (and ignored). With this change, Kallithea will handle \ unescaping of \\ and \", the usual letters like \r and \n and \t, and octal numbers like \033 (for ESC) . Filenames with \ and " will work perfectly (when not on Windows). Filenames with \t and ESC will work fine, but without helpful display in the UI. Filenames with \r and \n will still make the UI fail when trying to generate URLs. Thanks to stypr of Flatt Security for raising this.
author Mads Kiilerich <mads@kiilerich.com>
date Sat, 14 Nov 2020 15:41:39 +0100
parents fd61f678577f
children 0a84ef075575
files kallithea/lib/diffs.py kallithea/tests/models/test_diff_parsers.py
diffstat 2 files changed, 24 insertions(+), 6 deletions(-) [+]
line wrap: on
line diff
--- a/kallithea/lib/diffs.py	Sat Nov 14 15:20:40 2020 +0100
+++ b/kallithea/lib/diffs.py	Sat Nov 14 15:41:39 2020 +0100
@@ -549,6 +549,19 @@
 _header_next_check = re.compile(br'''(?!@)(?!literal )(?!delta )''')
 
 
+_git_bs_escape_re = re.compile(r'\\(?:([^0-9])|([0-9]{3}))')
+
+
+_git_bs_escape_dict = {'\\': '\\', '"': '"', 'r': '\r', 'n': '\n', 't': '\t'}
+
+
+def _git_bs_unescape_m(m):
+    c = m.group(1)
+    if c is not None:
+        return _git_bs_escape_dict.get(c) or ('\\' + c)
+    return chr(int(m.group(2), 8))
+
+
 def _get_header(vcs, diff_chunk):
     """
     Parses a Git diff for a single file (header and chunks) and returns a tuple with:
@@ -569,6 +582,11 @@
     if match is None:
         raise Exception('diff not recognized as valid %s diff: %r' % (vcs, safe_str(bytes(diff_chunk[:1000]))))
     meta_info = {k: None if v is None else safe_str(v) for k, v in match.groupdict().items()}
+    if vcs == 'git':
+        for k in ['a_path', 'b_path', 'a_file', 'b_file']:
+            v = meta_info.get(k)
+            if v:
+                meta_info[k] = _git_bs_escape_re.sub(_git_bs_unescape_m, v)
     rest = diff_chunk[match.end():]
     if rest:
         if _header_next_check.match(rest):
--- a/kallithea/tests/models/test_diff_parsers.py	Sat Nov 14 15:20:40 2020 +0100
+++ b/kallithea/tests/models/test_diff_parsers.py	Sat Nov 14 15:41:39 2020 +0100
@@ -269,7 +269,7 @@
           'ops': {RENAMED_FILENODE: 'file renamed from oh no to oh yes'}}),
     ],
     'git_diff_quoting.diff': [
-        (r'\"foo\"',  # TODO: quotes should not be escaped
+        ('"foo"',
          'added',
          {'added': 1,
           'binary': False,
@@ -281,19 +281,19 @@
           'binary': False,
           'deleted': 0,
           'ops': {1: 'new file 100644'}}),
-        ("'foo'" r'\"foo\"',  # TODO: quotes should not be escaped
+        ("'foo'" '"foo"',
          'added',
          {'added': 1,
           'binary': False,
           'deleted': 0,
           'ops': {1: 'new file 100644'}}),
-        (r'a\r\nb',  # TODO: escaped
+        ('a\r\nb',  # Note: will be parsed correctly, but other parts of Kallithea can't handle it
          'added',
          {'added': 1,
           'binary': False,
           'deleted': 0,
           'ops': {1: 'new file 100644'}}),
-        (r'foo\rfoo',  # TODO: escaped
+        ('foo\rfoo',  # Note: will be parsed correctly, but other parts of Kallithea can't handle it
          'added',
         {'added': 0,
          'binary': True,
@@ -311,13 +311,13 @@
           'binary': False,
           'deleted': 0,
           'ops': {1: 'new file 100644'}}),
-        (r'esc\033foo',  # TODO: escaped
+        ('esc\033foo',  # Note: will be parsed and handled correctly, but without good UI
          'added',
          {'added': 0,
           'binary': True,
           'deleted': 0,
           'ops': {1: 'new file 100644'}}),
-        (r'tab\tfoo',  # TODO: escaped
+        ('tab\tfoo',  # Note: will be parsed and handled correctly, but without good UI
          'added',
          {'added': 0,
           'binary': True,