Mercurial > kallithea
view docs/upgrade.rst @ 6532:33b71a130b16
templates: properly escape inline JavaScript values
TLDR: Kallithea has issues with escaping values for use in inline JS.
Despite judicious poking of the code, no actual security vulnerabilities
have been found, just lots of corner-case bugs. This patch fixes those,
and hardens the code against actual security issues.
The long version:
To embed a Python value (typically a 'unicode' plain-text value) in a
larger file, it must be escaped in a context specific manner. Example:
>>> s = u'<script>alert("It\'s a trap!");</script>'
1) Escaped for insertion into HTML element context
>>> print cgi.escape(s)
<script>alert("It's a trap!");</script>
2) Escaped for insertion into HTML element or attribute context
>>> print h.escape(s)
<script>alert("It's a trap!");</script>
This is the default Mako escaping, as usually used by Kallithea.
3) Encoded as JSON
>>> print json.dumps(s)
"<script>alert(\"It's a trap!\");</script>"
4) Escaped for insertion into a JavaScript file
>>> print '(' + json.dumps(s) + ')'
("<script>alert(\"It's a trap!\");</script>")
The parentheses are not actually required for strings, but may be needed
to avoid syntax errors if the value is a number or dict (object).
5) Escaped for insertion into a HTML inline <script> element
>>> print h.js(s)
("\x3cscript\x3ealert(\"It's a trap!\");\x3c/script\x3e")
Here, we need to combine JS and HTML escaping, further complicated by
the fact that "<script>" tag contents can either be parsed in XHTML mode
(in which case '<', '>' and '&' must additionally be XML escaped) or
HTML mode (in which case '</script>' must be escaped, but not using HTML
escaping, which is not available in HTML "<script>" tags). Therefore,
the XML special characters (which can only occur in string literals) are
escaped using JavaScript string literal escape sequences.
(This, incidentally, is why modern web security best practices ban all
use of inline JavaScript...)
Unsurprisingly, Kallithea does not do (5) correctly. In most cases,
Kallithea might slap a pair of single quotes around the HTML escaped
Python value. A typical benign example:
$('#child_link').html('${_('No revisions')}');
This works in English, but if a localized version of the string contains
an apostrophe, the result will be broken JavaScript. In the more severe
cases, where the text is user controllable, it leaves the door open to
injections. In this example, the script inserts the string as HTML, so
Mako's implicit HTML escaping makes sense; but in many other cases, HTML
escaping is actually an error, because the value is not used by the
script in an HTML context.
The good news is that the HTML escaping thwarts attempts at XSS, since
it's impossible to inject syntactically valid JavaScript of any useful
complexity. It does allow JavaScript errors and gibberish to appear on
the page, though.
In these cases, the escaping has been fixed to use either the new 'h.js'
helper, which does JavaScript escaping (but not HTML escaping), OR the
new 'h.jshtml' helper (which does both), in those cases where it was
unclear if the value might be used (by the script) in an HTML context.
Some of these can probably be "relaxed" from h.jshtml to h.js later, but
for now, using h.jshtml fixes escaping and doesn't introduce new errors.
In a few places, Kallithea JSON encodes values in the controller, then
inserts the JSON (without any further escaping) into <script> tags. This
is also wrong, and carries actual risk of XSS vulnerabilities. However,
in all cases, security vulnerabilities were narrowly avoided due to other
filtering in Kallithea. (E.g. many special characters are banned from
appearing in usernames.) In these cases, the escaping has been fixed
and moved to the template, making it immediately visible that proper
escaping has been performed.
Mini-FAQ (frequently anticipated questions):
Q: Why do everything in one big, hard to review patch?
Q: Why add escaping in specific case FOO, it doesn't seem needed?
Because the goal here is to have "escape everywhere" as the default
policy, rather than identifying individual bugs and fixing them one
by one by adding escaping where needed. As such, this patch surely
introduces a lot of needless escaping. This is no different from
how Mako/Pylons HTML escape everything by default, even when not
needed: it's errs on the side of needless work, to prevent erring
on the side of skipping required (and security critical) work.
As for reviewability, the most important thing to notice is not where
escaping has been introduced, but any places where it might have been
missed (or where h.jshtml is needed, but h.js is used).
Q: The added escaping is kinda verbose/ugly.
That is not a question, but yes, I agree. Hopefully it'll encourage us
to move away from inline JavaScript altogether. That's a significantly
larger job, though; with luck this patch will keep us safe and secure
until such a time as we can implement the real fix.
Q: Why not use Mako filter syntax ("${val|h.js}")?
Because of long-standing Mako bug #140, preventing use of 'h' in
filters.
Q: Why not work around bug #140, or even use straight "${val|js}"?
Because Mako still applies the default h.escape filter before the
explicitly specified filters.
Q: Where do we go from here?
Longer term, we should stop doing variable expansions in script blocks,
and instead pass data to JS via e.g. data attributes, or asynchronously
using AJAX calls. Once we've done that, we can remove inline JavaScript
altogether in favor of separate script files, and set a strict Content
Security Policy explicitly blocking inline scripting, and thus also the
most common kind of cross-site scripting attack.
| author | Søren Løvborg <sorenl@unity3d.com> |
|---|---|
| date | Tue, 28 Feb 2017 17:19:00 +0100 |
| parents | f973b866fffc |
| children | 2c3d30095d5e |
line wrap: on
line source
.. _upgrade: =================== Upgrading Kallithea =================== This describes the process for upgrading Kallithea, independently of the Kallithea installation method. .. note:: If you are upgrading from a RhodeCode installation, you must first install Kallithea 0.3.2 and follow the instructions in the 0.3.2 README to perform a one-time conversion of the database from RhodeCode to Kallithea, before upgrading to the latest version of Kallithea. 1. Stop the Kallithea web application ------------------------------------- This step depends entirely on the web server software used to serve Kallithea, but in any case, Kallithea should not be running during the upgrade. .. note:: If you're using Celery, make sure you stop all instances during the upgrade. 2. Create a backup of both database and configuration ----------------------------------------------------- You are of course strongly recommended to make backups regularly, but it is *especially* important to make a full database and configuration backup before performing a Kallithea upgrade. Back up your configuration ^^^^^^^^^^^^^^^^^^^^^^^^^^ Make a copy of your Kallithea configuration (``.ini``) file. If you are using :ref:`rcextensions <customization>`, you should also make a copy of the entire ``rcextensions`` directory. Back up your database ^^^^^^^^^^^^^^^^^^^^^ If using SQLite, simply make a copy of the Kallithea database (``.db``) file. If using PostgreSQL, please consult the documentation for the ``pg_dump`` utility. If using MySQL, please consult the documentation for the ``mysqldump`` utility. Look for ``sqlalchemy.url`` in your configuration file to determine database type, settings, location, etc. 3. Activate the Kallithea virtual environment (if any) ------------------------------------------------------ Verify that you are using the Python environment that you originally installed Kallithea in by running:: pip freeze This will list all packages installed in the current environment. If Kallithea isn't listed, activate the correct virtual environment. See the appropriate installation page for details. 4. Install new version of Kallithea ----------------------------------- Please refer to the instructions for the installation method you originally used to install Kallithea. If you originally installed using pip, it is as simple as:: pip install --upgrade kallithea If you originally installed from version control, it is as simple as:: cd my-kallithea-clone hg pull -u pip install -e . 5. Upgrade your configuration ----------------------------- Run the following command to upgrade your configuration (``.ini``) file:: paster make-config Kallithea my.ini This will display any changes made by the new version of Kallithea to your current configuration, and attempt an automatic merge. It is recommended that you check the contents after the merge. .. note:: Please always make sure your ``.ini`` files are up to date. Errors can often be caused by missing parameters added in new versions. .. _upgrade_db: 6. Upgrade your database ------------------------ .. note:: If you are *downgrading* Kallithea, you should perform the database migration step *before* installing the older version. (That is, always perform migrations using the most recent of the two versions you're migrating between.) First, run the following command to see your current database version:: alembic -c my.ini current Typical output will be something like "9358dc3d6828 (head)", which is the current Alembic database "revision ID". Write down the entire output for troubleshooting purposes. The output will be empty if you're upgrading from Kallithea 0.3.x or older. That's expected. If you get an error that the config file was not found or has no ``[alembic]`` section, see the next section. Next, if you are performing an *upgrade*: Run the following command to upgrade your database to the current Kallithea version:: alembic -c my.ini upgrade head If you are performing a *downgrade*: Run the following command to downgrade your database to the given version:: alembic -c my.ini downgrade 0.4 Alembic will show the necessary migrations (if any) as it executes them. If no "ERROR" is displayed, the command was successful. Should an error occur, the database may be "stranded" half-way through the migration, and you should restore it from backup. Enabling old Kallithea config files for Alembic use ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Kallithea configuration files created before the introduction of Alembic (i.e. predating Kallithea 0.4) need to be updated for use with Alembic. Without this, Alembic will fail with an error like this:: FAILED: No config file 'my.ini' found, or file has no '[alembic]' section If Alembic complains specifically about a missing ``alembic.ini``, it is likely because you did not specify a config file using the ``-c`` option. On the other hand, if the mentioned config file actually exists, you need to append the following lines to it:: [alembic] script_location = kallithea:alembic Your config file should now work with Alembic. 7. Rebuild the Whoosh full-text index ------------------------------------- It is recommended that you rebuild the Whoosh index after upgrading since new Whoosh versions can introduce incompatible index changes. 8. Start the Kallithea web application -------------------------------------- This step once again depends entirely on the web server software used to serve Kallithea. Before starting the new version of Kallithea, you may find it helpful to clear out your log file so that new errors are readily apparent. .. note:: If you're using Celery, make sure you restart all instances of it after upgrade. .. _virtualenv: http://pypi.python.org/pypi/virtualenv