This commit introduces a new configuration section [markup] to refine
Misaka's Markdown extensions (by default strikethrough, superscript and
autolink).
Furthermore, you can set custom HTML elements/attributes that are
allowed, e.g. to enable images, set
[markup]
allowed-elements = img
allowed-attributes = src
The refactorization separates HTML sanitization from Markdown -> HTML
and allows to include new markup languages such as BB Code or
reStructuredText.
Python's HTMLParser is smart enough to filter malicious tags but fails
to repair invalid, user-inputted HTML. Instead of re-inventing the
wheel, Isso now uses html5lib's HTMLSanitizer with a whitelist of all
tags generated by Sundown.
Disallowed tags are discarded from the output to match the previous
unittests. This feature is only available for html5lib 0.99(9) and
later. Earlier releases just escape disallowed tags.
To be compatible with comments from Disqus (and users unfamiliar with
Markdown), Misaka no longer disables user-inputted HTML, but the
generated HTML is now post-processed and all "unsafe" tags (not
possible with Markdown) are discarded.
Whitelist: p, a, pre, blockquote, h1-h6, em, sub, sup, del, ins, math,
dl, ol, ul, li
This commit also removes an unnecessary newline generated by
Misaka/Sundown.
This commit now sanitizes *all* HTML tags written by the user (also
prevents auto-link to "unsafe" web protocols and images) as intended.
Fortunately because of Sundown's typography support, it did not affect
JS injection, but custom style tags and iframes.
PS: thanks to the anonymous submitter of a comment including a style tag
for 24pt, red font ;-)
As a result, it is no longer possible to chain promises
(then().then().then(etc.)), but that is actually not an issue for Isso.
The deferred/promise implementation is roughly based on
http://stackoverflow.com/a/17722683 and stackp/promisejs.
Uses keyword arguments to use multiprocessing or uwsgi mixin. This
fixes an issue on exotic *BSDs such as NetBSD where Python comes not
with inter-process semaphores (issue 3307):
mod_wsgi (pid=14365): Target WSGI script '/var/www/vhosts/my.hostname.org/htdocs/isso.wsgi' cannot be loaded as Python module.
mod_wsgi (pid=14365): Exception occurred processing WSGI script '/var/www/vhosts/my.hostname.org/htdocs/isso.wsgi'.
Traceback (most recent call last):
File "/var/www/vhosts/my.hostname.org/htdocs/isso.wsgi", line 8, in <module>
application = make_app(Config.load("/var/www/vhosts/my.hostname.org/htdocs/isso.cfg"))
File "/usr/pkg/lib/python2.7/site-packages/isso/__init__.py", line 155, in make_app
isso = App(conf)
File "/usr/pkg/lib/python2.7/site-packages/isso/__init__.py", line 91, in __init__
super(Isso, self).__init__(conf)
File "/usr/pkg/lib/python2.7/site-packages/isso/core.py", line 223, in __init__
self.lock = multiprocessing.Lock()
File "/usr/pkg/lib/python2.7/multiprocessing/__init__.py", line 175, in Lock
from multiprocessing.synchronize import Lock
File "/usr/pkg/lib/python2.7/multiprocessing/synchronize.py", line 59, in <module>
" function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.
When using Gunicorn or uWSGI to run `isso.dispatch` it would
automatically initialize and a default Isso instance (and cause
several logging messages), although never used.
If you use uWSGI or Gunicorn, you have to change the module from
`isso` to `isso.run`.
The previous approach using a custom X-Custom header did work for the
client-side, but not for activation and deletion links. Now, you need
to add a `name = foo` option to the general section. `isso.dispatch`
then binds this configuration to /foo and can distinguish all API
calls without a special HTTP header.
When an attacker uses a <form> to downvote a comment, the browser
*should* add a `Content-Type: ...` header with three possible values:
* application/x-www-form-urlencoded
* multipart/form-data
* text/plain
If the header is not sent or requests `application/json`, the
request is not forged (XHR is restricted by CORS separately).
Revert "use Referer instead of Origin when using IE"
Revert "fix unittests"
Revert "check if Origin matches Host to mitigate CSRF, part of #40"
This reverts commit 9376511485c70deaf908aa67bcdc8f0c9a0b003e.
This reverts commit 9a03cca793.
This reverts commit 4c16ba76cc.
This reverts commit 32e4b70510.
Therefore, only raise Forbidden if Origin (or Referer for MSIE) is sent
(which is a protected header and all modern browsers (except IE)).
Also add a basic unit test which asserts the failure for false origins.
The URL sent in the email returns a short HTML document where
JS creates a modal dialog. If continued, the browser sends a
POST request to the same URL.
The <id> tag does not necessarily contains the full URL, but also
relative URLs:
<id>http://example.com/foo/bar.html</id>
<id>/foo/bar.html</id>
<id>foo/bar.html</id>
Also add an option `direct-reply` to control the number of comments
on a thread without referencing a child (to avoid a simple while loop
that `curl -XPOST ...` the url).
Defaults to 3, that means a /24 (or /48 for IPv6) address can only post
3 direct responses on a thread at all.
This is a severe issue which makes the current voters bloomfilter
completely useless. Functions are first-class objects in Python, which
lead to interesting "issues" like:
>>> def foo(x=[]):
... x.append(1)
... print x
...
>>> foo()
[1]
>>> foo()
[1, 1]
For Isso, this means the bloomfilter, which is usually only initialized
with the author's IP address, is now initialized with pretty much all
ip addresses from previous authors, thus makes it impossible for the
author to vote on other's people comments.
Keep Isso modular, not monolithic. Make it easy to integrate a
web interface or add XMPP notifications.
This refactorization includes minor bugfixes and changes:
* CORS middleware did not work properly due to wrong unit tests
* more type checks on JSON input
* new detection for origin and public url, closes#28
* new activation and delete url (no redirect for old urls, but you can
convert the old urls: copy hash after `/activate/` (or delete) and
open `/id/<id of comment>/activate/<hash>`
* move crypto.py to utils/
With this commit, SMTP is no longer automatically configured: add
`notify = smtp` to the `[general]` section to use SMTP.
An orphan comment is exported by Disqus but its thread id is
non-existent (probably deleted, moved). Usually from the earlier
days (or WordPress migration).
It is not possible to get the thread without manual intervention (
aka SQLite insertions).
All data-attributes beginning with `data-isso-` are stored in
`api.config` (without leading data-isso-). Isso tries to parse
the values with JSON (e.g. `-isso-foo="false"` returns false)
and falls back for a simple string value.
Cookies set from a different domain can not be read by JS executed in
the current domain. As a workaround, Isso sends both a Set-Cookie and
X-Set-Cookie header. The former is used by the browser to make the
HTTP request to the API, the latter is read by `embed.min.js` to
determine if a comment can be edited or deleted.
When a comment is deleted, the server sends an expired cookies in
Set-Cookie and X-Set-Cookie.
If you don't use a <h1> to markup your post's title (but h2), it
is no longer possible to reliable detect the site's title.
E.g. you have a single page with only one <h1> and that's the
*real* title of that page. But on the other hand, it is also
possible, that the <h1> tag is just your website's name and the
actual post title is marked up in <h2>.
Previously this led to unnecessary object creation which impacted the
rendering time (on my machine 200 comments -> 1200ms) just to create
the postbox per comment (just the object initialization)).
Markdown conversion is not the reason for 2s per 100 comments response,
the hash function is. When using the email/remote_addr from cache, the
response time is pretty fast.
* when uWSGI is available, use their caching framework
* for multi-threaded environment (the default), use a simple cache
shipped with werkzeug
Martin Zimmermann