[qutebrowser] qutebrowser v2.0.0 released (with better adblocker)!

[Florian Bruhin]

Hey,

I'm very happy to announce that I released qutebrowser v2.0.0 today!

This release integrates Brave's Rust adblocking library if the "adblock"
Python library is available. It also drops support for old Qt/Python
versions and comes with various other dependency changes.

This has been a rather long release cycle - so far, feature releases
have happened all two months or so, but v1.14.0 was released back in
October (with a patch release in December). Thus, this release includes
a *lot* of new features, bugfixes and other changes as well. It's a big
milestone in many ways, and a lot of things changed:

  $ git --no-pager diff --stat v1.14.1..v2.0.0
  [...]
  548 files changed, 13186 insertions(+), 10871 deletions(-)

Thanks to everyone involved, especially Árni Dagur for the excellent
work on the adblocking integration!

Here's the full changelog:

Major changes
~~~~~~~~~~~~~

- If the Python `adblock` library is available, it is now used to
  integrate Brave's Rust adblocker library for improved adblocking based on
  ABP-like filter lists (such as EasyList).
  If it is unavailable, qutebrowser falls back to host-blocking, i.e. the same
  blocking technique it used before this release. As part of this, various
  settings got renamed, see "Changed" below.
  **Note: If the `adblock` dependency is available, qutebrowser will ignore
  custom host blocking** via the `blocked-hosts` config file or `file:///` URLs
  supplied as host blocking lists. You will need to either migrate those to
  ABP-like lists, or set `content.blocking.method` to `both`.
- Various dependency upgrades - a quick checklist for packagers (see "Changed"
  below for details):
  * Ensure you're providing at least Python 3.6.1.
  * Ensure you're providing at least Qt 5.12 and PyQt 5.12.
  * Add a new optional dependency on the Python `adblock` library (if packaged -
    if not, consider packaging it, albeit optional it's very useful for users).
  * Remove the `cssutils` optional dependency (if present).
  * Remove the `attrs` (`attr`) dependency.
  * Remove the `pypeg2` dependency (and perhaps consider dropping the package
    if not used elsewhere - it's https://fdik.org/pyPEG2/[inactive upstream]
    and the repository was removed by Bitbucket).
  * Move the `pygments` dependency from required to optional.
  * Move the `setuptools` dependency from runtime (for `pkg_resources`) to
    build-time.
  * For Python 3.6, 3.7 or 3.8, add a dependency on the `importlib_resources`
    backport.
  * For Python 3.6 only, add a dependency on the `dataclasses` backport.
- Dropped support for old OS versions in binary releases:
  * Support for Windows 7 is dropped in the Windows binaries, the minimum
    required Windows version is now Windows 8.1.
  * Support for macOS 10.13 High Sierra is dropped in the macOS binaries, the
    minimum required macOS version is now macOS 10.14 Mojave.
- Various renamed settings and commands, see "Deprecated" and "Changed" below.

Removed
~~~~~~~

- The `--enable-webengine-inspector` flag (which was only needed for Qt 5.10 and
  below) is now dropped. With Qt 5.11 and newer, the inspector/devtools are
  enabled unconditionally.
- Support for moving qutebrowser data from versions before v1.0.0 has been
  removed.
- The `--old` flag for `:config-diff` has been removed. It used to show
  customized options for the old pre-v1.0 config files (in order to aid
  migration to v1.0).
- The `:inspector` command which was deprecated in v1.13.0 (in favor of
  `:devtools`) is now removed.

Deprecated
~~~~~~~~~~

- Several commands have been renamed for consistency and/or easier grouping of
  related commands. Their old names are still available, but deprecated and will
  be removed in qutebrowser v2.1.0.
  * `run-macro` -> `macro-run`
  * `record-macro` -> `macro-record`
  * `buffer` -> `tab-select`
  * `open-editor` -> `edit-text`
  * `toggle-selection` -> `selection-toggle`
  * `drop-selection` -> `selection-drop`
  * `reverse-selection` -> `selection-reverse`
  * `follow-selected` -> `selection-follow`
  * `follow-hint` -> `hint-follow`
  * `enter-mode` -> `mode-enter`
  * `leave-mode` -> `mode-leave`

Added
~~~~~

- New settings for the ABP-based adblocker:
   * `content.blocking.method` to decide which blocker(s) should be used.
   * `content.blocking.adblock.lists` to configure ABP-like lists to use.
- New `qt.environ` setting which makes it easier to set/unset environment
  variables for qutebrowser.
- New settings to use an external file picker (such as ranger or vifm):
  * `fileselect.handler` (`default` or `external`)
  * `fileselect.multiple_files.command`
  * `fileselect.single_file.command`
- When QtWebEngine has been updated but PyQtWebEngine hasn't yet, the dark mode
  settings might stop working. As a (currently undocumented) escape hatch, this
  version adds a `QUTE_DARKMODE_VARIANT=qt_515_2` environment variable which can
  be set to get the correct behavior in (transitive) situations like this.
- New `--desktop-file-name` commandline argument, which can be used to customize
  the desktop filename passed to Qt (which is used to set the `app_id` on
  Wayland).
- The `:open` completion now also completes local file paths and `file://` URLs,
  via a new `filesystem` entry in `completion.open_categories`. Also, a new
  `completion.favorite_paths` setting was added which can be used to add paths to
  show when `:open` is used without any input.
- New `QUTE_VERSION` variable for userscripts, which can be used to read
  qutebrowser's version.
- New "Copy URL" entry in the context menu for downloads.
- New `:bookmark-list` command which lists all bookmarks/quickmarks. The
  corresponding `qute://bookmarks` URL already existed since v0.8.0, but it was
  never exposed as a command.
- New `qt.workarounds.remove_service_workers` setting which can be used to
  remove the "Service Workers" directory on every start. Usage of this option is
  generally discouraged, except in situations where the underlying QtWebEngine bug
  is a known cause for crashes.
- Changelogs are now shown after qutebrowser was upgraded. By default, the
  changelog is only shown after minor upgrades (feature releases) but not patch
  releases. This can be adjusted (or disabled entirely) via a new
  `changelog_after_upgrade` setting.
- New userscripts:
  * `kodi` to play videos in Kodi
  * `qr` to generate a QR code of the current URL
  * `add-nextcloud-bookmarks` to create bookmarks in Nextcloud's Bookmarks app
  * `add-nextcloud-cookbook` to add recipes to Nextcloud's Cookbook app

Changed
~~~~~~~

- `config.py` files now are required to have either
  `config.load_autoconfig(False)` (don't load `autoconfig.yml`) or
  `config.load_autoconfig()` (do load `autoconfig.yml`) in them.
- Various host-blocking settings have been renamed to accomodate the new ABP-like
  adblocker:
  * `content.host_blocking.enabled` -> `content.blocking.enabled` (controlling both blockers)
  * `content.host_blocking.whitelist` -> `content.blocking.whitelist` (controlling both blockers)
  * `content.host_blocking.lists` -> `content.blocking.hosts.lists`
- Changes to default settings:
  * `tabs.background` is now `true` by default, so that new tabs get opened in the
    background.
  * `input.partial_timeout` is now set to 0 by default, so that partially typed
    key strings are never cleared.
  * `hints.leave_on_load` is now `false` by default, so that hint mode doesn't get
    left when a page finishes loading. This can lead to stale hints persisting in
    rare circumstances, but is better than leaving hint mode when the user entered
    it before loading was completed.
  * The default for `tabs.width` (tab bar width if vertical) is now 15% of the
    window width rather than 20%.
  * The default bindings for moving tabs (`tab-move -` and `tab-move +`) were
    changed from `gl` and `gr` to `gK` and `gJ`, to be consistent with the tab
    switching bindings.
  * The text color for warning messages is now black instead of white,
    for increased contrast and thus readability.
  * The default timeout for messages is now raised from 2s to 3s.
- On the first start, the history completion database is regenerated to remove
  a few problematic entries (such as long `qute://pdfjs` URLs). This might take
  a couple of minutes, but is a one-time operation. This should result in a
  performance improvement for the completion for affected users.
- qutebrowser now shows an error if its history database version is newer than
  expected. This currently should never happen, but allows for potentially
  backwards-incompatible changes in future versions.
- At least Python 3.6.1 is now required to run qutebrowser, support for Python
  3.5 (and 3.6.0) is dropped. Note that Python 3.5 is
  https://www.python.org/downloads/release/python-3510/[no longer supported
  upstream] since September 2020.
- At least Qt/PyQt 5.12 is now required to run qutebrowser, support for 5.7 to
  5.11 (inclusive) is dropped. While Debian Buster ships Qt 5.11, it's based on a
  Chromium version from 2018 with
  https://www.debian.org/releases/buster/amd64/release-notes/ch-information.en.html#browser-security[no Debian security support]
  and unsupported upstream since May 2019.
  It also has compatibility issues with various websites (GitHub, Twitch, Android
  Developer documentation, YouTube, ...). Since no newer Debian Stable is released
  at the time of writing, it's recommended to
  https://github.com/qutebrowser/qutebrowser/blob/master/doc/install.asciidoc#installing-qutebrowser-with-virtualenv[install qutebrowser in a virtualenv]
  with a newer version of Qt/PyQt.
- New optional dependency on the Python `adblock` library (see above for details).
- The (formerly optional) `cssutils` dependency is now removed. It was only
  needed for improved behavior in corner cases when using `:download --mhtml`
  with the (non-default) QtWebKit backend, and as such it's unlikely anyone is
  still relying on it. The `cssutils` project is also dead upstream, with its
  repository being gone after Bitbucket
  https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket[removed Mercurial support].
- The (formerly required) `pygments` dependency is now optional. It is only
  used when using `:view-source` with QtWebKit, or when forcing it via
  `:view-source --pygments` on QtWebEngine. If it is unavailable, an
  unhighlighted fallback version of the page's source is shown.
- The former runtime dependency on the `pkg_resources` module (part of the
  `setuptools` project) got dropped. Note that `setuptools` is still required
  to run `setup.py`.
- A new dependency on the `importlib_resources` module got introduced for
  Python versions up to and including 3.8. Note that the stdlib
  `importlib.resources` module for Python 3.7 and 3.8 is missing the needed APIs,
  thus requiring the backports for those versions as well.
- The former dependency on the `attrs`/`attr` package is now dropped in favour
  of `dataclasses` in the Python standard library. On Python 3.6, a new
  dependency on the `dataclasses` backport is now required.
- The former dependency on the `pypeg2` package is now dropped. This might cause
  some changes for certain corner-cases for suggested filenames when downloading
  files with the QtWebKit backend.
- Windows and macOS releases now ship Python 3.9 rather than 3.7.
- The `colors.webpage.darkmode.*` settings are now also supported with older Qt
  versions (Qt 5.12 and 5.13) rather than just with Qt 5.14 and above.
- For regexes in the config (`hints.{prev,next}_regexes`), certain patterns
  which will change meanings in future Python versions are now disallowed. This is
  the case for character sets starting with a literal `[` or containing literal
  character sequences `--`, `&&`, `~~`, or `||`. To avoid a warning, remove the
  duplicate characters or escape them with a backslash.
- If `prompt(..., "default")` is used via JS, the default text is now
  pre-selected in the prompt shown by qutebrowser.
- URLs such as `::1/foo` are now handled as a search term or local file rather
  than IPv6. Use `[::1]/foo` to force parsing as IPv6 instead.
- The `mkvenv.py` script now runs a "smoke test" after setting up the virtual
  environment to ensure it's working as expected. If necessary, the test can be
  skipped via a new `--skip-smoke-test` flag.
- Both qutebrowser userscripts and Greasemonkey scripts are now additionally
  picked up from qutebrowser's config directory (the `userscripts` and
  `greasemonkey` subdirectories of e.g. `~/.config/qutebrowser/`) rather than only
  the data directory (the same subdirectories of e.g.
  `~/.local/share/qutebrowser/`).
- The `:later` command now understands a time specification like `5m` or
  `1h5m2s`, rather than just taking milliseconds.
- The `importer.py` script doesn't use a browser argument anymore; instead its
  `--input-format` switch can be used to configure the input format. The help also
  was expanded to explain how to use it properly.
- If `tabs.tabs_are_windows` is set, the `tabs.last_close` setting is now
  ignored and the window is always closed when using `:close` (`d`).
- With the (default) QtWebEngine backend, if a custom `accept` header is set via
  `content.headers.custom`, the custom value is now ignored for XHR
  (`XMLHttpRequest`) requests. Instead, the sent value is now `*/*` or the header
  set from JavaScript, as it would be if `content.headers.custom` wasn't set.
- The `:tab-select` completion now shows the underlying renderer process PID if
  doing so is supported (on QtWebEngine 5.15).
- If `tabs.favicons.show` is set to `never`, favicons aren't unnecessarily
  downloaded anymore. Thus, disabling favicons can help with a possible
  https://www.ghacks.net/2021/01/22/favicons-may-be-used-to-track-users/[fingerprinting vector].
- "Super" is now understood as a modifier (i.e. as alias to "Meta").
- Initial support for Python 3.10 (currently in Alpha stage).
- Various performance improvements, including for the startup time.

Fixed
~~~~~

- With interpolated color settings (`colors.tabs.indicator.*` and
  `colors.downloads.*`), the alpha channel is now handled correctly.
- Fixes to userscripts:
    * `format_json` now uses `env` in its shebang, making it work
      correctly on systems where `bash` isn't located in `/bin`.
    * `qute-pass` now handles the MIME output format introduced in gopass 1.10.0.
    * `qute-lastpass` now types multiple `<` or `>` characters correctly.
- The `:undo` completion now sorts its entries correctly (by the numerical index
  rather than lexicographically).
- The `completion.web_history.ignore` setting now works properly when set in
  `config.py` (rather than via `:set`). Additionally, a `:config-source` will not
  result in a history rebuild if the value wasn't actually changed.
- When downloading a `data:` URL, the suggested filename is now improved and
  contains a proper extension. Before this fix, qutebrowser would use the URL's
  data contents as filename with QtWebEngine; or "binary blob" with the Qt network
  stack.
- When `:tab-only` is run before a tab is available, an error is now shown
  instead of crashing.
- A couple of long URLs (such as `qute://pdfjs` URLs) are now not added to the
  history database anymore.
- A bug in QtWebEngine 5.15.2 causes "renderer process killed" errors on
  websites like LinkedIn and TradingView. There is now a workaround in qutebrowser
  to prevent this from happening.
- Nextcloud Calendars started using `String.replaceAll` which was only added to
  Chromium recently (Chrome 85), so won't work with current QtWebEngine
  versions. This release includes a workaround (a polyfill as a
  site-specific-quirk).

Enjoy, and as usual, let me know if there are any issues!

Florian

-- 
            me at the-compiler.org | https://www.qutebrowser.org 
       https://bruhin.software/ | https://github.com/sponsors/The-Compiler/
       GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc
             I love long mails! | https://email.is-not-s.ms/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://listi.jpberlin.de/pipermail/qutebrowser/attachments/20210128/13f323f0/attachment.asc>