Metadata-Version: 2.4
Name: plone.recipe.vinylcache
Version: 9.0.3.0
Summary: Build and/or configure Vinyl Cache (formerly Varnish Cache) with zc.buildout
Author: Wichert Akkerman, et al
Author-email: wichert@wiggy.net
License: BSD
Project-URL: Homepage, https://pypi.python.org/pypi/plone.recipe.vinylcache
Keywords: buildout,varnish,vinylcache,vinyl-cache,cache,proxy
Classifier: Framework :: Buildout
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: Proxy Servers
Classifier: Development Status :: 4 - Beta
Requires-Python: >=3.10
Description-Content-Type: text/x-rst
License-File: LICENSE.rst
Requires-Dist: jinja2>=3.0
Requires-Dist: setuptools
Requires-Dist: zc.buildout
Requires-Dist: zc.recipe.cmmi
Provides-Extra: test
Requires-Dist: interlude; extra == "test"
Dynamic: license-file

.. This README is meant for consumption by humans and PyPI. PyPI can render rst files so please do not use Sphinx features.
   If you want to learn more about writing documentation, please check out: http://docs.plone.org/about/documentation_styleguide.html
   This text does not appear on PyPI or github. It is a comment.

.. image:: https://img.shields.io/pypi/v/plone.recipe.vinylcache.svg
    :target: https://pypi.python.org/pypi/plone.recipe.vinylcache/
    :alt: Latest Version

.. image:: https://img.shields.io/pypi/pyversions/plone.recipe.vinylcache.svg?style=plastic
   :target: https://pypi.python.org/pypi/plone.recipe.vinylcache/
   :alt: Supported - Python Versions

.. image:: https://img.shields.io/pypi/l/plone.recipe.vinylcache.svg
    :target: https://pypi.python.org/pypi/plone.recipe.vinylcache/
    :alt: License

Vinyl Cache recipe for buildout
================================

``plone.recipe.vinylcache`` is a `zc.buildout`_ recipe to install `Vinyl
Cache`_ (formerly known as Varnish Cache; the upstream project renamed
itself in early 2026). Even though the name contains the name Plone,
there is nothing Plone-specific about this recipe: it works for
non-Zope sites just as well.

This recipe is a fork of `plone.recipe.varnish`_, adapted for the new
project name and the current Vinyl Cache 9.0.x stable release line.
**It supports Vinyl Cache 9.0.x only.** If you are still running the
previous Varnish 6.0 LTS line, keep using ``plone.recipe.varnish``
instead -- that package is unaffected by this fork and continues to
target 6.0 LTS.

Note: as of the 9.0.x release line, the rename is a project/brand
rename -- the daemon this recipe builds and runs is still the
``varnishd`` binary (the source tree's internal directory layout has
been renamed, e.g. ``bin/vinyld/``, but the compiled/installed program,
its ``-V`` self-identification, its ``Server:``/``X-Varnish`` HTTP
headers, and the ``varnish-modules`` VMOD collection are all still
named/branded "varnish" as of 9.0.3). This recipe therefore still
refers to the daemon as ``varnishd`` throughout; only the recipe's own
package name and download defaults track the new "Vinyl Cache" branding
and release line.


What's new compared to plone.recipe.varnish
---------------------------------------------

Besides targeting Vinyl Cache 9.0.x instead of Varnish 6.0 LTS, this
fork adds a handful of things not present in ``plone.recipe.varnish``:

* The default ``vcl_hash`` now includes ``req.http.host`` (matching
  Vinyl Cache's own built-in default), avoiding cache cross-contamination
  between different vhosts/backends serving overlapping URL paths on the
  same instance.
* ``PATCH`` is treated like ``PUT``/``POST``/``DELETE`` in the method
  whitelist, and WebSocket upgrade requests are piped through instead of
  hitting normal GET/HEAD caching logic.
* ``Accept-Encoding`` is normalized to reduce cache fragmentation, and
  large files (by extension) are streamed (``beresp.do_stream``) instead
  of piped, keeping them cacheable and visible to logging.
* A new ``shard`` value for the ``balancer`` option (consistent-hashing
  director, better cache hit ratio than ``round_robin``/``random`` across
  several backends).
* The purge ACL suppresses Vinyl Cache 9.0's (harmless but noisy)
  ACL-folding compiler warning for common setups.
* ``verbose-headers`` is a real, working option (diagnostic
  ``X-Cache``/``X-Cacheable``/``grace`` response headers, off by
  default).
* A new ``purge-by-id`` option, compatible with `collective.purgebyid
  <https://github.com/collective/collective.purgebyid>`_, for purging
  every cached variant of a piece of content by id rather than needing
  to enumerate cached URLs -- via ``ban()`` (default, no vmod needed) or
  via the ``xkey`` vmod (opt-in, more efficient).
* A new ``max-cacheable-size`` option: objects larger than this (by
  ``Content-Length``) are never cached, preventing a single large object
  from evicting many smaller ones under LRU pressure.
* A new ``tls-config`` option (``script``) mapping to ``varnishd -A``, a
  Vinyl Cache 9.0 addition letting ``varnishd`` terminate TLS itself.
* A new ``plone.recipe.vinylcache:selfsigned`` recipe to generate a
  matching self-signed certificate for internal/dev/testing use of
  ``tls-config``.

See ``CHANGES.rst`` for the full list, including bugfixes carried over
from fixing dead/no-op options this fork was originally forked with.


Configuring it is very simple. For example::

    [varnish-build]
    recipe = plone.recipe.vinylcache:build

    [varnish-configuration]
    recipe = plone.recipe.vinylcache:configuration
    backends = 127.0.0.1:8081

    [varnish-script]
    recipe = plone.recipe.vinylcache:script
    bind = 127.0.0.1:8000
    cache-size = 512M


This configures three buildout parts:

``varnish-build``
    which will download, compile and install Vinyl Cache,

``varnish-configuration``
    which generates the VCL configuration file,
    sending requests to a backend at 127.0.0.1:8081, and

``varnish-script``
    which runs the ``varnishd`` daemon, configured to listen on
    127.0.0.1:8000 for requests, using a 512 megabyte cache.

A wrapper script for the ``varnishd`` startup command is created in the
``bin`` directory of your buildout.

Please note that the configuration generated by this recipe is
specifically meant for Vinyl Cache 9.0.x, and generates VCL syntax
version 4.1.


Area's of interest to look at when you use this recipe
------------------------------------------------------

Even though this recipe tries to provide sane defaults for most parameters for
a drop in placement, there are a few area's of expertise which you can learn
about to understand and test for a performant but stable Vinyl Cache set up.

* If you use this recipe, you will use buildout and a common setup for
  'single' projects is that your Vinyl Cache will only cache for one Plone
  site. An inherent problem with caching is cache invalidation: when an editor
  changes a page you want to make sure the next delivery for this content
  item fetched freshly from the backend (Plone) site. Plone's caching setup
  can be configured to send a purge requests for changed content to Vinyl
  Cache. The calculated has plays an essential role in this type of purging:
  if the calculated hash from the client request is different from the
  calculated hash on the purge request, purging will fail.

* There are clever alternative purge request setups, which can improve freshness
  but you really have to know what you are doing and experience so far is that
  more advanced schemes have broken between Varnish/Vinyl Cache upgrades.

* Especially if you have multiple backends and you let Vinyl Cache do the load
  balancing, don't forget to enable the grace-sick and grace-healthy options.
  When the recipe notices there parameters, it automatically enables health probe
  settings in the generated vcl. grace helps serving pages temporarily when your
  backends are temporarily down.

* As soon as a cookie appears on an incoming request, Vinyl Cache will not cache
  the request and even store the url in a 'hit for pass' buffer so that subsequent
  similar requests don't get delayed by waiting in the backend request queue. The
  generated vcl has a function which strips off most irrelevant cookies from
  incomiing requests before they get passed to the backend to increase cache hit
  rate. the __ac cookie is the most notable exception, this indicates for Plone
  that a user is logged in and caching should be disabled.

* You can monitor Vinyl Cache caching operations in great detail by learning how
  to use ``varnishlog`` and the query language, but it will take at least a few
  hours if you have never used this tool before. It is easy to draw the wrong
  conclusions from just poking around a bit in ``varnishlog`` and seeing hits
  or misses.


Virtual hosting
---------------

Vinyl Cache supports virtual hosting by selecting a different backend server
based on headers on the incoming request. You can configure the backends
through the backends option::

  [varnish-configuration]
  backends =
     plone.org:127.0.0.1:8000
     plone.net:127.0.0.1:9000

This will generate a configuration which sends all traffic for the plone.org
host to a backend server running on port 8000 while all traffic for the
plone.net host is send to port 9000.


Zope 2 hosting (with Virtual Host Monster)
------------------------------------------

If you are using Zope 2 as backend server you will need to rewrite the URL
so the Zope Virtual Host Monster (VHM) can generate correct links for links in
your pages. This can be done either by a web server such as Apache or nginx
(placed either in front or behind Vinyl Cache) but can also be done by Vinyl
Cache itself.

The three options are described below.

Option 1 (rewrites after Vinyl Cache)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If generating these VHM-style URLs in a proxy *behind* Vinyl Cache (or if using
VHM's 'mapping' feature), no extra Vinyl Cache configuration is needed.
Just make sure the ``backends`` option directs the traffic to the proxy.

Option 2 (rewrites before Vinyl Cache)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If generating these VHM-style URLs in a proxy *in front* of Vinyl Cache, no
extra Vinyl Cache configuration is needed as long as the original hostname is
still retained in the URL. If the hostname is not retained, you can tell
Vinyl Cache to direct requests based on the "path" instead of the hostname.
For example::

  [varnish-configuration]
  backends =
    /VirtualHostBase/http/plone.org:80/Plone:127.0.0.1:8000
    /VirtualHostBase/http/plone.net:80/Plone:127.0.0.1:9000

This will generate a configuration which sends all traffic for any request whose
path starts with ``/VirtualHostBase/http/plone.org:80/Plone`` to a backend server
running at 127.0.0.1 on port 8000, while request paths starting with
``/VirtualHostBase/http/plone.net:80/Plone`` are sent to port 9000.

Option 3 (rewrites within Vinyl Cache)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To have Vinyl Cache generate these VHM-style URLs, you can use the
``zope2_vhm_map`` option.
Here is an example::

  [varnish-configuration]
  zope2_vhm_map =
      plone.org:/plone
      plone.net:/plone

This tells us that the domain plone.org should be mapped to the location
``/plone`` in the backend. By combining this with the information from the
``backends`` option a configuration will be generated that maps URLs
correctly.

Load Balancing
--------------

Vinyl Cache supports load balancing by configuring a director for a pool of
backends. This director sends the incoming requests that cannot be fulfilled
by Vinyl Cache to backends in the pool in either random or round robin
fashion. You can configure the director via the ``balancer`` option::

  [varnish-configuration]
  balancer = random

This will generate a configuration which sends all traffic to the director,
which will choose a 'random' backend server to fulfill the request if the
content requested is not cached by Vinyl Cache itself.


plone.recipe.vinylcache reference
----------------------------------

The ``plone.recipe.vinylcache`` recipe does one or more of the following:

``plone.recipe.vinylcache:build``
    compiles Vinyl Cache from sources

``plone.recipe.vinylcache:configuration``
    generates a VCL-configuration file

``plone.recipe.vinylcache:script``
    generates a wrapper script inside your buildout that will start Vinyl
    Cache with the correct configuration.

``plone.recipe.vinylcache:selfsigned``
    generates a self-signed TLS certificate and a config file suitable
    for ``script``'s ``tls-config`` option.



Build Vinyl Cache from sources
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``build`` is based on
`zc.recipe.cmmi <https://pypi.python.org/pypi/zc.recipe.cmmi>`_ - so all
parameters from that recipe are available here too (but rarely used). These options are available for the recipe part plone.recipe.vinylcache:build.

Three parameters are different/ extra:

``url``
    Location used for download of Vinyl Cache sources. Defaults to the
    current stable 9.0.x release.

``jobs``
    Passes the number of parallel jobs to ``make``, defaults to ``4``. Adjust as
    needed to your CPU resources.

``compile-vmods``
    Boolean flag defaults to False, used for building Varnish/Vinyl Cache
    modules. By defaults, the modules are from `the module collection by
    Varnish Software <https://github.com/varnish/varnish-modules/releases>`_
    (note: that module collection dropped support for Varnish 6.0 LTS; the
    release this recipe defaults to targets the 9.0.x line).

VCL Configuration Generator
~~~~~~~~~~~~~~~~~~~~~~~~~~~

These options are available for the recipe part plone.recipe.vinylcache:configuration.

``backends``
    Specifies the backend or backends which will process the (uncached)
    requests. The syntax for backends:

    ``[<hostname>][/<path>]:<ip address>:<port>``

    The optional ``hostname`` and ``path`` allows you to do virtual hosting.
    If multiple backends are specified then each backend must include
    either a hostname or path (or both) so that Vinyl Cache can direct the
    matching request to the appropriate backend. Defaults to
    ``127.0.0.1:8080``.

``balancer``
    If included and set to ``random``, ``round_robin`` or ``shard``, this
    option configures Vinyl Cache to load balance the servers specified
    by the ``backends`` directive. Possible values: ``none`` (default),
    ``round_robin``, ``random`` or ``shard``. ``shard`` uses Vinyl
    Cache's consistent-hashing director: the same request (by default
    keyed on ``client.identity``) always lands on the same backend,
    which gives a much better cache hit ratio than ``round_robin``/
    ``random`` when several backends could each independently cache the
    same content.

``between-bytes-timeout``
    If specified, this option configures the timeout (in seconds) for Vinyl
    Cache waiting between bytes when receiving data from a backend. Vinyl
    Cache will only wait this many seconds between bytes before giving up. A
    value of 0s means this will never time out. Defaults to *60s*, as per
    Vinyl Cache's default settings.

``bind``
    Hostname and port on which Vinyl Cache will listen for requests. Defaults
    to ``127.0.0.1:8000``.

``connect-timeout``
    If specified, this option configures the connection timeout (in seconds)
    for Vinyl Cache connecting to a backend server. Vinyl Cache will only try
    to connect to a given backend for this many seconds before giving up.
    Defaults to 0.4s, as per Vinyl Cache's default settings.

``cookie-pass``
    This list consists of lines with a cookie-match and urlexclude in the form:
    ``"cookiematch":"urlexcludes"``. If *cookiematch* applies for the cookiename,
    then the request is passed directly to the configured backend
    bypassing any caching. Additionally, if the current url matches urlexcludes,
    then the cookies are removed, and the request piped to the backend.
    Defaults are optimized for Plone, one line:
    ``"auth_token|__ac(|_(name|password|persistent))=":"\.(js|css|woff|woff2)$"``
    So when you are authenticated, the request is always handled by Plone.
    When an authenticated user requests a js/css/kss file,
    Plone will see you as anonymous because no cookies reach Plone.

``cookie-pass-not-exclude``
    If url matches this regexp, ``cookie-pass`` exclude rules are skipped. This is useful
    for url like ``++resource++zmi`` that requires authentication also for resources
    like js, css, ...

``cookie-whitelist``
    When the ``cookie-pass`` is processed and does not match, this means you are
    anonymous, at least with the default ``cookie-pass`` settings.
    In that case, this whitelist is used to to sanitize cookie data on the request.
    Cookie data to be sent to the backend includes only cookies with the given names.
    Defaults are optimized for Zope2/Plone:
    ``statusmessages __ac _ZopeId __cp auth_token``
    The ``__ac`` and ``auth_token`` cookies should not be needed, as they are
    already in the ``cookie-pass`` list, but they are here for safety in case
    you have customized the ``cookie-pass`` setting to not include them.
    If you have custom code that sets cookies and needs to read them in the backend,
    then you must add the cookie names to this list.

``first-byte-timeout``
    If specified, this option configures the timeout (in seconds) for Vinyl
    Cache receiving the first byte from a backend. Vinyl Cache will only wait
    for this many seconds before giving up. A value of 0s means Vinyl Cache
    will never time out. Defaults to 300s.

``purge-hosts``
    Specifies hostnames or IP addresses for purge ACL. By default ``localhost`` and
    the backends are allowed to purge. Additional allowed hosts are listed here.

``purge-by-id``
    Enables `collective.purgebyid
    <https://github.com/collective/collective.purgebyid>`_-compatible
    purging: a backend response carrying an
    ``X-Ids-Involved: #uuid1#uuid2#...#`` header gets those ids
    associated with the cached object, and
    ``GET /@@purgebyid/<id>`` (from an IP allowed by ``purge-hosts``)
    purges every cached object tagged with that id -- without needing to
    enumerate every cached URL variant of that content. Possible values:

    * ``off`` (default): disabled.
    * ``ban`` (also ``on``): purges via a ``ban()`` matching the
      ``X-Ids-Involved`` header. No vmod required, works everywhere.
    * ``xkey``: purges via the ``xkey`` vmod's secondary-key support
      instead of a ban scan; more efficient on busy caches, but requires
      ``[varnish-build] compile-vmods = true`` (the ``xkey`` VCL import
      is only emitted in this mode, so ``off``/``ban`` never break
      compilation for setups that haven't built vmods).

``max-cacheable-size``
    Objects whose backend response ``Content-Length`` exceeds this value
    are never cached (``beresp.uncacheable = true``). Unset (default): no
    size limit. Value must be a valid VCL BYTES literal, e.g. ``50MB`` or
    ``1GB``. Without a limit, a single object near or above the cache's
    size can, under LRU pressure, evict a large fraction of the cache
    just to make room for itself.

``verbose-headers``
    Enable sending extra diagnostic response headers (``X-Cache``,
    ``X-Cacheable``, ``grace``) that expose what Vinyl Cache did with the
    request and the cache status. Useful for debugging cache settings and
    optimizations; leave off in production to avoid exposing cache
    internals to clients. Possible values: ``on`` or ``off`` (default).

``vcl_recv``, ``vcl_hit``, ``vcl_miss``, ``vcl_backend_fetch``, ``vcl_backend_response``, ``vcl_deliver``, ``vcl_pipe``, ``vcl_purge``, ``vcl_hash``, ``vcl_import``, ``vcl_init``, ``vcl_pass``, ``vcl_synth``
    Insert arbitrary VCL code into the generated config.

``zope2_vhm_map``
    Defines a virtual host mapping for Zope servers. This is a list of
    ``hostname:ZODB location`` entries which specify the location inside
    Zope where the website for a virtual host lives.

``zope2_vhm_port``
    Defines a virtual host mapping port to use in the VHM URL to send back to
    clients. Useful if there is another port mapping in front of Vinyl Cache,
    such as haproxy. Defaults to bind port.

``zope2_vhm_ssl``
    If specified, this maps VHM URLs to ``https`` for all requests.
    Possible values: ``on`` or ``off`` (default).

``zope2_vhm_ssl_port``
    Defines a virtual host mapping port to use in the VHM URL to send back to
    clients. Useful if there is another port mapping in front of Vinyl Cache,
    such as haproxy. Defaults to 443.

This recipe always generates VCL syntax version 4.1; there is no
``vcl-version`` option.

``health-probe-*``
    Settings for backend health probes. Probes are activated if `grace-healthy` is set.

    See the `VCL reference on the Vinyl Cache documentation site
    <https://vinyl-cache.org/docs/9.0/reference/vcl.html#probes>`_ for a
    detailed explanation of each setting.

    * `health-probe-url`: defaults to ``/ok``
    * `health-probe-timeout`: defaults to ``5s``
    * `health-probe-interval`: defaults to ``15s``
    * `health-probe-window`: defaults to ``10``
    * `health-probe-threshold`: defaults to ``8``
    * `health-probe-initial`: If not given Vinyl Cache will default to `threshold -1`

``grace-healthy``
    Grace in the context of Vinyl Cache means delivering otherwise expired
    objects when circumstances call for it. This can happen because:
    (1) the backend-director selected is down, or
    (2) a different thread has already made a request to the backend that's
    not yet finished.

    If the backend is healthy, accept objects that are this number of seconds
    old. Clients will be delivered content that is no more than number of
    seconds past its TTL.

    Format: number followed by a time unit: ms, s, m, h.

    Defaults to ``None``. If this is set to ``None`` the grace
    feature is disabled.

``grace-sick``
    If the backend is sick, accept objects that are this old.
    See also ``grace-healthy``.

    Format: number followed by a time unit: ms, s, m, h.

    Defaults to ``600s``. Should be greater than ``grace-healthy``.


To test the generated configuration for syntactic correctness, run
``varnishd -C -f ./parts/varnish-configuration/varnish.vcl``.


Create script to start Vinyl Cache
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Start Vinyl Cache as a daemon or in foreground with the given settings. These options are available for the recipe part plone.recipe.vinylcache:script.

``bind``
    Hostname and port on which Vinyl Cache will listen for requests. Defaults
    to ``127.0.0.1:8000``.

``build-part``
    References the buildout part in order to get settings from there. Defaults
    to ``varnish-build``. Set it to ``false`` in order to switch it off.

``cache-location``
    Customise the location for the Vinyl Cache file storage.  Option only
    applicable when used with the ``file`` cache-type option.  Defaults to
    using a file named ``storage`` inside the relevant parts directory
    (eg ``parts/varnish/storage``).  Changing the default location can be
    useful in putting the storage somewhere with quicker read speeds
    (e.g. RAM disk).

``cache-size``
    The size of the cache (limited to 2G on 32bit systems). Defaults to
    256M.

``cache-type``
    Specify the type of cache storage to use with Vinyl Cache.
    Possible values: ``file`` (storage for each object is allocated from an
    arena backed by a file), ``malloc`` (storage for each object is
    allocated with malloc; in memory), or other storage engines supported by
    your Vinyl Cache build (e.g. ``umem``).
    Defaults to ``file``.

``configuration-file``
    Path to a VCL configuration file to use. Defaults to the generated
    file from the ``configuration-part`` setting.
    If no configuration was generated, this setting is mandatory.

``configuration-part``
    Names the buildout part to get settings from.
    Defaults to ``varnish-configuration``.

``daemon``
    The file and path of the Vinyl Cache daemon ``varnishd`` to use (the
    installed binary is still called ``varnishd`` as of Vinyl Cache 9.0.x,
    despite the project/brand rename).
    If not given, it looks for the build part
    (see the ``build-part`` setting)
    and uses its ``location`` setting plus the string ``/sbin/varnishd``.
    If there is no build part, it defaults to ``/usr/sbin/varnishd`` - the
    most common place
    where it's found on many Unix systems. Adjust it if needed.

``group``
    The name of the group that Vinyl Cache should switch to before accepting
    any request. This defaults to the main group for the specified user.

``mode``
    Specify whether the Vinyl Cache daemon should run in ``daemon`` or
    ``foreground`` mode.  The latter is useful when Vinyl Cache is run by
    service supervision tools like daemontools or runit. Defaults to
    ``daemon``.

``name``
    Sets the name of the Vinyl Cache instance. Defaults to
    ``${buildout:directory}/var/<this part's name>``, **not**
    ``varnishd``'s own system default -- which is typically somewhere
    under ``/var/run`` and requires root, the single most common
    "Permission denied" trap when running ``varnishd`` unprivileged from
    a buildout. Deliberately not placed under ``parts`` (that tree is
    meant to be disposable/regenerated, e.g. whenever ``varnish-build``
    is recompiled, which would lose this runtime state) and keyed by the
    part's own name so multiple Vinyl Cache instances in the same
    buildout don't collide. Set this explicitly if you want the
    traditional host-name-based default or a different location.

    From ``varnishd``'s manpage:

      Amongst other things, this name is used to construct the name of the
      directory in which varnishd keeps temporary files and persistent state.
      If the specified name begins with a forward slash, it is interpreted as
      the absolute path to the directory which should be used for this purpose.

``runtime-parameters``
    Runtime parameter configuration options. The full list of available options
    can be found in the manpage varnishd(1) for your version of Vinyl Cache.
    Examples include ``thread_pool_max``, ``thread_pool_min``, ``sess_timeout``.

``telnet``
    If specified sets the hostname and port on which Vinyl Cache will listen
    for commands using its telnet interface.

``tls-config``
    Path to a hitch-style TLS configuration file, mapping to
    ``varnishd -A`` (a Vinyl Cache 9.0 addition). Lets ``varnishd``
    terminate TLS itself instead of needing a separate TLS terminator
    (e.g. Hitch, nginx, haproxy) in front of it. See the
    ``plone.recipe.vinylcache:selfsigned`` recipe below for a quick way
    to generate a matching self-signed certificate and config file for
    internal/dev/testing use. Not set by default (no ``-A`` flag is
    emitted).

``script-filename``
    Name of the start script file in ``buildout:bin-directory``.
    Defaults to the name of this buildout part.

``secret-file``

    The telnet interface is not usable without authentication by default. A
    pre shared key mechanism has been put in place which requires both the
    Vinyl Cache daemon and a client connection over telnet (like the
    ``varnishadm`` tool) to have a shared key to authenticate.
    By default if no secret-file is specified, it's no longer possible to
    authenticate to the telnet interface.

    To disable this security feature use ``secret-file = disabled``. This is
    discouraged.

    To enable the secret-file, give the path to a file on the filesystem that
    preferably has random content and is both accessible to the Vinyl Cache
    daemon and a command line utility like ``varnishadm``.

    An example buildout part to generate such a file could be::

        [varnish-secret]
        recipe = plone.recipe.command
        command = dd if=/dev/random of=${buildout:directory}/var/varnish_secret count=1
                  chmod 600 ${buildout:directory}/var/varnish_secret

    Giving secret-file the location of this file will pass on the secret to
    the Vinyl Cache daemon when it starts up. Afterwards you can use
    ``varnishadm`` with the parameters -T host:port -S /path/to/varnish_secret
    to connect to the admin telnet interface.

``user``
    The name of the user Vinyl Cache should switch to before accepting any
    requests. Defaults to ``nobody``.


Generate a self-signed TLS certificate
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``plone.recipe.vinylcache:selfsigned`` generates a self-signed
certificate/private key (via the ``openssl`` command line tool, which
must be available on ``PATH``) and a ready-to-use hitch-style config
file, for pairing with the ``script`` part's ``tls-config`` option.
Meant for internal/dev/testing use -- being self-signed, clients need to
explicitly trust this certificate (or ignore validation errors); it is
**not** a substitute for a CA-issued certificate on anything
internet-facing.

Generation is idempotent: an already-present certificate/key is not
regenerated (and so not rotated/invalidated) on later buildout runs.

Example::

    [varnish-tls-cert]
    recipe = plone.recipe.vinylcache:selfsigned
    common-name = internal.example.org
    bind = *:8443

    [varnish-script]
    recipe = plone.recipe.vinylcache:script
    tls-config = ${varnish-tls-cert:config-file}

``bind``
    Hostname (or ``*``) and port the generated hitch-style config file's
    ``frontend`` block will listen on. Defaults to ``*:8443``.

``common-name``
    The ``CN`` (Common Name) of the self-signed certificate. Defaults to
    ``localhost``.

``days``
    Certificate validity, in days. Defaults to ``3650`` (10 years) --
    long-lived since this is meant for internal/dev use where rotation
    ceremony isn't the point.

``key-size``
    RSA key size in bits. Defaults to ``2048``.

``key-file``, ``cert-file``, ``combined-file``, ``config-file``
    Output paths for, respectively: the private key, the certificate,
    the two concatenated together (what the hitch-style config's
    ``pem-file`` directive points at), and the hitch-style config file
    itself (what you point ``tls-config`` at). All default to sensible
    locations inside this part's own ``parts`` directory.


.. _Vinyl Cache: https://vinyl-cache.org/
.. _plone.recipe.varnish: https://pypi.org/project/plone.recipe.varnish/
.. _zc.buildout: https://pypi.org/project/zc.buildout/

Examples:
---------

Use system Vinyl Cache at ``/usr/sbin/varnishd``, generate start script in
``./bin/varnish`` using a VCL-file in
``./parts/varnish-configuration/varnish.vcl``::

    [buildout]
    parts =
        varnish-script
        varnish-configuration

    [varnish-script]
    recipe = plone.recipe.vinylcache:script

    [varnish-configuration]
    recipe = plone.recipe.vinylcache:configuration


Credits
-------

``plone.recipe.vinylcache`` is a fork of `plone.recipe.varnish`_, created to
track the Varnish Cache -> Vinyl Cache project rename and the 9.0.x release
line. All credit for the original design, the VCL generation approach and
the years of maintenance goes to the ``plone.recipe.varnish`` authors and
contributors. 

If you don't need Vinyl Cache 9.0.x and are happy on Varnish 6.0 LTS, please 
use `plone.recipe.varnish`_ directly; it is unaffected by this fork and keeps
receiving its own maintenance.

Changelog
=========

9.0.3.0 (2026-09-25)
--------------------

- Moved the package source to a ``src`` layout and migrated packaging
  metadata from ``setup.py`` to ``pyproject.toml`` (``setup.py`` now only
  carries the legacy ``namespace_packages``/``packages``/``package_dir``
  arguments that PEP 621's ``[project]`` table cannot express). Dropped
  Python 3.9 support, added 3.12 and 3.13.
  [mamico]

- BUGFIX: ``plone.recipe.vinylcache:build``'s ``url`` (and
  ``vmods_url``) option now falls back to the built-in default download
  URL when set to an *empty* string, not just when entirely absent. An
  inherited/extended ``buildout.cfg`` declaring ``url =`` with no value
  as a placeholder for a downstream override previously made
  ``setdefault()`` a no-op (the key already existed), which
  ``zc.recipe.cmmi`` then handed to ``zc.buildout.download.Download()``
  as if it were a local path, failing with ``FileNotFoundError: [Errno
  2] No such file or directory: ''`` instead of using the default.
  [mamico]

- BUGFIX: the new default ``name`` (``${buildout:directory}/var/<part
  name>``, see above) broke ``varnishd`` startup and even ``varnishd -C``
  syntax checks with "Error: Cannot create working directory ...: No
  such file or directory": ``varnishd`` only creates the leaf directory
  of its ``-n`` working directory, not missing parents (e.g. ``var/``
  itself). ``plone.recipe.vinylcache:script`` now creates that directory
  (and any missing parents) itself during install. Caught by actually
  running the compiled binary against the generated script in CI/locally,
  not just by inspecting the VCL. [mamico]

- New features, beyond parity with ``plone.recipe.varnish``:

  - ``vcl_hash`` default now includes ``req.http.host`` (falling back to
    ``server.ip``), matching Varnish's own built-in default. The previous
    default hashed on ``req.url`` alone, which can cross-contaminate the
    cache between two different vhosts/backends serving overlapping URL
    paths on the same Vinyl Cache instance.
  - ``PATCH`` added to the method whitelist in ``vcl_recv`` (alongside
    ``PUT``/``POST``/``DELETE``), for REST APIs (e.g. ``plone.restapi``).
  - WebSocket upgrade requests (``Upgrade: websocket``) are now piped
    straight through in ``vcl_recv`` instead of falling into normal
    GET/HEAD caching logic.
  - ``Accept-Encoding`` is normalized in ``vcl_recv`` (to ``gzip`` or
    unset) to avoid fragmenting the cache per client-specific
    ``Accept-Encoding`` strings.
  - Large files (by extension) are now streamed (``beresp.do_stream``)
    instead of piped in ``vcl_backend_response`` -- streaming keeps the
    response cacheable and visible to logging, unlike pipe.
  - ``balancer`` accepts a new value, ``shard``, using Varnish's
    consistent-hashing director: the same request lands on the same
    backend every time, which is better for cache hit ratio than
    ``round_robin``/``random`` when several backends could each
    independently cache the same content. Emits the documented
    ``.reconfigure()`` call after adding backends.
  - The purge ACL (``acl list_purge``) now sets ``+fold(-report)``,
    keeping Vinyl Cache 9.0's default ACL-folding optimization but
    silencing the (harmless but noisy) compiler warning it emits for
    common setups, e.g. a backend on 127.0.0.1 overlapping with the
    "localhost" entry.
  - ``verbose-headers`` is a real, working option again (it was removed
    earlier in this fork's history because it was dead code in the
    version it was forked from). When ``off`` (the default), the
    diagnostic ``X-Cache``, ``X-Cacheable`` and ``grace`` response
    headers are stripped before delivery; set to ``on`` to keep them
    for debugging.
  - New ``purge-by-id`` option (default ``off``) for
    ``plone.recipe.vinylcache:configuration``, compatible with
    `collective.purgebyid <https://github.com/collective/collective.purgebyid>`_:
    ``GET /@@purgebyid/<id>`` purges every cached object tagged with
    that id (via the backend's ``X-Ids-Involved`` header) without
    needing to enumerate every cached URL variant of that content. Two
    modes: ``ban`` (also ``on``, the default when enabled) purges via
    ``ban()`` and needs no vmod, matching a real-world production
    pattern found in the wild that avoids the vmod-compile step
    entirely; ``xkey`` purges via the ``xkey`` vmod's secondary-key
    support instead (more efficient, requires
    ``[varnish-build] compile-vmods = true`` -- the ``xkey`` import is
    only emitted in this mode).
  - New ``tls-config`` option for ``plone.recipe.vinylcache:script``,
    mapping to ``varnishd -A`` (a Vinyl Cache 9.0 addition letting
    ``varnishd`` terminate TLS itself via a hitch-like config file,
    instead of needing a separate TLS terminator in front of it).
  - New ``max-cacheable-size`` option for
    ``plone.recipe.vinylcache:configuration`` (default: unset, no limit).
    Objects whose backend response ``Content-Length`` exceeds this value
    (a VCL BYTES literal, e.g. ``50MB``) are marked uncacheable in
    ``vcl_backend_response``. Without a limit, a single very large
    object can nuke a large fraction of the cache under LRU pressure
    just to make room for itself, evicting many still-useful smaller
    objects along the way.
  - New ``plone.recipe.vinylcache:selfsigned`` recipe: generates a
    self-signed certificate/key (via ``openssl``, idempotently -- it
    won't regenerate an already-present certificate on later buildout
    runs) plus a ready-to-use ``-A``-style config file, for pairing with
    ``tls-config`` in internal/dev/testing setups where a CA-issued
    certificate isn't warranted. The key file, the combined cert+key
    file, and the directory holding them are all written with
    restricted permissions (0600/0600/0700) since they contain private
    key material.

  [mamico]

- Default ``plone.recipe.vinylcache:script``'s ``name`` option (which maps
  to ``varnishd -n``, controlling its working directory) to
  ``${buildout:directory}/var/<part name>`` instead of leaving it unset.
  Without it, ``varnishd`` picks its own system default working directory
  (typically under ``/var/run``), which requires root and is the most
  common "Permission denied: Cannot create working directory" trap when
  running ``varnishd`` unprivileged from a buildout. Deliberately placed
  under ``var``, not ``parts`` (which is disposable/regenerated e.g. on a
  ``varnish-build`` recompile), and keyed by the part's own name so
  multiple instances in one buildout don't collide. Set ``name``
  explicitly to override. [mamico]

- BUGFIX: revert to legacy (``pkg_resources``-declared) ``plone``/
  ``plone.recipe`` namespace packages (restoring ``plone/__init__.py``
  and ``plone/recipe/__init__.py``, and ``namespace_packages=`` in
  ``setup.py``) instead of pure PEP 420 native namespaces. Real-world
  Plone buildouts still mix in many ``plone.*`` eggs that declare the
  namespace the legacy way; when this package was native-only, its
  develop egg became invisible to ``zc.buildout``'s ``pkg_resources``-
  based dependency/recipe resolution as soon as it was combined with
  those other eggs in the same buildout (reproduced with a real
  Plone-buildout-shaped setup: worked in isolation, failed with
  "Couldn't find index page for 'plone.recipe.vinylcache'" once other
  legacy-``plone``-namespace eggs were also present -- the exact same
  root cause as the ``zc.recipe.testrunner`` CI fix above, this time
  hitting real users). [mamico]

- CI: py39 is only tested against Plone 6.0 -- Plone's floating
  ``6.1-latest``/``6.2-latest`` requirements.txt now pin a ``pip``
  release requiring Python>=3.10. [mamico]

- Fork of `plone.recipe.varnish <https://pypi.org/project/plone.recipe.varnish/>`_,
  created to track the Varnish Cache -> Vinyl Cache project rename (early
  2026) and the new 9.0.x stable release line. All credit for the original
  recipe design and years of maintenance goes to the
  ``plone.recipe.varnish`` authors and contributors. If you don't need
  Vinyl Cache 9.0.x, please keep using ``plone.recipe.varnish`` -- it is
  unaffected by this fork and continues to target Varnish 6.0 LTS.
  [mamico]

- BREAKING: only support Vinyl Cache (formerly Varnish Cache) version
  9.0.x and generate config (VCL) for this version only. [mamico]

- BUGFIX: ``vcl_hit``'s grace-handling code used ``return(miss)``, which
  is no longer a valid VCL return action from ``vcl_hit`` as of VCL
  syntax 4.1 (confirmed against the real Vinyl Cache 9.0.3 VCC returns
  table -- discovered by actually compiling the generated VCL in CI, not
  just by reading docs). Replaced with ``return(restart)``, which
  re-enters VCL processing at ``vcl_recv`` and naturally resolves to a
  fresh fetch once an object's grace window has genuinely expired.
  [mamico]

- Drop Python 3.8 from the test matrix and ``python_requires`` (matches
  upstream ``plone.recipe.varnish``, which dropped it for the same
  reason: Plone's floating "-latest" requirements.txt files now pin
  setuptools/packaging releases with no Python 3.8 wheels). [mamico]

- Pin ``zc.recipe.egg`` to ``3.0.0`` in ``versions.cfg``: the current
  ``zc.recipe.egg`` (4.0.0, pulled in transitively by
  ``zc.recipe.testrunner``) requires ``zc.buildout>=5.0.0``, which
  conflicts with Plone 6.1's pinned ``zc.buildout==4.2.0``. [mamico]

- Pin ``zc.recipe.testrunner`` to ``3.2``: releases from 4.0 onwards
  dropped the ``namespace_packages.txt`` metadata shim that
  ``pkg_resources`` (still used internally by ``zc.buildout``) needs to
  merge a purely-PEP-420 package into a ``zc`` namespace that is
  otherwise legacy-declared -- which it is here, because of the
  ``zc.recipe.egg`` pin above. Without this, buildout's own entry-point
  loader fails with ``ModuleNotFoundError: No module named
  'zc.recipe.testrunner'`` even though the same package installs and
  imports fine via plain ``pip install``. Reproduced locally against both
  a Plone-6.0/6.1-like (``zc.buildout`` 4.x) and Plone-6.2-like
  (``zc.buildout`` 5.x) environment; 3.2 works in both. [mamico]

- Update the default download URL to Vinyl Cache 9.0.3 and the default
  ``varnish-modules`` URL to release 0.28.0 (the release targeting the
  9.0 line; ``varnish-modules`` dropped support for Varnish 6.0 LTS
  starting with the releases that target 7.0+). Note: despite the
  project's rename from Varnish Cache to Vinyl Cache, the installed
  daemon binary is still called ``varnishd`` as of 9.0.x (confirmed
  against the upstream source), so this recipe does *not* rename it;
  only the download defaults and the recipe's own package name track
  the new branding/release line. [mamico]

- BUGFIX: ``vcl_init`` and ``vcl_pass`` custom VCL snippet options were
  silently dropped by the recipe (present in the template and documented
  in the README, but missing from the option-collection list); they are
  now correctly included in the generated VCL. [mamico]

- BUGFIX: ``user``/``group`` options for ``plone.recipe.vinylcache:script``
  were silently ignored because of a dead, version-gated template branch
  left over from supporting older Varnish releases; they now correctly
  emit ``-j unix,user=...[,ccgroup=...]``. [mamico]

- Fix the internal VCL syntax version marker to be the string ``"4.1"``
  instead of the float ``4.0`` -- this recipe always generates VCL syntax
  version 4.1. [mamico]

- Remove the ``verbose-headers`` option: it computed a value that was
  never actually forwarded to the VCL template and so never had any
  effect. [mamico]

- Remove the ``saint-mode`` option: saint mode was removed from Varnish
  itself years ago, and this recipe's own implementation of it was
  already disabled. Setting ``saint-mode`` to anything other than
  ``off`` now raises a clear error instead of being silently ignored.
  [mamico]

- Remove the undocumented-in-code, docs-only ``vcl-version`` option; no
  code ever read it. This recipe always targets VCL syntax version 4.1.
  [mamico]

- Drop Python 2.7 support. Modernize packaging: native PEP 420 namespace
  packages (no more ``plone/__init__.py``/``plone/recipe/__init__.py``
  ``declare_namespace`` calls), remove the broken ``setup.py`` test
  command (``setuptools.command.test`` was removed in setuptools>=72),
  update classifiers/``python_requires`` to Python 3.8+. [mamico]

- Test against Plone 6.0, 6.1 and 6.2 (and their respective zc.buildout
  versions) in CI; drop the Plone 5.2 / Python 2.7 test variant.
  [mamico]

- Carried over from ``plone.recipe.varnish``'s own history (already fixed
  there, not novel to this fork): the ``cookie-pass-not-exclude`` config
  option and ``vcl_synth`` custom VCL insertion point (6.0.13); the
  hostname-matching regex character-class bug for ``hostname:path``
  backends, fixed the same way independently while porting (6.0.18); and
  the ``cookie-pass`` default extension list dropping ``kss`` in favour of
  ``woff``/``woff2`` (6.0.13.1, [erral]). [mamico]
