Planet Varnish

  • 2015-05-15 15:05 Lasse Karstensen

    The last couple of weeks we’ve been pretty busy making SSL/TLS support for Varnish Cache Plus 4. Now that the news is out, I can follow up with some notes here.

    The setup will be a TLS terminating proxy in front, speaking PROXY protocol to Varnish. Backend/origin support for SSL/TLS has been added, so VCP can now talk encrypted to your backends.

    On the client-facing side we are forking the abandoned TLS proxy called stud, and giving a new name: hitch.

    hitch will live on github as a standalone open source project, and we are happy to review patches/pull requests made by the community. Here is the source code: https://github.com/varnish/hitch

    We’ve picked all the important patches from the flora of forks, and merged it all into a hopefully stable tool. Some of the new stuff includes: TLS1.1, TLS1.2, SNI, wildcard certs, multiple listening sockets. See the CHANGES.rst file updates.

    Varnish Software will provide support on it for commercial uses, under the current Varnish Plus product package.


    by Lasse Karstensen at 2015-05-15 15:05
  • 2015-05-15 15:05 Lasse Karstensen

    The last couple of weeks we’ve been pretty busy making SSL/TLS support for Varnish Cache Plus 4. Now that the news is out, I can follow up with some notes here.

    The setup will be a TLS terminating proxy in front, speaking PROXY protocol to Varnish. Backend/origin support for SSL/TLS has been added, so VCP can now talk encrypted to your backends.

    On the client-facing side we are forking the abandoned TLS proxy called stud, and giving a new name: hitch.

    hitch will live on github as a standalone open source project, and we are happy to review patches/pull requests made by the community. Here is the source code: https://github.com/varnish/hitch

    We’ve picked all the important patches from the flora of forks, and merged it all into a hopefully stable tool. Some of the new stuff includes: TLS1.1, TLS1.2, SNI, wildcard certs, multiple listening sockets. See the CHANGES.rst file updates.

    Varnish Software will provide support on it for commercial uses, under the current Varnish Plus product package.


    by Planet Varnish at 2015-05-15 15:05
  • 2015-04-27 13:41 Mattias Geniar

    The post Varnish VCL: Case Insensitive Regex appeared first on ma.ttias.be.

    By default, a regex match in Varnish happens case sensitive. If you want to use a case insensitive check, you can use the (?i) flag.

    This is a normal, case sensitive regex check:

    if (req.http.host ~ "^domain.(be|nl|com)$") { ... }

    To make the same check case insensitive, add the (?i) modifier at the beginning.

    if (req.http.host ~ "(?i)^domain.(be|nl|com)$") { ... }

    Odd as it may be, some people type in their domain names with caps, and the browser does not convert those to lowercase on submit. Though in practice, every Host-header is treated as a lowercase value in all webserver configs.

    The post Varnish VCL: Case Insensitive Regex appeared first on ma.ttias.be.

    Related posts: Using Perl regex for faster grep Something I've only just noticed, but I find it interesting.... Combine Apache’s HTTP authentication with X-Forwarded-For IP whitelisting in Varnish Such a long title for a post. If you want... Explicitly Approving (Whitelisting) Cookies in Varnish With Libvmod-Cookie In all my previous Varnish 3.x configs, I've always used...
    by Mattias Geniar at 2015-04-27 13:41
  • 2015-04-14 14:18 Mattias Geniar

    The post Varnish Cache 3.0 Is End Of Life appeared first on ma.ttias.be.

    A year after the release of Varnish 4, version 3.0 has been declared end-of-life.

    A year has passed since the release of Varnish Cache 4.0.0.

    According to our normal release schedule for Varnish Cache, this means
    that the previous stable version will stop receiving regular
    maintenance.

    As of April 10th 2015, Varnish Cache 3.0 reached end of life (EOL) status.

    Please use this opportunity to upgrade to Varnish Cache 4.0.

    For paying Varnish Plus customers we'll support Varnish Cache 3.0 and
    Varnish Cache Plus 3.0 for at least another year. Please contact me
    directly if you have any questions in this regard.
    varnish-announce mailing list

    Seems fast. I must have missed the warning signals that Varnish 3.0 would be reaching end of life.

    If you're looking at upgrading to Varnish 4, here are a few useful links;

    Varnish VCL configuration templates for Varnish 4.0 Upgrading to Varnish 4 official documentation

    Either way, time to take the upgrade to Varnish 4.0 serious.

    The post Varnish Cache 3.0 Is End Of Life appeared first on ma.ttias.be.

    Related posts: Reload Varnish VCL without losing cache data You can reload the Varnish VCL configuration without actually restarting... Varnish 4.0.0 released together with configuration templates Good news! Today, Varnish 4.0.0 has been released!. Among the... Useful Varnish 3.0 Commands: one-liners with varnishtop and varnishlog Here are some useful commands if you're toying around with...
    by Mattias Geniar at 2015-04-14 14:18
  • 2015-03-15 22:26 Mattias Geniar

    The post Running Varnish 4.x on systemd appeared first on ma.ttias.be.

    If you're thinking about running Varnish 4.x on a systemd system, you may be surprised that many of your "older" configs no longer work.

    Now I don't mean the actual VCL files, those have a seriously changed syntax and there are proper documentations on handling a 3.x to 4.x upgrade.

    I mean the /etc/sysconfig/varnish config, that will no longer work in a systemd world. It's being replaced by a /etc/varnish/varnish.params file, that is being included by systemd.

    To see what's going on under the hood, check out the systemd configuration file at /usr/lib/systemd/system/varnish.service.

    $ cat /usr/lib/systemd/system/varnish.service [Unit] Description=Varnish a high-perfomance HTTP accelerator After=syslog.target network.target [Service] # Maximum number of open files (for ulimit -n) LimitNOFILE=131072 # Locked shared memory (for ulimit -l) # Default log size is 82MB + header LimitMEMLOCK=82000 # Maximum size of the corefile. LimitCORE=infinity EnvironmentFile=/etc/varnish/varnish.params Type=forking PIDFile=/var/run/varnish.pid PrivateTmp=true ExecStartPre=/usr/sbin/varnishd -C -f $VARNISH_VCL_CONF ExecStart=/usr/sbin/varnishd \ -P /var/run/varnish.pid \ -f $VARNISH_VCL_CONF \ -a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \ -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \ -t $VARNISH_TTL \ -u $VARNISH_USER \ -g $VARNISH_GROUP \ -S $VARNISH_SECRET_FILE \ -s $VARNISH_STORAGE \ $DAEMON_OPTS ExecReload=/usr/sbin/varnish_reload_vcl [Install] WantedBy=multi-user.target

    Most importantly, it loads the file /etc/varnish/varnish.params that can/should contain environment variables, that you can use to manipulate the systemd service.

    At the very end, it contains the $DAEMON_OPTS variable. Previous sysconfig files would have that contain the entire startup parameter for varnish, including the -a parameter (what port to listen on), -S (the secret file), ... etc. With the Varnish 4.x configs on systemd, the $DAEMON_OPTS should only contain the additional parameters that aren't already specified in the varnish.service file.

    For example, you should limit the varnish.params file to something like this.

    $ cat /etc/varnish/varnish.params # Varnish environment configuration description. This was derived from # the old style sysconfig/defaults settings RELOAD_VCL=1 VARNISH_VCL_CONF=/etc/varnish/default.vcl VARNISH_LISTEN_PORT=80 VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1 VARNISH_ADMIN_LISTEN_PORT=6082 VARNISH_SECRET_FILE=/etc/varnish/secret VARNISH_STORAGE="file,/var/lib/varnish/varnish_storage.bin,1G" VARNISH_TTL=120 VARNISH_USER=varnish VARNISH_GROUP=varnish #DAEMON_OPTS="-p thread_pool_min=5 -p thread_pool_max=500 -p thread_pool_timeout=300"

    If you're migrating from a sysconfig-world, one of the most important changes is that the systemd-config requires a user and group environment variable, which wasn't set previously.

    $ cat /etc/varnish/varnish.params ... VARNISH_USER=varnish VARNISH_GROUP=varnish ...

    For all other changed parameters in the $DAEMON_OPTS list, check out the Varnish man-pages (man varnishd) that contain very accurate documentations on what parameters are allowed and which have been changed.

    The post Running Varnish 4.x on systemd appeared first on ma.ttias.be.

    Related posts: Debug Varnish 4.x on systemd That Fails to Start So you're stuck in systemctl start varnish, now what? Well,...
    by Mattias Geniar at 2015-03-15 22:26
  • 2015-03-15 22:14 Mattias Geniar

    The post Debug Varnish 4.x on systemd That Fails to Start appeared first on ma.ttias.be.

    So you're stuck in systemctl start varnish, now what?

    Well, by default, systemd won't tell you much.

    $ systemctl start varnish Job for varnish.service failed. See 'systemctl status varnish.service' and 'journalctl -xn' for details.

    View the status of the service:

    $ systemctl status varnish varnish.service - Varnish a high-perfomance HTTP accelerator Loaded: loaded (/usr/lib/systemd/system/varnish.service; enabled) Active: failed (Result: exit-code) since Sun 2015-03-15 21:07:41 CET; 15s ago Process: 10062 ExecStart=/usr/sbin/varnishd -P /var/run/varnish.pid -f $VARNISH_VCL_CONF -a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} -t $VARNISH_TTL -u $VARNISH_USER -g $VARNISH_GROUP -S $VARNISH_SECRET_FILE -s $VARNISH_STORAGE $DAEMON_OPTS (code=exited, status=2) Process: 10049 ExecStartPre=/usr/sbin/varnishd -C -f $VARNISH_VCL_CONF (code=exited, status=0/SUCCESS) Main PID: 6187 (code=exited, status=0/SUCCESS) site.be varnishd[10049]: .miss_func = VGC_function_vcl_miss, site.be varnishd[10049]: .hit_func = VGC_function_vcl_hit, site.be varnishd[10049]: .deliver_func = VGC_function_vcl_deliver, site.be varnishd[10049]: .synth_func = VGC_function_vcl_synth, site.be varnishd[10049]: .backend_fetch_func = VGC_function_vcl_backend_fetch, site.be varnishd[10049]: .backend_response_func = VGC_function_vcl_backend_response, site.be varnishd[10049]: .backend_error_func = VGC_function_vcl_backend_error, site.be varnishd[10049]: .init_func = VGC_function_vcl_init, site.be varnishd[10049]: .fini_func = VGC_function_vcl_fini, site.be varnishd[10049]: };

    It'll show the message that Varnish failed to start, and it will show the last 10 lines of output the program sent to stdout/stderr. But in Varnish' case, that's just the compiled VCL and it won't actually tell you the error.

    To start, test the syntax of your Varnish 4 VCL file.

    $ varnishd -d -f /etc/varnish/default.vcl ... ----------------------------- Varnish Cache CLI 1.0 ----------------------------- ...

    If you see a "Varnish Cache CLI", your VCL compiled and is working. That means the problem could be in the way systemd starts its service.

    $ grep systemd /var/log/messages Mar 15 21:07:41 lb01 systemd: Starting Varnish a high-perfomance HTTP accelerator... Mar 15 21:07:41 lb01 systemd: varnish.service: control process exited, code=exited status=2 Mar 15 21:07:41 lb01 systemd: Failed to start Varnish a high-perfomance HTTP accelerator. Mar 15 21:07:41 lb01 systemd: Unit varnish.service entered failed state.

    So in this case, systemd failed to start the service with my requested parameters. Check your varnish.service in /usr/lib/systemd/system/varnish.service file for any typos or mis-configured environment variables and try again!

    The post Debug Varnish 4.x on systemd That Fails to Start appeared first on ma.ttias.be.

    Related posts: Running Varnish 4.x on systemd If you're thinking about running Varnish 4.x on a systemd...
    by Mattias Geniar at 2015-03-15 22:14
  • 2015-03-06 00:41 ingvar

    varnish-4.0.3 was released recently. I have wrapped packages for Fedora and EPEL, and requested updates for epel7, f21 and f22. They will trickle down as stable updates within some days. I have also built packages for el6, and after som small patching, even for el5. These builds are based on the Fedora package, but should be only cosmetically different from the el6 and el7 packages available from http://varnish-cache.org/.

    Also note that Red Hat finally caught up, and imported the necessary selinux-policy changes for Varnish from fedora into el7. With selinux-policy-3.13.1-23.el7, Varnish starts fine in enforcing mode. See RHBA-2015-0458.

    My builds for el5 and el6 are available here: http://users.linpro.no/ingvar/varnish/4.0.3/. Note that they need other packages from EPEL to work.

    Update 1: I also provide an selinux module for those running varnish-4.0 on el6. It should work for all versions of varnish-4.0, including mine and the ones from varnish-cache.org.

    Update 2: Updated builds with a patch for bugzilla ticket 1200034 are pushed for testing in f21, f22 and epel7. el5 and el6 builds are available on link above.

    Enjoy.

    Ingvar

    Varnish Cache is powerful and feature rich front side web cache. It is also very fast, that is, Fast as in on steroids, and powered by The Dark Side of the Force.

    Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at redpill-linpro.com.

    by Ingvar Hagelund at 2015-03-06 00:41
  • 2015-03-06 00:41 ingvar

    varnish-4.0.3 was released recently. I have wrapped packages for Fedora and EPEL, and requested updates for epel7, f21 and f22. They will trickle down as stable updates within some days. I have also built packages for el6, and after som small patching, even for el5. These builds are based on the Fedora package, but should be only cosmetically different from the el6 and el7 packages available from http://varnish-cache.org/.

    Also note that Red Hat finally caught up, and imported the necessary selinux-policy changes for Varnish from fedora into el7. With selinux-policy-3.13.1-23.el7, Varnish starts fine in enforcing mode. See RHBA-2015-0458.

    My builds for el5 and el6 are available here: http://users.linpro.no/ingvar/varnish/4.0.3/. Note that they need other packages from EPEL to work.

    Update 1: I also provide an selinux module for those running varnish-4.0 on el6. It should work for all versions of varnish-4.0, including mine and the ones from varnish-cache.org.

    Update 2: Updated builds with a patch for bugzilla ticket 1200034 are pushed for testing in f21, f22 and epel7. el5 and el6 builds are available on link above.

    Enjoy.

    Ingvar

    Varnish Cache is powerful and feature rich front side web cache. It is also very fast, that is, Fast as in on steroids, and powered by The Dark Side of the Force.

    Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at redpill-linpro.com.

    by Planet Varnish at 2015-03-06 00:41
  • 2015-02-18 17:32 Vincent ROBERT

    Notable changes:

    26 reported bugs fixed. Replaced objects are now expired immediately, instead of kept around until expiry. Memory usage on chunked backend responses is lower.

    More complete list of changes can be found at:

    https://www.varnish-cache.org/trac/browser/doc/changes.rst?rev=b8c4a34

    Source download:

    https://repo.varnish-cache.org/source/varnish-4.0.3.tar.gz

    by Dr Carter at 2015-02-18 17:32
  • 2015-01-19 12:38 Lasse Karstensen

    Dag has been working implementing support for HAProxy’s PROXY protocol[1] in Varnish. This is a protocol adds a small header on each incoming TCP connection that describes who the real client is, added by (for example) an SSL terminating process. (since srcip is the terminating proxy)

    We’re aiming for merging this into Varnish master (so perhaps in 4.1?) when it is ready.

    The code is still somewhat unfinished, timeouts are lacking and some polishing needed, but it works and can be played with in a development setup.

    Code can be found here: https://github.com/daghf/varnish-cache/tree/PROXY

    I think Dag is using haproxy to test it with. I’ve run it with stunnel (some connection:close issues to figure out still), and I’d love if someone could test it with ELB, stud or other PROXY implementations.

    1: http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt


    by Lasse Karstensen at 2015-01-19 12:38

Pages