123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474 |
- Redis cache backends
- ====================
- This package provides two different Redis cache backends. If you want to use
- Redis as cache backend, you have to choose one of the two, but you cannot use
- both at the same time. Well, it will be technically possible, but it would be
- quite a dumb thing to do.
- Predis
- ------
- This implementation uses the Predis PHP library. It is compatible PHP 5.3
- only.
- PhpRedis
- --------
- This implementation uses the PhpRedis PHP extention. In order to use it, you
- probably will need to compile the extension yourself.
- Redis version
- -------------
- This module requires Redis version to be 2.6.0 or later with LUA scrpting
- enabled due to the EVAL command usage.
- If you can't upgrade you Redis server:
- - 3.x release will only officially support Redis server <= 2.8 and 3.x
- nevertheless you may use it with Redis 2.4 if you configure your cache
- backend to operate in sharding mode.
- - For Redis 2.4 use the latest 2.x release of this module or use the
- 3.x release with sharding mode enabled.
- - For Redis <=2.3 use any version of this module <=2.6
- Notes
- -----
- Both backends provide the exact same functionalities. The major difference is
- because PhpRedis uses a PHP extension, and not PHP code, it will performe a
- lot better (Predis needs PHP userland code to be loaded).
- Difference is not that visible, it's really a few millisec on my testing box,
- in case you attempt to profile the code, traces will be a lot bigger.
- Note that most of the settings are shared. See next sections.
- Getting started
- ===============
- Quick setup
- -----------
- Here is a simple yet working easy way to setup the module.
- This method will allow Drupal to use Redis for all caches and locks
- and path alias cache replacement.
- $conf['redis_client_interface'] = 'PhpRedis'; // Can be "Predis".
- $conf['redis_client_host'] = '1.2.3.4'; // Your Redis instance hostname.
- $conf['lock_inc'] = 'sites/all/modules/redis/redis.lock.inc';
- $conf['path_inc'] = 'sites/all/modules/redis/redis.path.inc';
- $conf['cache_backends'][] = 'sites/all/modules/redis/redis.autoload.inc';
- $conf['cache_default_class'] = 'Redis_Cache';
- See next chapters for more information.
- Is there any cache bins that should *never* go into Redis?
- ----------------------------------------------------------
- TL;DR: No. Except for 'cache_form' if you use Redis with LRU eviction.
- Redis has been maturing a lot over time, and will apply different sensible
- settings for different bins; It's today very stable.
- Advanced configuration
- ======================
- Use the compressed cache
- ------------------------
- Please note this is for now an experimental feature. As a personnal note
- from the module author, it should be safe to use.
- Use this cache class setting to enable compression. This will save usually
- about 80% RAM at the cost of some milliseconds server time.
- $conf['cache_default_class'] = 'Redis_CacheCompressed';
- Additionnaly, you can alter the default size compression threshold, under which
- entries will not be compressed (size is in bytes, set 0 to always compress):
- $conf['cache_compression_size_threshold'] = 100;
- You can also change the compression level, which an positive integer between
- 1 and 9, 1 being the lowest but fastest compression ratio, 9 being the most
- aggressive compression but is a lot slower. From testing, setting it to the
- lower level (1) gives 80% memory usage decrease, which is more than enough.
- $conf['cache_compression_ratio'] = 5;
- Please note that those settings are global and not on a cache bin basis, you can
- already control whenever the compression is to be used or not by selecting a
- different cache class on per cache bin basis.
- If you switch from the standard default backend (without compression) to the
- compressed cache backend, it will recover transparently uncompressed data and
- proceed normally without additional cache eviction, it safe to upgrade.
- Donwgrading from compressed data to uncompressed data won't work, but the
- cache backend will just give you cache hit miss and it will work seamlessly
- too without any danger for the site.
- Choose the Redis client library to use
- --------------------------------------
- Add into your settings.php file:
- $conf['redis_client_interface'] = 'PhpRedis';
- You can replace 'PhpRedis' with 'Predis', depending on the library you chose.
- Note that this is optional but recommended. If you don't set this variable the
- module will proceed to class lookups and attempt to choose the best client
- available, having always a preference for the Predis one.
- Tell Drupal to use the cache backend
- ------------------------------------
- Usual cache backend configuration, as follows, to add into your settings.php
- file like any other backend:
- $conf['cache_backends'][] = 'sites/all/modules/redis/redis.autoload.inc';
- $conf['cache_class_cache'] = 'Redis_Cache';
- $conf['cache_class_cache_menu'] = 'Redis_Cache';
- $conf['cache_class_cache_bootstrap'] = 'Redis_Cache';
- // ... Any other bins.
- Tell Drupal to use the lock backend
- -----------------------------------
- Usual lock backend override, update you settings.php file as this:
- $conf['lock_inc'] = 'sites/all/modules/redis/redis.lock.inc';
- Tell Drupal to use the path alias backend
- -----------------------------------------
- Usual path backend override, update you settings.php file as this:
- $conf['path_inc'] = 'sites/all/modules/redis/redis.path.inc';
- Notice that there is an additional variable for path handling that is set
- per default which will ignore any path that is an admin path, gaining a few
- SQL queries. If you want to be able to set aliases on admin path and restore
- an almost default Drupal core behavior, you should add this line into your
- settings.php file:
- $conf['path_alias_admin_blacklist'] = FALSE;
- Drupal 6 and lock backend
- -------------------------
- Considering this is a Drupal 7 module only downloading it in Drupal 6 will make
- the module UI telling you this module is unsupported yet you can use the lock
- backend on Drupal 6.
- Read your Drupal 6 core documentation and use the redis.lock.inc file as
- lock_inc replacement the same way its being done for Drupal 7 and it should
- work. Note that this is untested by the module maintainer (feedback will be
- greatly appreciated).
- Common settings
- ===============
- Connect throught a UNIX socket
- ------------------------------
- All you have to do is specify this line:
- $conf['redis_client_socket'] = '/some/path/redis.sock';
- Both drivers support it.
- Connect to a remote host
- ------------------------
- If your Redis instance is remote, you can use this syntax:
- $conf['redis_client_host'] = '1.2.3.4';
- $conf['redis_client_port'] = 1234;
- Port is optional, default is 6379 (default Redis port).
- Using a specific database
- -------------------------
- Per default, Redis ships the database "0". All default connections will be use
- this one if nothing is specified.
- Depending on you OS or OS distribution, you might have numerous database. To
- use one in particular, just add to your settings.php file:
- $conf['redis_client_base'] = 12;
- Please note that if you are working in shard mode, you should never set this
- variable.
- Connection to a password protected instance
- -------------------------------------------
- If you are using a password protected instance, specify the password this way:
- $conf['redis_client_password'] = "mypassword";
- Depending on the backend, using a wrong auth will behave differently:
- - Predis will throw an exception and make Drupal fail during early boostrap.
- - PhpRedis will make Redis calls silent and creates some PHP warnings, thus
- Drupal will behave as if it was running with a null cache backend (no cache
- at all).
- Prefixing site cache entries (avoiding sites name collision)
- ------------------------------------------------------------
- If you need to differenciate multiple sites using the same Redis instance and
- database, you will need to specify a prefix for your site cache entries.
- Important note: most people don't need that feature since that when no prefix
- is specified, the Redis module will attempt to use the a hash of the database
- credentials in order to provide a multisite safe default behavior. This means
- that the module will also safely work in CLI scripts.
- Cache prefix configuration attemps to use a unified variable accross contrib
- backends that support this feature. This variable name is 'cache_prefix'.
- This variable is polymorphic, the simplest version is to provide a raw string
- that will be the default prefix for all cache bins:
- $conf['cache_prefix'] = 'mysite_';
- Alternatively, to provide the same functionality, you can provide the variable
- as an array:
- $conf['cache_prefix']['default'] = 'mysite_';
- This allows you to provide different prefix depending on the bin name. Common
- usage is that each key inside the 'cache_prefix' array is a bin name, the value
- the associated prefix. If the value is explicitely FALSE, then no prefix is
- used for this bin.
- The 'default' meta bin name is provided to define the default prefix for non
- specified bins. It behaves like the other names, which means that an explicit
- FALSE will order the backend not to provide any prefix for any non specified
- bin.
- Here is a complex sample:
- // Default behavior for all bins, prefix is 'mysite_'.
- $conf['cache_prefix']['default'] = 'mysite_';
- // Set no prefix explicitely for 'cache' and 'cache_bootstrap' bins.
- $conf['cache_prefix']['cache'] = FALSE;
- $conf['cache_prefix']['cache_bootstrap'] = FALSE;
- // Set another prefix for 'cache_menu' bin.
- $conf['cache_prefix']['cache_menu'] = 'menumysite_';
- Note that this last notice is Redis only specific, because per default Redis
- server will not namespace data, thus sharing an instance for multiple sites
- will create conflicts. This is not true for every contributed backends.
- Sharding vs normal mode
- -----------------------
- Per default the Redis cache backend will be in "normal" mode, meaning that
- every flush call will trigger and EVAL lua script that will proceed to cache
- wipeout and cleanup the Redis database from stalled entries.
- Nevertheless, if you are working with a Redis server < 2.6 or in a sharded
- environment, you cannot multiple keys per command nor proceed to EVAL'ed
- scripts, you will then need to switch to the sharded mode.
- Sharded mode will never delete entries on flush calls, but register a key
- with the current flush time instead. Cache entries will then be deleted on
- read if the entry checksum does not match or is older than the latest flush
- call. Note that this mode is fast and safe, but must be used accordingly
- with the default lifetime for permanent items, else your Redis server might
- keep stalled entries into its database forever.
- In order to enable the sharded mode, set into your settings.php file:
- $conf['redis_flush_mode'] = 3;
- Please note that the value 3 is there to keep backward compatibility with
- older versions of the Redis module and will not change.
- Note that previous Redis module version allowed to set a per-bin setting for
- the clear mode value; nevertheless the clear mode is not a valid setting
- anymore and the past issues have been resolved. Only the global value will
- work as of now.
- Sharding and pipelining
- -----------------------
- Whe using this module with sharding mode you may have a sharding proxy able to
- do command pipelining. If that is the case, you should switch to "sharding with
- pipelining" mode instead:
- $conf['redis_flush_mode'] = 4;
- Note that if you use the sharding mode because you use an older version of the
- Redis server, you should always use this mode to ensure the best performances.
- Default lifetime for permanent items
- ------------------------------------
- Redis when reaching its maximum memory limit will stop writing data in its
- storage engine: this is a feature that avoid the Redis server crashing when
- there is no memory left on the machine.
- As a workaround, Redis can be configured as a LRU cache for both volatile or
- permanent items, which means it can behave like Memcache; Problem is that if
- you use Redis as a permanent storage for other business matters than this
- module you cannot possibly configure it to drop permanent items or you'll
- loose data.
- This workaround allows you to explicity set a very long or configured default
- lifetime for CACHE_PERMANENT items (that would normally be permanent) which
- will mark them as being volatile in Redis storage engine: this then allows you
- to configure a LRU behavior for volatile keys without engaging the permenent
- business stuff in a dangerous LRU mechanism; Cache items even if permament will
- be dropped when unused using this.
- Per default the TTL for permanent items will set to safe-enough value which is
- one year; No matter how Redis will be configured default configuration or lazy
- admin will inherit from a safe module behavior with zero-conf.
- For advanturous people, you can manage the TTL on a per bin basis and change
- the default one:
- // Make CACHE_PERMANENT items being permanent once again
- // 0 is a special value usable for all bins to explicitely tell the
- // cache items will not be volatile in Redis.
- $conf['redis_perm_ttl'] = 0;
- // Make them being volatile with a default lifetime of 1 year.
- $conf['redis_perm_ttl'] = "1 year";
- // You can override on a per-bin basis;
- // For example make cached field values live only 3 monthes:
- $conf['redis_perm_ttl_cache_field'] = "3 months";
- // But you can also put a timestamp in there; In this case the
- // value must be a STRICTLY TYPED integer:
- $conf['redis_perm_ttl_cache_field'] = 2592000; // 30 days.
- Time interval string will be parsed using DateInterval::createFromDateString
- please refer to its documentation:
- http://www.php.net/manual/en/dateinterval.createfromdatestring.php
- Please also be careful about the fact that those settings are overriden by
- the 'cache_lifetime' Drupal variable, which should always be set to 0.
- Moreover, this setting will affect all cache entries without exception so
- be careful and never set values too low if you don't want this setting to
- override default expire value given by modules on temporary cache entries.
- Lock backends
- -------------
- Both implementations provides a Redis lock backend. Redis lock backend proved to
- be faster than the default SQL based one when using both servers on the same box.
- Both backends, thanks to the Redis WATCH, MULTI and EXEC commands provides a
- real race condition free mutexes by using Redis transactions.
- Queue backend
- -------------
- This module provides an experimental queue backend. It is for now implemented
- only using the PhpRedis driver, any attempt to use it using Predis will result
- in runtime errors.
- If you want to change the queue driver system wide, set this into your
- setting.php file:
- $conf['queue_default_class'] = 'Redis_Queue';
- $conf['queue_default_reliable_class'] = 'Redis_Queue';
- Note that some queue implementations such as the batch queue are hardcoded
- within Drupal and will always use a database dependent implementation.
- If you need to proceed with finer tuning, you can set a per-queue class in
- such way:
- $conf['queue_class_NAME'] = 'Redis_Queue';
- Where NAME is the arbitrary module given queue name, used as first parameter
- for the method DrupalQueue::get().
- THIS IS STILL VERY EXPERIMENTAL. The queue should work without any problems
- except it does not implement the item lease time correctly, this means that
- items that are too long to process won't be released back and forth but will
- block the thread processing it instead. This is the only side effect I am
- aware of at the current time.
- Failover, sharding and partionning
- ==================================
- Important notice
- ----------------
- There are numerous support and feature request issues about client sharding,
- failover ability, multi-server connection, ability to read from slave and
- server clustering opened in the issue queue. Note that there is not one
- universally efficient solution for this: most of the solutions require that
- you cannot use the MULTI/EXEC command using more than one key, and that you
- cannot use complex UNION and intersection features anymore.
- This module does not implement any kind of client side key hashing or sharding
- and never intended to; We recommend that you read the official Redis
- documentation page about partionning.
- The best solution for clustering and sharding today seems to be the proxy
- assisted partionning using tools such as Twemproxy.
- Current components state
- ------------------------
- As of now, provided components are simple enough so they never use WATCH or
- MULTI/EXEC transaction blocks on multiple keys : this means that you can use
- them in an environment doing data sharding/partionning. This remains true
- except when you use a proxy that blocks those commands such as Twemproxy.
- Lock
- ----
- Lock backend works on a single key per lock, it theorically guarantees the
- atomicity of operations therefore is usable in a sharded environement. Sadly
- if you use proxy assisted sharding such as Twemproxy, WATCH, MULTI and EXEC
- commands won't pass making it non shardable.
- Path
- ----
- Path backend does not use on transactions, it is safe to use in a sharded
- environment. Note that this backend uses a single HASH key per language
- and per way (alias to source or source to alias) and therefore won't benefit
- greatly if not at all from being sharded.
- Cache
- -----
- Cache uses pipelined transactions but does not uses it to guarantee any kind
- of data consistency. If you use a smart sharding proxy it is supposed to work
- transparently without any hickups.
- Queue
- -----
- Queue is still in development. There might be problems in the long term for
- this component in sharded environments.
- Testing
- =======
- Due to Drupal unit testing API being incredibly stupid, the unit tests can only
- work with PHP >=5.3 while the module will work gracefully with PHP 5.2 (at least
- using the PhpRedis client).
- I did not find any hint about making tests being configurable, so per default
- the tested Redis server must always be on localhost with default configuration.
|