README.txt 18 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474
  1. Redis cache backends
  2. ====================
  3. This package provides two different Redis cache backends. If you want to use
  4. Redis as cache backend, you have to choose one of the two, but you cannot use
  5. both at the same time. Well, it will be technically possible, but it would be
  6. quite a dumb thing to do.
  7. Predis
  8. ------
  9. This implementation uses the Predis PHP library. It is compatible PHP 5.3
  10. only.
  11. PhpRedis
  12. --------
  13. This implementation uses the PhpRedis PHP extention. In order to use it, you
  14. probably will need to compile the extension yourself.
  15. Redis version
  16. -------------
  17. This module requires Redis version to be 2.6.0 or later with LUA scrpting
  18. enabled due to the EVAL command usage.
  19. If you can't upgrade you Redis server:
  20. - 3.x release will only officially support Redis server <= 2.8 and 3.x
  21. nevertheless you may use it with Redis 2.4 if you configure your cache
  22. backend to operate in sharding mode.
  23. - For Redis 2.4 use the latest 2.x release of this module or use the
  24. 3.x release with sharding mode enabled.
  25. - For Redis <=2.3 use any version of this module <=2.6
  26. Notes
  27. -----
  28. Both backends provide the exact same functionalities. The major difference is
  29. because PhpRedis uses a PHP extension, and not PHP code, it will performe a
  30. lot better (Predis needs PHP userland code to be loaded).
  31. Difference is not that visible, it's really a few millisec on my testing box,
  32. in case you attempt to profile the code, traces will be a lot bigger.
  33. Note that most of the settings are shared. See next sections.
  34. Getting started
  35. ===============
  36. Quick setup
  37. -----------
  38. Here is a simple yet working easy way to setup the module.
  39. This method will allow Drupal to use Redis for all caches and locks
  40. and path alias cache replacement.
  41. $conf['redis_client_interface'] = 'PhpRedis'; // Can be "Predis".
  42. $conf['redis_client_host'] = '1.2.3.4'; // Your Redis instance hostname.
  43. $conf['lock_inc'] = 'sites/all/modules/redis/redis.lock.inc';
  44. $conf['path_inc'] = 'sites/all/modules/redis/redis.path.inc';
  45. $conf['cache_backends'][] = 'sites/all/modules/redis/redis.autoload.inc';
  46. $conf['cache_default_class'] = 'Redis_Cache';
  47. See next chapters for more information.
  48. Is there any cache bins that should *never* go into Redis?
  49. ----------------------------------------------------------
  50. TL;DR: No. Except for 'cache_form' if you use Redis with LRU eviction.
  51. Redis has been maturing a lot over time, and will apply different sensible
  52. settings for different bins; It's today very stable.
  53. Advanced configuration
  54. ======================
  55. Use the compressed cache
  56. ------------------------
  57. Please note this is for now an experimental feature. As a personnal note
  58. from the module author, it should be safe to use.
  59. Use this cache class setting to enable compression. This will save usually
  60. about 80% RAM at the cost of some milliseconds server time.
  61. $conf['cache_default_class'] = 'Redis_CacheCompressed';
  62. Additionnaly, you can alter the default size compression threshold, under which
  63. entries will not be compressed (size is in bytes, set 0 to always compress):
  64. $conf['cache_compression_size_threshold'] = 100;
  65. You can also change the compression level, which an positive integer between
  66. 1 and 9, 1 being the lowest but fastest compression ratio, 9 being the most
  67. aggressive compression but is a lot slower. From testing, setting it to the
  68. lower level (1) gives 80% memory usage decrease, which is more than enough.
  69. $conf['cache_compression_ratio'] = 5;
  70. Please note that those settings are global and not on a cache bin basis, you can
  71. already control whenever the compression is to be used or not by selecting a
  72. different cache class on per cache bin basis.
  73. If you switch from the standard default backend (without compression) to the
  74. compressed cache backend, it will recover transparently uncompressed data and
  75. proceed normally without additional cache eviction, it safe to upgrade.
  76. Donwgrading from compressed data to uncompressed data won't work, but the
  77. cache backend will just give you cache hit miss and it will work seamlessly
  78. too without any danger for the site.
  79. Choose the Redis client library to use
  80. --------------------------------------
  81. Add into your settings.php file:
  82. $conf['redis_client_interface'] = 'PhpRedis';
  83. You can replace 'PhpRedis' with 'Predis', depending on the library you chose.
  84. Note that this is optional but recommended. If you don't set this variable the
  85. module will proceed to class lookups and attempt to choose the best client
  86. available, having always a preference for the Predis one.
  87. Tell Drupal to use the cache backend
  88. ------------------------------------
  89. Usual cache backend configuration, as follows, to add into your settings.php
  90. file like any other backend:
  91. $conf['cache_backends'][] = 'sites/all/modules/redis/redis.autoload.inc';
  92. $conf['cache_class_cache'] = 'Redis_Cache';
  93. $conf['cache_class_cache_menu'] = 'Redis_Cache';
  94. $conf['cache_class_cache_bootstrap'] = 'Redis_Cache';
  95. // ... Any other bins.
  96. Tell Drupal to use the lock backend
  97. -----------------------------------
  98. Usual lock backend override, update you settings.php file as this:
  99. $conf['lock_inc'] = 'sites/all/modules/redis/redis.lock.inc';
  100. Tell Drupal to use the path alias backend
  101. -----------------------------------------
  102. Usual path backend override, update you settings.php file as this:
  103. $conf['path_inc'] = 'sites/all/modules/redis/redis.path.inc';
  104. Notice that there is an additional variable for path handling that is set
  105. per default which will ignore any path that is an admin path, gaining a few
  106. SQL queries. If you want to be able to set aliases on admin path and restore
  107. an almost default Drupal core behavior, you should add this line into your
  108. settings.php file:
  109. $conf['path_alias_admin_blacklist'] = FALSE;
  110. Drupal 6 and lock backend
  111. -------------------------
  112. Considering this is a Drupal 7 module only downloading it in Drupal 6 will make
  113. the module UI telling you this module is unsupported yet you can use the lock
  114. backend on Drupal 6.
  115. Read your Drupal 6 core documentation and use the redis.lock.inc file as
  116. lock_inc replacement the same way its being done for Drupal 7 and it should
  117. work. Note that this is untested by the module maintainer (feedback will be
  118. greatly appreciated).
  119. Common settings
  120. ===============
  121. Connect throught a UNIX socket
  122. ------------------------------
  123. All you have to do is specify this line:
  124. $conf['redis_client_socket'] = '/some/path/redis.sock';
  125. Both drivers support it.
  126. Connect to a remote host
  127. ------------------------
  128. If your Redis instance is remote, you can use this syntax:
  129. $conf['redis_client_host'] = '1.2.3.4';
  130. $conf['redis_client_port'] = 1234;
  131. Port is optional, default is 6379 (default Redis port).
  132. Using a specific database
  133. -------------------------
  134. Per default, Redis ships the database "0". All default connections will be use
  135. this one if nothing is specified.
  136. Depending on you OS or OS distribution, you might have numerous database. To
  137. use one in particular, just add to your settings.php file:
  138. $conf['redis_client_base'] = 12;
  139. Please note that if you are working in shard mode, you should never set this
  140. variable.
  141. Connection to a password protected instance
  142. -------------------------------------------
  143. If you are using a password protected instance, specify the password this way:
  144. $conf['redis_client_password'] = "mypassword";
  145. Depending on the backend, using a wrong auth will behave differently:
  146. - Predis will throw an exception and make Drupal fail during early boostrap.
  147. - PhpRedis will make Redis calls silent and creates some PHP warnings, thus
  148. Drupal will behave as if it was running with a null cache backend (no cache
  149. at all).
  150. Prefixing site cache entries (avoiding sites name collision)
  151. ------------------------------------------------------------
  152. If you need to differenciate multiple sites using the same Redis instance and
  153. database, you will need to specify a prefix for your site cache entries.
  154. Important note: most people don't need that feature since that when no prefix
  155. is specified, the Redis module will attempt to use the a hash of the database
  156. credentials in order to provide a multisite safe default behavior. This means
  157. that the module will also safely work in CLI scripts.
  158. Cache prefix configuration attemps to use a unified variable accross contrib
  159. backends that support this feature. This variable name is 'cache_prefix'.
  160. This variable is polymorphic, the simplest version is to provide a raw string
  161. that will be the default prefix for all cache bins:
  162. $conf['cache_prefix'] = 'mysite_';
  163. Alternatively, to provide the same functionality, you can provide the variable
  164. as an array:
  165. $conf['cache_prefix']['default'] = 'mysite_';
  166. This allows you to provide different prefix depending on the bin name. Common
  167. usage is that each key inside the 'cache_prefix' array is a bin name, the value
  168. the associated prefix. If the value is explicitely FALSE, then no prefix is
  169. used for this bin.
  170. The 'default' meta bin name is provided to define the default prefix for non
  171. specified bins. It behaves like the other names, which means that an explicit
  172. FALSE will order the backend not to provide any prefix for any non specified
  173. bin.
  174. Here is a complex sample:
  175. // Default behavior for all bins, prefix is 'mysite_'.
  176. $conf['cache_prefix']['default'] = 'mysite_';
  177. // Set no prefix explicitely for 'cache' and 'cache_bootstrap' bins.
  178. $conf['cache_prefix']['cache'] = FALSE;
  179. $conf['cache_prefix']['cache_bootstrap'] = FALSE;
  180. // Set another prefix for 'cache_menu' bin.
  181. $conf['cache_prefix']['cache_menu'] = 'menumysite_';
  182. Note that this last notice is Redis only specific, because per default Redis
  183. server will not namespace data, thus sharing an instance for multiple sites
  184. will create conflicts. This is not true for every contributed backends.
  185. Sharding vs normal mode
  186. -----------------------
  187. Per default the Redis cache backend will be in "normal" mode, meaning that
  188. every flush call will trigger and EVAL lua script that will proceed to cache
  189. wipeout and cleanup the Redis database from stalled entries.
  190. Nevertheless, if you are working with a Redis server < 2.6 or in a sharded
  191. environment, you cannot multiple keys per command nor proceed to EVAL'ed
  192. scripts, you will then need to switch to the sharded mode.
  193. Sharded mode will never delete entries on flush calls, but register a key
  194. with the current flush time instead. Cache entries will then be deleted on
  195. read if the entry checksum does not match or is older than the latest flush
  196. call. Note that this mode is fast and safe, but must be used accordingly
  197. with the default lifetime for permanent items, else your Redis server might
  198. keep stalled entries into its database forever.
  199. In order to enable the sharded mode, set into your settings.php file:
  200. $conf['redis_flush_mode'] = 3;
  201. Please note that the value 3 is there to keep backward compatibility with
  202. older versions of the Redis module and will not change.
  203. Note that previous Redis module version allowed to set a per-bin setting for
  204. the clear mode value; nevertheless the clear mode is not a valid setting
  205. anymore and the past issues have been resolved. Only the global value will
  206. work as of now.
  207. Sharding and pipelining
  208. -----------------------
  209. Whe using this module with sharding mode you may have a sharding proxy able to
  210. do command pipelining. If that is the case, you should switch to "sharding with
  211. pipelining" mode instead:
  212. $conf['redis_flush_mode'] = 4;
  213. Note that if you use the sharding mode because you use an older version of the
  214. Redis server, you should always use this mode to ensure the best performances.
  215. Default lifetime for permanent items
  216. ------------------------------------
  217. Redis when reaching its maximum memory limit will stop writing data in its
  218. storage engine: this is a feature that avoid the Redis server crashing when
  219. there is no memory left on the machine.
  220. As a workaround, Redis can be configured as a LRU cache for both volatile or
  221. permanent items, which means it can behave like Memcache; Problem is that if
  222. you use Redis as a permanent storage for other business matters than this
  223. module you cannot possibly configure it to drop permanent items or you'll
  224. loose data.
  225. This workaround allows you to explicity set a very long or configured default
  226. lifetime for CACHE_PERMANENT items (that would normally be permanent) which
  227. will mark them as being volatile in Redis storage engine: this then allows you
  228. to configure a LRU behavior for volatile keys without engaging the permenent
  229. business stuff in a dangerous LRU mechanism; Cache items even if permament will
  230. be dropped when unused using this.
  231. Per default the TTL for permanent items will set to safe-enough value which is
  232. one year; No matter how Redis will be configured default configuration or lazy
  233. admin will inherit from a safe module behavior with zero-conf.
  234. For advanturous people, you can manage the TTL on a per bin basis and change
  235. the default one:
  236. // Make CACHE_PERMANENT items being permanent once again
  237. // 0 is a special value usable for all bins to explicitely tell the
  238. // cache items will not be volatile in Redis.
  239. $conf['redis_perm_ttl'] = 0;
  240. // Make them being volatile with a default lifetime of 1 year.
  241. $conf['redis_perm_ttl'] = "1 year";
  242. // You can override on a per-bin basis;
  243. // For example make cached field values live only 3 monthes:
  244. $conf['redis_perm_ttl_cache_field'] = "3 months";
  245. // But you can also put a timestamp in there; In this case the
  246. // value must be a STRICTLY TYPED integer:
  247. $conf['redis_perm_ttl_cache_field'] = 2592000; // 30 days.
  248. Time interval string will be parsed using DateInterval::createFromDateString
  249. please refer to its documentation:
  250. http://www.php.net/manual/en/dateinterval.createfromdatestring.php
  251. Please also be careful about the fact that those settings are overriden by
  252. the 'cache_lifetime' Drupal variable, which should always be set to 0.
  253. Moreover, this setting will affect all cache entries without exception so
  254. be careful and never set values too low if you don't want this setting to
  255. override default expire value given by modules on temporary cache entries.
  256. Lock backends
  257. -------------
  258. Both implementations provides a Redis lock backend. Redis lock backend proved to
  259. be faster than the default SQL based one when using both servers on the same box.
  260. Both backends, thanks to the Redis WATCH, MULTI and EXEC commands provides a
  261. real race condition free mutexes by using Redis transactions.
  262. Queue backend
  263. -------------
  264. This module provides an experimental queue backend. It is for now implemented
  265. only using the PhpRedis driver, any attempt to use it using Predis will result
  266. in runtime errors.
  267. If you want to change the queue driver system wide, set this into your
  268. setting.php file:
  269. $conf['queue_default_class'] = 'Redis_Queue';
  270. $conf['queue_default_reliable_class'] = 'Redis_Queue';
  271. Note that some queue implementations such as the batch queue are hardcoded
  272. within Drupal and will always use a database dependent implementation.
  273. If you need to proceed with finer tuning, you can set a per-queue class in
  274. such way:
  275. $conf['queue_class_NAME'] = 'Redis_Queue';
  276. Where NAME is the arbitrary module given queue name, used as first parameter
  277. for the method DrupalQueue::get().
  278. THIS IS STILL VERY EXPERIMENTAL. The queue should work without any problems
  279. except it does not implement the item lease time correctly, this means that
  280. items that are too long to process won't be released back and forth but will
  281. block the thread processing it instead. This is the only side effect I am
  282. aware of at the current time.
  283. Failover, sharding and partionning
  284. ==================================
  285. Important notice
  286. ----------------
  287. There are numerous support and feature request issues about client sharding,
  288. failover ability, multi-server connection, ability to read from slave and
  289. server clustering opened in the issue queue. Note that there is not one
  290. universally efficient solution for this: most of the solutions require that
  291. you cannot use the MULTI/EXEC command using more than one key, and that you
  292. cannot use complex UNION and intersection features anymore.
  293. This module does not implement any kind of client side key hashing or sharding
  294. and never intended to; We recommend that you read the official Redis
  295. documentation page about partionning.
  296. The best solution for clustering and sharding today seems to be the proxy
  297. assisted partionning using tools such as Twemproxy.
  298. Current components state
  299. ------------------------
  300. As of now, provided components are simple enough so they never use WATCH or
  301. MULTI/EXEC transaction blocks on multiple keys : this means that you can use
  302. them in an environment doing data sharding/partionning. This remains true
  303. except when you use a proxy that blocks those commands such as Twemproxy.
  304. Lock
  305. ----
  306. Lock backend works on a single key per lock, it theorically guarantees the
  307. atomicity of operations therefore is usable in a sharded environement. Sadly
  308. if you use proxy assisted sharding such as Twemproxy, WATCH, MULTI and EXEC
  309. commands won't pass making it non shardable.
  310. Path
  311. ----
  312. Path backend does not use on transactions, it is safe to use in a sharded
  313. environment. Note that this backend uses a single HASH key per language
  314. and per way (alias to source or source to alias) and therefore won't benefit
  315. greatly if not at all from being sharded.
  316. Cache
  317. -----
  318. Cache uses pipelined transactions but does not uses it to guarantee any kind
  319. of data consistency. If you use a smart sharding proxy it is supposed to work
  320. transparently without any hickups.
  321. Queue
  322. -----
  323. Queue is still in development. There might be problems in the long term for
  324. this component in sharded environments.
  325. Testing
  326. =======
  327. Due to Drupal unit testing API being incredibly stupid, the unit tests can only
  328. work with PHP >=5.3 while the module will work gracefully with PHP 5.2 (at least
  329. using the PhpRedis client).
  330. I did not find any hint about making tests being configurable, so per default
  331. the tested Redis server must always be on localhost with default configuration.