README.txt 20 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404
  1. Search API
  2. ----------
  3. This module provides a framework for easily creating searches on any entity
  4. known to Drupal, using any kind of search engine. For site administrators, it is
  5. a great alternative to other search solutions, since it already incorporates
  6. facetting support and the ability to use the Views module for displaying search
  7. results, filters, etc. Also, with the Apache Solr integration [1], a
  8. high-performance search engine is available for use with the Search API.
  9. If you need help with the module, please post to the project's issue queue [2].
  10. [1] http://drupal.org/project/search_api_solr
  11. [2] http://drupal.org/project/issues/search_api
  12. Content:
  13. - Glossary
  14. - Information for users
  15. - Information for developers
  16. - Included components
  17. Glossary
  18. --------
  19. Terms as used in this module.
  20. - Service class:
  21. A type of search engine, e.g. using the database, Apache Solr,
  22. Sphinx or any other professional or simple indexing mechanism. Takes care of
  23. the details of all operations, especially indexing or searching content.
  24. - Server:
  25. One specific place for indexing data, using a set service class. Can
  26. e.g. be some tables in a database, a connection to a Solr server or other
  27. external services, etc.
  28. - Index:
  29. A configuration object for indexing data of a specific type. What and how data
  30. is indexed is determined by its settings. Also keeps track of which items
  31. still need to be indexed (or re-indexed, if they were updated). Needs to lie
  32. on a server in order to be really used (although configuration is independent
  33. of a server).
  34. - Item type:
  35. A type of data which can be indexed (i.e., for which indexes can be created).
  36. Most entity types (like Content, User, Taxonomy term, etc.) are available, but
  37. possibly also other types provided by contrib modules.
  38. - Entity:
  39. One object of data, usually stored in the database. Might for example
  40. be a node, a user or a file.
  41. - Field:
  42. A defined property of an entity, like a node's title or a user's mail address.
  43. All fields have defined datatypes. However, for indexing purposes the user
  44. might choose to index a property under a different data type than defined.
  45. - Data type:
  46. Determines how a field is indexed. While "Fulltext" fields can be completely
  47. searched for keywords, other fields can only be used for filtering. They will
  48. also be converted to fit their respective value ranges.
  49. How types other than "Fulltext" are handled depends on the service class used.
  50. Its documentation should state how the type-selection affect the indexed
  51. content. However, service classes will always be able to handle all data
  52. types, it is just possible that the type doesn't affect the indexing at all
  53. (apart from "Fulltext vs. the rest").
  54. - Boost:
  55. Number determining how important a certain field is, when searching for
  56. fulltext keywords. The higher the value is, the more important is the field.
  57. E.g., when the node title has a boost of 5.0 and the node body a boost of 1.0,
  58. keywords found in the title will increase the score as much as five keywords
  59. found in the body. Of course, this has only an effect when the score is used
  60. (for sorting or other purposes). It has no effect on other parts of the search
  61. result.
  62. - Data alteration:
  63. A component that is used when indexing data. It can add additional fields to
  64. the indexed entity or prevent certain entities from being indexed. Fields
  65. added by callbacks have to be enabled on the "Fields" page to be of any use,
  66. but this is done by default.
  67. - Processor:
  68. An object that is used for preprocessing indexed data as well as search
  69. queries, and for postprocessing search results. Usually only work on fulltext
  70. fields to control how content is indexed and searched. E.g., processors can be
  71. used to make searches case-insensitive, to filter markup out of indexed
  72. content, etc.
  73. Information for users
  74. ---------------------
  75. IMPORTANT: Access checks
  76. In general, the Search API doesn't contain any access checks for search
  77. results. It is your responsibility to ensure that only accessible search
  78. results are displayed – either by only indexing such items, or by filtering
  79. appropriately at search time.
  80. For search on general site content (item type "Node"), this is already
  81. supported by the Search API. To enable this, go to the index's "Workflow" tab
  82. and activate the "Node access" data alteration. This will add the necessary
  83. field, "Node access information", to the index (which you have to leave as
  84. "indexed"). If both this field and "Published" are set to be indexed, access
  85. checks will automatically be executed at search time, showing only those
  86. results that a user can view. Some search types (e.g., search views) also
  87. provide the option to disable these access checks for individual searches.
  88. Please note, however, that these access checks use the indexed data, while
  89. usually the current data is displayed to users. Therefore, users might still
  90. see inappropriate content as long as items aren't indexed in their latest
  91. state. If you can't allow this for your site, please use the index's "Index
  92. immediately" feature (explained below) or possibly custom solutions for
  93. specific search types, if available.
  94. As stated above, you will need at least one other module to use the Search API,
  95. namely one that defines a service class (e.g. search_api_db ("Database search"),
  96. provided with this module).
  97. - Creating a server
  98. (Configuration > Search API > Add server)
  99. The most basic thing you have to create is a search server for indexing content.
  100. Go to Configuration > Search API in the administration pages and select
  101. "Add server". Name and description are usually only shown to administrators and
  102. can be used to differentiate between several servers, or to explain a server's
  103. use to other administrators (for larger sites). Disabling a server makes it
  104. unusable for indexing and searching and can e.g. be used if the underlying
  105. search engine is temporarily unavailable.
  106. The "service class" is the most important option here, since it lets you select
  107. which backend the search server will use. This cannot be changed after the
  108. server is created.
  109. Depending on the selected service class, further, service-specific settings will
  110. be available. For details on those settings, consult the respective service's
  111. documentation.
  112. - Creating an index
  113. (Configuration > Search API > Add index)
  114. For adding a search index, choose "Add index" on the Search API administration
  115. page. Name, description and "enabled" status serve the exact same purpose as
  116. for servers.
  117. The most important option in this form is the indexed entity type. Every index
  118. contains data on only a single type of entities, e.g. nodes, users or taxonomy
  119. terms. This is therefore the only option that cannot be changed afterwards.
  120. The server on which the index lies determines where the data will actually be
  121. indexed. It doesn't affect any other settings of the index and can later be
  122. changed with the only drawback being that the index' content will have to be
  123. indexed again. You can also select a server that is at the moment disabled, or
  124. choose to let the index lie on no server at all, for the time being. Note,
  125. however, that you can only create enabled indexes on an enabled server. Also,
  126. disabling a server will disable all indexes that lie on it.
  127. The "Index items immediately" option specifies that you want items to be
  128. directly re-indexed after being changed, instead of waiting for the next cron
  129. run. Use this if it is important that users see no stale data in searches, and
  130. only when your setup enables relatively fast indexing.
  131. Lastly, the "Cron batch size" option allows you to set whether items will be
  132. indexed when cron runs (as long as the index is enabled), and how many items
  133. will be indexed in a single batch. The best value for this setting depends on
  134. how time-consuming indexing is for your setup, which in turn depends mostly on
  135. the server used and the enabled data alterations. You should set it to a number
  136. of items which can easily be indexed in 10 seconds' time. Items can also be
  137. indexed manually, or directly when they are changed, so even if this is set to
  138. 0, the index can still be used.
  139. - Indexed fields
  140. (Configuration > Search API > [Index name] > Fields)
  141. Here you can select which of the entities' fields will be indexed, and how.
  142. Fields added by (enabled) data alterations will be available here, too.
  143. Without selecting fields to index, the index will be useless and also won't be
  144. available for searches. Select the "Fulltext" data type for fields which you
  145. want search for keywords, and other data types when you want to use the field
  146. for filtering (e.g., as facets). The "Item language" field will always be
  147. indexed as it contains important information for processors and hooks.
  148. You can also add fields of related entities here, via the "Add related fields"
  149. form at the bottom of the page. For instance, you might want to index the
  150. author's username to the indexed data of a node, and you need to add the "Body"
  151. entity to the node when you want to index the actual text it contains.
  152. - Index workflow
  153. (Configuration > Search API > [Index name] > Workflow)
  154. This page lets you customize how the created index works, and what metadata will
  155. be available, by selecting data alterations and processors (see the glossary for
  156. further explanations).
  157. Data alterations usually only add one or more fields to the entity and their
  158. order is mostly irrelevant.
  159. The order of processors, however, often is important. Read the processors'
  160. descriptions or consult their documentation for determining how to use them most
  161. effectively.
  162. - Index status
  163. (Configuration > Search API > [Index name] > Status)
  164. On this page you can view how much of the entities are already indexed and also
  165. control indexing. With the "Index now" button (displayed only when there are
  166. still unindexed items) you can directly index a certain number of "dirty" items
  167. (i.e., items not yet indexed in their current state). Setting "-1" as the number
  168. will index all of those items, similar to the cron batch size setting.
  169. When you change settings that could affect indexing, and the index is not
  170. automatically marked for re-indexing, you can do this manually with the
  171. "Re-index content" button. All items in the index will be marked as dirty and be
  172. re-indexed when subsequently indexing items (either manually or via cron runs).
  173. Until all content is re-indexed, the old data will still show up in searches.
  174. This is different with the "Clear index" button. All items will be marked as
  175. dirty and additionally all data will be removed from the index. Therefore,
  176. searches won't show any results until items are re-indexed, after clearing an
  177. index. Use this only if completely wrong data has been indexed. It is also done
  178. automatically when the index scheme or server settings change too drastically to
  179. keep on using the old data.
  180. - Hidden settings
  181. search_api_index_worker_callback_runtime:
  182. By changing this variable, you can determine the time (in seconds) the Search
  183. API will spend indexing (for all indexes combined) in each cron run. The
  184. default is 15 seconds.
  185. Information for developers
  186. --------------------------
  187. | NOTE:
  188. | For modules providing new entities: In order for your entities to become
  189. | searchable with the Search API, your module will need to implement
  190. | hook_entity_property_info() in addition to the normal hook_entity_info().
  191. | hook_entity_property_info() is documented in the entity module.
  192. | For making certain non-entities searchable, see "Item type" below.
  193. | For custom field types to be available for indexing, provide a
  194. | "property_type" key in hook_field_info(), and optionally a callback at the
  195. | "property_callbacks" key.
  196. | Both processes are explained in [1].
  197. |
  198. | [1] http://drupal.org/node/1021466
  199. Apart from improving the module itself, developers can extend search
  200. capabilities provided by the Search API by providing implementations for one (or
  201. several) of the following classes. Detailed documentation on the methods that
  202. need to be implemented are always available as doc comments in the respective
  203. interface definition (all found in their respective files in the includes/
  204. directory). The details for hooks can be looked up in the search_api.api.php
  205. file.
  206. For all interfaces there are handy base classes which can (but don't need to) be
  207. used to ease custom implementations, since they provide sensible generic
  208. implementations for many methods. They, too, should be documented well enough
  209. with doc comments for a developer to find the right methods to override or
  210. implement.
  211. - Service class
  212. Interface: SearchApiServiceInterface
  213. Base class: SearchApiAbstractService
  214. Hook: hook_search_api_service_info()
  215. The service classes are the heart of the API, since they allow data to be
  216. indexed on different search servers. Since these are quite some work to get
  217. right, you should probably make sure a service class for a specific search
  218. engine doesn't exist already before programming it yourself.
  219. When your module supplies a service class, please make sure to provide
  220. documentation (at least a README.txt) that clearly states the datatypes it
  221. supports (and in what manner), how a direct query (a query where the keys are
  222. a single string, instead of an array) is parsed and possible limitations of the
  223. service class.
  224. The central methods here are the indexItems() and the search() methods, which
  225. always have to be overridden manually. The configurationForm() method allows
  226. services to provide custom settings for the user.
  227. See the SearchApiDbService class for an example implementation.
  228. - Query class
  229. Interface: SearchApiQueryInterface
  230. Base class: SearchApiQuery
  231. You can also override the query class' behaviour for your service class. You
  232. can, for example, change key parsing behaviour, add additional parse modes
  233. specific to your service, or override methods so the information is stored more
  234. suitable for your service.
  235. For the query class to become available (other than through manual creation),
  236. you need a custom service class where you override the query() method to return
  237. an instance of your query class.
  238. - Item type
  239. Interface: SearchApiDataSourceControllerInterface
  240. Base class: SearchApiAbstractDataSourceController
  241. Hook: hook_search_api_item_type_info()
  242. If you want to index some data which is not defined as an entity, you can
  243. specify it as a new item type here. For defining a new item type, you have to
  244. create a data source controller for the type and track new, changed and deleted
  245. items of the type by calling the search_api_track_item_*() functions.
  246. An instance of the data source controller class will then be used by indexes
  247. when handling items of your newly-defined type.
  248. If you want to make external data that is indexed on some search server
  249. available to the Search API, there is a handy base class for your data source
  250. controller (SearchApiExternalDataSourceController in
  251. includes/datasource_external.inc) which you can extend. For a minimal use case,
  252. you will then only have to define the available fields that can be retrieved by
  253. the server.
  254. - Data type
  255. Hook: hook_search_api_data_type_info()
  256. You can specify new data types for indexing fields. These new types can then be
  257. selected on indexes' „Fields“ tabs. You just have to implement the hook,
  258. returning some information on your data type, and specify in your module's
  259. documentation the format of your data type and how it should be used.
  260. For a custom data type to have an effect, in most cases the server's service
  261. class has to support that data type. A service class can advertize its support
  262. of a data type by declaring support for the "search_api_data_type_TYPE" feature
  263. in its supportsFeature() method. If this support isn't declared, a fallback data
  264. type is automatically used instead of the custom one.
  265. If a field is indexed with a custom data type, its entry in the index's options
  266. array will have the selected type in "real_type", while "type" contains the
  267. fallback type (which is always one of the default data types, as returned by
  268. search_api_default_field_types().
  269. - Data-alter callbacks
  270. Interface: SearchApiAlterCallbackInterface
  271. Base class: SearchApiAbstractAlterCallback
  272. Hook: hook_search_api_alter_callback_info()
  273. Data alter callbacks can be used to change the field data of indexed items, or
  274. to prevent certain items from being indexed. They are only used when indexing,
  275. or when selecting the fields to index. For adding additional information to
  276. search results, you have to use a processor.
  277. Data-alter callbacks are called "data alterations" in the UI.
  278. - Processors
  279. Interface: SearchApiProcessorInterface
  280. Base class: SearchApiAbstractProcessor
  281. Hook: hook_search_api_processor_info()
  282. Processors are used for altering the data when indexing or searching. The exact
  283. specifications are available in the interface's doc comments. Just note that the
  284. processor description should clearly state assumptions or restrictions on input
  285. types (e.g. only tokenized text), item language, etc. and explain concisely what
  286. effect it will have on searches.
  287. See the processors in includes/processor.inc for examples.
  288. Included components
  289. -------------------
  290. - Service classes
  291. * Database search
  292. A search server implementation that uses the normal database for indexing
  293. data. It isn't very fast and the results might also be less accurate than
  294. with third-party solutions like Solr, but it's very easy to set up and good
  295. for smaller applications or testing.
  296. See contrib/search_api_db/README.txt for details.
  297. - Data alterations
  298. * URL field
  299. Provides a field with the URL for displaying the entity.
  300. * Aggregated fields
  301. Offers the ability to add additional fields to the entity, containing the
  302. data from one or more other fields. Use this, e.g., to have a single field
  303. containing all data that should be searchable, or to make the text from a
  304. string field, like a taxonomy term, also fulltext-searchable.
  305. The type of aggregation can be selected from a set of values: you can, e.g.,
  306. collect the text data of all contained fields, or add them up, count their
  307. values, etc.
  308. * Bundle filter
  309. Enables the admin to prevent entities from being indexed based on their
  310. bundle (content type for nodes, vocabulary for taxonomy terms, etc.).
  311. * Complete entity view
  312. Adds a field containing the whole HTML content of the entity as it is viewed
  313. on the site. The view mode used can be selected.
  314. Note, however, that this might not work for entities of all types. All core
  315. entities except files are supported, though.
  316. * Index hierarchy
  317. Allows to index a hierarchical field along with all its parents. Most
  318. importantly, this can be used to index taxonomy term references along with
  319. all parent terms. This way, when an item, e.g., has the term "New York", it
  320. will also be matched when filtering for "USA" or "North America".
  321. - Processors
  322. * Ignore case
  323. Makes all fulltext searches (and, optionally, also filters on string values)
  324. case-insensitive. Some servers might do this automatically, for others this
  325. should probably always be activated.
  326. * HTML filter
  327. Strips HTML tags from fulltext fields and decodes HTML entities. If you are
  328. indexing HTML content (like node bodies) and the search server doesn't
  329. handle HTML on its own, this should be activated to avoid indexing HTML
  330. tags, as well as to give e.g. terms appearing in a heading a higher boost.
  331. * Tokenizer
  332. This processor allows you to specify how indexed fulltext content is split
  333. into seperate tokens – which characters are ignored and which treated as
  334. white-space that seperates words.
  335. * Stopwords
  336. Enables the admin to specify a stopwords file, the words contained in which
  337. will be filtered out of the text data indexed. This can be used to exclude
  338. too common words from indexing, for servers not supporting this natively.
  339. - Additional modules
  340. * Search pages
  341. This module lets you create simple search pages for indexes.
  342. * Search views
  343. This integrates the Search API with the Views module [1], enabling the user
  344. to create views which display search results from any Search API index.
  345. * Search facets
  346. For service classes supporting this feature (e.g. Solr search), this module
  347. automatically provides configurable facet blocks on pages that execute
  348. a search query.
  349. [1] http://drupal.org/project/views