bachir/performance-art.fr
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

all contrib module security updates done

Browse Source
This commit is contained in:
bachir
2020-09-24 22:47:32 +02:00
parent 7d87153100
commit 0fbb9c4610
1612 changed files with 116663 additions and 32269 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sites/all/modules/contrib/migrate/migrate/BACKPORT.txt
-5
View File
@@ -1,5 +0,0 @@
Conventions to make porting changes between Drupal 6 and Drupal 7 easier:
Try to always use specific DBTNG functions such as db_select() instead of the
more general db_query(), which needs to be renamed to dbtng_query() under
Drupal 6.
sites/all/modules/contrib/migrate/migrate/CHANGELOG.txt
+281 -14
View File
@@ -1,3 +1,282 @@
Next release
============
Features and enhancements
- #2552189 - Return migration status from drush migrate-import.
- #2550793 - Option to always use chunk IDs in MigrateContentParser.
- #2505683 - Pass item ID to MigrateContentParser implementations.
- #2532222 - Allow spreadsheet source data on a row different that 1.
- #1406802 - Add callback() field mapping method, to pass additional arguments
to callbacks.
- #2516828 - Add mdreg alias for migrate-deregister.
- #2504517 - Add timezone handling to timestamp().
Bug fixes
- #2612110 - Notice when units omitted on --limit.
- #2597606 - Escape MySQL database names.
- #2577091 - Do strict type-check on XML ids.
- #2537206 - copy() return not checked.
- #2536616 - Improve message when sourceMigration not found.
- #2578391 - Improve message when class not found.
- #2565043 - Handle empty idlist in MigrateSourceMultiItems.
- #2510010 - Stop stripping group name prefixes from migration names.
- #2542520 - Fallback for missing source key description.
- #2499861 - Properly save/disable/restore mail system.
- #2541996 - Prevent NULL file_mask being passed to file_scan_directory().
Migrate 2.8
===========
Features and enhancements
- #2379289 - Better handle interaction of --update with highwater marks.
- #2403643 - Support an additional level of subfields.
- #2472045 - Add language subfields only if field is translatable.
- #2474809 - Provide better message for bad dependencies.
- #2397791 - Provide detailed field validation errors.
- #2309563 - Add support for running migrations via wildcard name.
- #2095841 - Abstract mail system disablement for more flexibility.
- #2419373 - Provide ability to cache map lookups.
- #2141687 - Provide detailed message on file copy error.
Bug fixes
Field sanitization added to prevent possibility of XSS - see security advisory
https://security.drupal.org/node/155268.
- #2447115 - Add xpath handling to the field mapping editor.
- #2497015 - Term reference handler would ignore all terms if one was NULL.
- #2488560 - MigrateSourceList/MigrateSourceMultiItems iterators prematurely
return.
- #2446105 - Keep coded DNM source field mappings from overriding UI mappings.
- #2415977 - Use temporary:// instead of /tmp for drush logging.
- #2475473 - Fix handling of --idlist when map not joined.
- #2465387 - Fix handling of --stop option on migrate-import.
Migrate 2.7
===========
Features and enhancements
- #2296911 - Add a source handler for IBM DB2.
- #2256761 - Add a destination handler for variables.
- #2047815 - Support multi-column source keys in idlist.
- #1751438 - Add spreadsheet source plugin.
Bug fixes
- #2415597 - Make batching of SQL sources optional, and force map_joinable FALSE.
- #2403593 - SQL batching messes up cases with altered queries, such as idlist.
- #2298969 - Verify wizard validation function exists.
- #2268863 - Fix drush --all option.
- #2410523 - Remove inconsistent escaping of migrate_drush_path.
Migrate 2.6
===========
IMPORTANT CHANGES SINCE MIGRATE 2.5
-----------------------------------
Migration developers will need to add the "advanced migration information"
permission to their roles to continue seeing all the info in the UI they're
used to.
Auto-registration (having classes be registered just based on their class name,
with no call to registerMigration or definition in hook_migrate_api()) is no
longer supported. Registration of classes defined in hook_migrate_api() is no
longer automatic - do a drush migrate-register or use the Register button in the
UI to register them.
Migration class constructors should now always accept an $arguments array as
the first parameter and pass it to its parent. This version does support legacy
migrations which pass a group object, or nothing, but these methods are
deprecated.
Features and enhancements
- #2390229 - Make literal strings monolithic.
- #2391789 - Add extender getter to better support extending wizards.
- #2363015 - Add support for modifying wizards defined by other modules.
- #2353527 - Add getter/setter for trackLastImported.
- #2302929 - Explicitly count IDs for JSON source counts.
- #2296187 - Batch MigrateSQLSource queries.
- #2260211 - Allow skipping of file contents in MigrateListFiles.
- #2259089 - Display actual query for SQL sources.
- #2224297 - Destination handler for custom blocks.
- #2370677 - Add removeStep() method to Wizard API.
- #2306953 - Give basic users a little more information.
- #2261357 - Add prepareCallback/completeCallback to table destination.
- #2306923 - Propagate message statuses to drupal_set_message in batch.
- #2301679 - Display all source key fields in the message view.
- #2312075 - Add --ignore-highwater option to drush imports
- #1961316 - Get data from all XML namespaces
- #1298724 - Add a node revision destination plugin.
- #1890610 - Add a MongoDB source plugin
- #2065295 - Add ability to disable other module's hooks during migration.
- #2051547 - Add ability to revert UI-defined field mappings.
- #1998632 - Extend MigrateItemsXML to handle an array of XML files.
- #1990612 - Add a row status argument to MigrateException, allowing rows to
be cleanly skipped by throwing exceptions.
- #2017835 - Add MigrateFileUriAsIs file class, and make file migrations more
flexible.
- #2004426 - UI support for editing dependencies; enable setting dependencies
through arguments.
- #1826112 - A new API has been added to support external modules in developing
easy-to-use wizard-based UIs.
- #1833380 - Major refactoring of the UI, breaking groups and migrations
(tasks) into separate pages, introducing an advanced permission and
presenting a simplified UI to those with only the basic permission.
- #1860450 - The UI now has the capability of spawning import/rollback operations
in drush, with email notifications of completion. The notification
ability is available when running drush at the command line, which
may be useful for running imports via cron.
- #1996602 - A default field handler is now provided that should handle many if
not most field types without custom handlers.
- #1901980 - Support encrypting particular arguments saved to the database, to
support wizard implementations that may be prompting for secure
database credentials.
- #1896096 - Add ability to define field mappings via arguments at registration
time, and use those to override mappings in code.
- #1961620 - When editing field mappings, allow destination or source fields to
be explicitly set DNM.
- #1996280 - Allow all field/subfields/options to be edited, and indent
subfields/options to make the relationship more clear.
- #1996350 - Allow sourceMigration to be edited.
- #1996736 - Add support for hook_migrate_api_alter().
- #1984568 - Add setters to enable constructing migrations according to a
dependency injection pattern.
- #1835822 - Hash source rows to detect changes.
- #1975180 - Explicitly check dependency existence so missing dependencies aren't
reported as circular.
- #1984534 - Updating/cleanup of examples to reflect current best practices.
- #1972600 - Add getter for the MigrateXMLReader member of MigrateSourceXML.
- #1856694 - Allow disabling of uri encoding.
- #1835142 - Add names-only option to drush ms command.
- #1892296 - Increase time limit for batch UI migrations.
- #1839644 - Groups have been extended to record titles and arguments in the db.
- #1896920 - Remove auto-registration, and make static registration expicit.
- #1928956 - Clean up registration/instantiation chicken-and-egg problems, by
supporting consistent constructor arguments.
- #1550878 - Added UI for deregistering migrations, including one-button
deregistration of orphans.
- #1792894 - Remove empty field values, permitting proper defaulting.
- #1903236 - Allow reset of migrate_migrations() cache.
- #1894344 - Support warn_on_override for bulk unmigrated methods.
- #1886404 - Allow subfields to be mapped before primary fields.
Bug fixes
- #2391891 - idlist with XML source causes exception.
- #2314077 - Don't merge group dependencies into migration dependencies.
- #2392683 - Set vocabulary name (bundle) explicitly for term reference fields.
- #2266395 - Check for valid picture file saving users.
- #2225551 - Add field validation to entity destination plugins.
- #2285966 - Added exception handling around constructors.
- #2370877 - Fix reloading of wizard steps after first.
- #2358767 - Topological sort to help avoid false circular dependency errors.
- #2019193 - Properly handle destination DNM status on field mapping save.
- #2250081 - Fix PostgreSQL failure updating node statistics on comments.
- #2157933 - Fix PostgreSQL error with empty source migrations.
- #2177313 - Make sure reused file entities are saved to pick up field changes.
- #2352971 - Treat empty default values as NULL in field mapping editor.
- #2347205 - Explicitly default the language for terms.
- #2307599 - Notice for non-advanced users on rollback.
- #2305105 - Remove pedantically-deprecated hyphens.
- #2325237 - Handle language arrays for file fields.
- #2258909 - Use migrated languages for file fields.
- #2323605 - Handle single-parent terms properly.
- #2313495 - Handle multilingual path aliases.
- #2261227 - Fixed missing subfields on edit mapping form.
- #2154385 - Add file/line context to source plugin exceptions.
- #2190255 - Remove duplicate warnings on missing XML source.
- #2129609 - Remove warnings on MS SQL counts.
- #2109931 - Pass drush global options to subprocess.
- #2236279 - Prevent ignore_case from causing created terms to be lower-cased.
- #2104149 - Sanity-check decrypted arguments.
- #2047523 - Consistently document destination handler fields() implementations.
- #2244759 - Rollback migrations in proper (reverse) order in UI.
- #2184641 - Make saveHighwater() work with PostgreSQL.
- #2076035 - Properly check key values in saveMessage().
- #2076035 - Validate that term names are not empty.
- #2225809 - Use proper API for registering wizard classes.
- #2105037 - Expose nid/uid destinations when is_new is on.
- #2145213 - Add SQLMap constructor documentation.
- #2213033 - Add batch callback documentation.
- #2095829 - Remove all variables on uninstall.
- #2072721 - Proper title for migrate_example_baseball feature.
- #1999228 - Removed AJAX from field mapping form for performance.
- #2237755 - Fix activeUrl handling in JSON source.
- #2237891 - Use drupal_register_shutdown_function().
- #2141409 - Handle drush spawning when Drupal in subdirectory.
- #2191335 - Remove unused columns from CSV source row.
- #2210533 - Properly pass directory by reference.
- #2141569 - Explicitly create group in baseball example.
- #2159291 - Add message for missing argument to drush migrate-messages.
- #2019193 - Fix unsetting of DNM source fields through the UI.
- #2227061 - Fix to subfields in a multi-value context.
- #2170177 - Properly handle multi-value fields when a subfield is mapped first.
- #2120205 - Fix bug when upgrading lastimported from D6 to D7.
- #2109821 - Handle multi-value subfields in the default field handler.
- #2039649 - Smarter setting of file 'type' field.
- #2106925 - Fix machine_name generation for legacy migrations.
- #2102087 - Don't pass --group to drush subprocess.
- #2099585 - Enable group option on drush migrate-stop.
- #2088255 - Eliminate notices on migrate_map_hash with track_changes.
- #2014849 - Make sure (again) statistics properly default to 0.
- #2042535 - Properly set file_usage for user pictures.
- #2060631 - Fix notices on poll vote import.
- #2049689 - Empty values for node is_new caused SQL errors.
- #2042399 - Node import did not respect revision flag.
- #2034885 - For XML sources, don't override values populated in prepareRow().
- #2041267 - Handle lack of potential dependencies on edit page.
- #2040101 - Breadcrumbs in migration UI missing sections
- #2033947 - Empty migration display names when matching group names.
- #2037265 - Static migrations were not registered on module enable.
- #2021457 - Coding standard improvements for xml.inc.
- #2030559 - Add deprecation messages for arguments().
- #1854382 - Fix handling of terms with leading spaces.
- #2025137 - Ignore --update in the presence of --idlist.
- #2023813 - Apply defaultValue() for empty XML values.
- #2023657 - Prevent duplicate aliases when updating nodes.
- #2021973 - Drush deregistration needs to handle mixed-case machine names.
- #2017443 - Properly hash XML source rows with track_changes on.
- #2020095 - Preserve arguments in legacy constructors.
- #2018841 - Default field handler needs to account for empty column
descriptions.
- #2016173 - Fix fatal error editing MigrationBase migrations.
- #2015327 - Remove broken automatic field mapping feature.
- #2014849 - Make sure statistics properly default to 0.
- #2011024 - Wildcard groups/tasks in the menu hierarchy, saving the need to
rebuild menus when migrations are registered/deregistered.
- #2010884 - Clean up empty default groups on dbupdate.
- #2004296 - Enforce map/message table names with prefixes.
- #2009222 - Properly update map/message tables on a non-default connection.
- #2006158 - Fix warnings on default field handler fields().
- #1999918 - Move permissions to core Migrate module.
- #1999290 - Check that an XMLMigration has actually populated $row->xml.
- #1989012 - Support preserve_files for MigrateFileFid.
- #1982564 - Properly handle messages for ignored rows.
- #1989022 - Strip HTML tags from drush fields output.
- #1988008 - Restore group dependency support.
- #1985750 - Add missing prepareRollback/completeRollback for file destinations.
- #1978702 - Check highwater marks against the starting highwater.
- #1974216 - Change field mapping table PK to a simple serial column.
- #1977578 - Bad exception handling in MigrateDestinationTableCopy.
- #1973030 - Handle zero-valued timestamps properly.
- #1973092 - Fix preservation of sourceMigration when editing mappings.
- #1968358 - Prevent unnecessary reimport when using track_changes.
- #1968014 - Fix menu notices on task list page.
- #1967946 - Missing check on existence of mapping.
- #1849350 - Make sure encoding of uris doesn't break query strings.
- #1844316 - Ensure rollback_action is added to all map tables.
- #1913462 - Fix update functions returning arrays.
- #1831940 - Allow empty string source keys in handleSourceMigration().
- #1896042 - Fix incorrect usage of originalQuery.
- #1954936 - Allow overriding source count in isComplete.
- #1952430 - Prevent bogus "no error provided" messages.
- #1931168 - Properly cache class info in _migrate_class_list().
- #1900914 - Disable email with no custom mail system configured.
- #1901648 - Missing bundle property on file migrations.
- #1885362 - Make access callback explicit, to avoid conflict with admin_views.
- #1880512 - Strip tags from descriptions on drush migrate-mappings.
- #1872446 - Properly handle updates on role migration.
- #1871764 - Pass zero values through handleSourceMigration.
- #1839534 - Handle missing chunk separator in MigrateItemFile.
- #1854382 - Prevent duplicate terms due to leading/trailing spaces.
- #1794236 - Namespace detail pages within menu system.
- #1836426 - Proper check on activeUrl for multiple XML files.
Migrate 2.5
===========
@@ -10,12 +289,6 @@ If your migrations are not explicitly registered, you must request auto-registra
with a "drush mar" (drush migrate-auto-register) command, or by clicking the
"Register" button at admin/content/migrate/registration.
Bug fixes
- #1827052 - Properly check for bad XML.
Migrate 2.5 Release Candidate 2
===============================
Features and enhancements
- #1824024 - Destination and field handlers may now be registered through
hook_migrate_api(). Automatic registration of all migration and
@@ -26,14 +299,6 @@ Features and enhancements
hook_migrate_api().
- #1819730 - Make migration of files in a field context non-fatal.
- #1816652 - Provide useful warning when file subfield arrays don't line up.
Bug fixes
- #1824118 - Make --force work for rollbacks.
Migrate 2.5 Release Candidate 1
===============================
Features and enhancements
- #1778952 - Enable registration of dynamic migrations via hook_migrate_api().
Add explicit auto-registration of non-dynamic migrations, remove
performing registration on cache clear.
@@ -54,6 +319,8 @@ Features and enhancements
- #1621906 - Support DESTINATION system-of-records for menu links.
Bug fixes
- #1827052 - Properly check for bad XML.
- #1824118 - Make --force work for rollbacks.
- #1659150 - Change 'ok' message types to 'status'.
- #1690080 - Deal with self-references in handleSourceMigration().
- #1797644 - Remove bogus assignment on term update.
sites/all/modules/contrib/migrate/migrate/README.txt
+14 -12
View File
@@ -1,15 +1,17 @@
The Migrate module provides a flexible framework for migrating content into Drupal
from other sources (e.g., when converting a web site from another CMS to Drupal).
The Migrate module provides a flexible framework for migrating content into
Drupal from other sources
(e.g., when converting a web site from another CMS to Drupal).
Out-of-the-box, support for creating Drupal nodes, taxonomy terms, comments, and
users are included. Plugins permit migration of other types of content.
Usage
-----
Documentation is at http://drupal.org/node/415260. To get started, enable the
migrate_example module and browse to admin/content/migrate to see its dashboard.
The code for this migration is in migrate_example/beer.inc (advanced examples are
in wine.inc). Mimic that file in order to specify your own migrations.
Documentation is at http://drupal.org/migrate. To get started, enable the
migrate_example module, enable migrate_ui, and then browse to
admin/content/migrate to see its dashboard.
The code for this migration is in migrate_example/beer.inc
(advanced examples are in wine.inc). Mimic that file in order to specify your
own migrations.
The Migrate module itself has support for migration into core objects. Support
for migration involving contrib modules is in the migrate_extras module.
@@ -17,8 +19,8 @@ for migration involving contrib modules is in the migrate_extras module.
Known issues
------------
A user migration with systemOfRecord == DESTINATION will drop pictures from user
records due to core bug http://drupal.org/node/935592 - the simpletests report an
error reflecting this. We have not developed a work-around.
records due to core bug http://drupal.org/node/935592 - the simpletests report
an error reflecting this. We have not developed a work-around.
Upgrading
---------
@@ -30,9 +32,9 @@ projects.
Acknowledgements
----------------
Much of the Migrate module functionality was sponsored by Cyrve, for its clients GenomeWeb
(http://www.genomeweb.com), The Economist (http://www.economist.com), and Examiner.com
(http://www.examiner.com).
Much of the Migrate module functionality was sponsored by Cyrve, for its clients
GenomeWeb (http://www.genomeweb.com), The Economist (http://www.economist.com),
and Examiner.com (http://www.examiner.com).
Authors
-------
sites/all/modules/contrib/migrate/migrate/includes/base.inc
+557 -147
View File
File diff suppressed because it is too large Load Diff
sites/all/modules/contrib/migrate/migrate/includes/destination.inc
+31 -12
View File
@@ -13,11 +13,12 @@
* MigrateDestinationEntity for an example.
*/
abstract class MigrateDestination {
/**
* To support MigrateSQLMap maps, derived destination classes should return
* schema field definition(s) corresponding to the primary key of the destination
* being implemented. These are used to construct the destination key fields
* of the map table for a migration using this destination.
* schema field definition(s) corresponding to the primary key of the
* destination being implemented. These are used to construct the destination
* key fields of the map table for a migration using this destination.
*
* abstract static public function getKeySchema()
*/
@@ -36,6 +37,7 @@ abstract class MigrateDestination {
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -43,9 +45,9 @@ abstract class MigrateDestination {
abstract public function fields();
/**
* Derived classes must implement either bulkRollback or rollback() according to
* the signatures below, to rollback (usually by deletion) previously-migrated
* items.
* Derived classes must implement either bulkRollback or rollback() according
* to the signatures below, to rollback (usually by deletion)
* previously-migrated items.
*
* $ids is an array of single-field keys to be deleted
* abstract public function bulkRollback(array $ids);
@@ -55,9 +57,10 @@ abstract class MigrateDestination {
*/
/**
* Derived classes must implement import(), to construct one new object (pre-pppulated
* using field mappings in the Migration). It is expected to call prepare and
* complete handlers, passing them $row (the raw data from the source).
* Derived classes must implement import(), to construct one new object
* (pre-pppulated using field mappings in the Migration). It is expected to
* call prepare and complete handlers, passing them $row (the raw data from
* the source).
*/
abstract public function import(stdClass $object, stdClass $row);
@@ -78,10 +81,13 @@ abstract class MigrateDestination {
* @var int
*/
protected $numCreated = 0;
public function getCreated() {
return $this->numCreated;
}
protected $numUpdated = 0;
public function getUpdated() {
return $this->numUpdated;
}
@@ -105,11 +111,24 @@ abstract class MigrateDestination {
* All destination handlers should be derived from MigrateDestinationHandler
*/
abstract class MigrateDestinationHandler extends MigrateHandler {
// abstract function arguments(...)
// Any one or more of these methods may be implemented
/**
* Any one or more of these methods may be implemented
* Documentation of any fields added to the destination by this handler.
*
* @param $entity_type
* The entity type (node, user, etc.) for which to list fields.
* @param $bundle
* The bundle (article, blog, etc.), if any, for which to list fields.
* @param Migration $migration
* Optionally, the migration providing the context.
*
* @return array
* An array keyed by field name, with field descriptions as values.
*/
//abstract public function fields();
//abstract public function fields($entity_type, $bundle, $migration = NULL);
//abstract public function prepare($entity, stdClass $row);
//abstract public function complete($entity, stdClass $row);
}
sites/all/modules/contrib/migrate/migrate/includes/exception.inc
+22 -1
View File
@@ -6,13 +6,34 @@
*/
class MigrateException extends Exception {
/**
* The level of the error being reported (a Migration::MESSAGE_* constant)
*
* @var int
*/
protected $level;
public function getLevel() {
return $this->level;
}
public function __construct($message, $level = Migration::MESSAGE_ERROR) {
/**
* The status to record in the map table for the current item (a
* MigrateMap::STATUS_* constant)
*
* @var int
*/
protected $status;
public function getStatus() {
return $this->status;
}
public function __construct($message, $level = Migration::MESSAGE_ERROR,
$status = MigrateMap::STATUS_FAILED) {
$this->level = $level;
$this->status = $status;
parent::__construct($message);
}
}
sites/all/modules/contrib/migrate/migrate/includes/field_mapping.inc
+51 -4
View File
@@ -7,6 +7,7 @@
*/
class MigrateFieldMapping {
/**
* Destination field name for the mapping. If empty, the mapping is just a
* stub for annotating the source field.
@@ -14,6 +15,7 @@ class MigrateFieldMapping {
* @var string
*/
protected $destinationField;
public function getDestinationField() {
return $this->destinationField;
}
@@ -25,10 +27,28 @@ class MigrateFieldMapping {
* @var string
*/
protected $sourceField;
public function getSourceField() {
return $this->sourceField;
}
/**
* @var int
*/
const MAPPING_SOURCE_CODE = 1;
const MAPPING_SOURCE_DB = 2;
protected $mappingSource = self::MAPPING_SOURCE_CODE;
public function getMappingSource() {
return $this->mappingSource;
}
public function setMappingSource($mapping_source) {
$this->mappingSource = $mapping_source;
}
/**
* Default value for simple mappings, when there is no source mapping or the
* source field is empty. If both this and the sourceField are omitted, the
@@ -37,6 +57,7 @@ class MigrateFieldMapping {
* @var mixed
*/
protected $defaultValue;
public function getDefaultValue() {
return $this->defaultValue;
}
@@ -48,6 +69,7 @@ class MigrateFieldMapping {
* @var string
*/
protected $separator;
public function getSeparator() {
return $this->separator;
}
@@ -61,6 +83,7 @@ class MigrateFieldMapping {
* An array of source migrations, or string for a single migration.
*/
protected $sourceMigration;
public function getSourceMigration() {
return $this->sourceMigration;
}
@@ -71,6 +94,7 @@ class MigrateFieldMapping {
* @var string
*/
protected $callbacks = array();
public function getCallbacks() {
return $this->callbacks;
}
@@ -86,6 +110,7 @@ class MigrateFieldMapping {
* @var string
*/
protected $dedupe;
public function getDedupe() {
return $this->dedupe;
}
@@ -93,40 +118,49 @@ class MigrateFieldMapping {
/**
* Argument overrides. If present this will be an array, keyed by
* a field API array key, with one or both of these entries:
* 'source_field' - Name of the source field in the incoming row containing the
* value to be assigned
* 'default_value' - A constant value to be assigned in the absence of source_field
* 'source_field' - Name of the source field in the incoming row containing
* the value to be assigned
* 'default_value' - A constant value to be assigned in the absence of
* source_field Deprecated - subfield notation is now preferred.
*
* @var array
*/
protected $arguments;
public function getArguments() {
return $this->arguments;
}
protected $description = '';
public function getDescription() {
return $this->description;
}
protected $issueGroup;
public function getIssueGroup() {
return $this->issueGroup;
}
protected $issueNumber;
public function getIssueNumber() {
return $this->issueNumber;
}
protected $issuePriority = self::ISSUE_PRIORITY_OK;
public function getIssuePriority() {
return $this->issuePriority;
}
const ISSUE_PRIORITY_OK = 1;
const ISSUE_PRIORITY_LOW = 2;
const ISSUE_PRIORITY_MEDIUM = 3;
const ISSUE_PRIORITY_BLOCKER = 4;
public static $priorities = array();
@@ -163,7 +197,17 @@ class MigrateFieldMapping {
}
public function callbacks($callbacks) {
$this->callbacks = func_get_args();
foreach (func_get_args() as $callback) {
$this->callback($callback);
}
return $this;
}
public function callback($callback) {
$this->callbacks[] = array(
'callback' => $callback,
'params' => array_slice(func_get_args(), 1),
);
return $this;
}
@@ -173,6 +217,9 @@ class MigrateFieldMapping {
}
public function arguments($arguments) {
if (variable_get('migrate_deprecation_warnings', 1)) {
MigrationBase::displayMessage(t('The field mapping arguments() method is now deprecated - please use subfield notation instead.'));
}
$this->arguments = $arguments;
return $this;
}
sites/all/modules/contrib/migrate/migrate/includes/group.inc
+125 -26
View File
@@ -6,22 +6,48 @@
*/
class MigrateGroup {
/**
* The name of the group - used to identify it in drush commands.
* The machine name of the group - used to identify it in drush commands.
*
* @var string
*/
protected $name;
public function getName() {
return $this->name;
}
/**
* The user-visible title of the group.
*
* @var string
*/
protected $title;
public function getTitle() {
return $this->title;
}
/**
* Domain-specific arguments for the group (to be applied to all migrations
* in the group).
*
* @var array
*/
protected $arguments = array();
public function getArguments() {
return $this->arguments;
}
/**
* List of groups this group is dependent on.
*
* @var array
*/
protected $dependencies = array();
public function getDependencies() {
return $this->dependencies;
}
@@ -32,12 +58,15 @@ class MigrateGroup {
* @var array
*/
static protected $groupList = array();
static public function groups() {
$groups = array();
$dependent_groups = array();
$dependencies_list = array();
$required_groups = array();
foreach (self::$groupList as $name => $group) {
$dependencies = $group->getDependencies();
$dependencies_list[$name] = $dependencies;
if (count($dependencies) > 0) {
// Set groups with dependencies aside for reordering
$dependent_groups[$name] = $group;
@@ -48,29 +77,11 @@ class MigrateGroup {
$groups[$name] = $group;
}
}
$iterations = 0;
while (count($dependent_groups) > 0) {
if ($iterations++ > 20) {
$group_names = implode(',', array_keys($dependent_groups));
throw new MigrateException(t('Failure to sort migration list - most likely due ' .
'to circular dependencies involving groups !group_names',
array('!group_names' => $group_names)));
}
foreach ($dependent_groups as $name => $group) {
$ready = TRUE;
// Scan all the dependencies for this group and make sure they're all
// in the final list
foreach ($group->getDependencies() as $dependency) {
if (!isset($groups[$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this group to the final list
$groups[$name] = $group;
unset($dependent_groups[$name]);
}
$ordered_groups = migrate_order_dependencies($dependencies_list);
foreach ($ordered_groups as $name) {
if (!isset($groups[$name])) {
$groups[$name] = $dependent_groups[$name];
}
}
@@ -86,8 +97,10 @@ class MigrateGroup {
* @param array $dependencies
* List of dependent groups.
*/
public function __construct($name, $dependencies = array()) {
public function __construct($name, $dependencies = array(), $title = '', $arguments = array()) {
$this->name = $name;
$this->title = $title;
$this->arguments = $arguments;
$this->dependencies = $dependencies;
}
@@ -102,8 +115,94 @@ class MigrateGroup {
*/
static public function getInstance($name, $dependencies = array()) {
if (empty(self::$groupList[$name])) {
self::$groupList[$name] = new MigrateGroup($name, $dependencies);
$row = db_select('migrate_group', 'mg')
->fields('mg')
->condition('name', $name)
->execute()
->fetchObject();
if ($row) {
$arguments = unserialize($row->arguments);
$arguments = MigrationBase::decryptArguments($arguments);
if (empty($dependencies) && isset($arguments['dependencies'])) {
$dependencies = $arguments['dependencies'];
}
self::$groupList[$name] = new MigrateGroup($name, $dependencies,
$row->title, $arguments);
}
else {
self::register($name);
self::$groupList[$name] = new MigrateGroup($name, $dependencies);
}
}
return self::$groupList[$name];
}
/**
* Register a new migration group in the migrate_group table.
*
* @param string $name
* The machine name (unique identifier) for the group.
*
* @param string $title
* A user-visible title for the group. Defaults to the machine name.
*
* @param array $arguments
* An array of group arguments - generally data that applies to all
* migrations in the group.
*/
static public function register($name, $title = NULL, array $arguments = array()) {
if (!$title) {
$title = $name;
}
$arguments = MigrationBase::encryptArguments($arguments);
// Register the migration if it's not already there; if it is,
// update the class and arguments in case they've changed.
db_merge('migrate_group')
->key(array('name' => $name))
->fields(array(
'title' => $title,
'arguments' => serialize($arguments),
))
->execute();
}
/**
* Deregister a migration group - remove it from the database, and also
* remove migrations attached to it.
*
* @param string $name
* (Machine) name of the group to deregister.
*/
static public function deregister($name) {
$result = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name'))
->condition('group_name', $name)
->execute();
foreach ($result as $row) {
Migration::deregisterMigration($row->machine_name);
}
db_delete('migrate_group')
->condition('name', $name)
->execute();
}
/**
* Remove any groups which no longer contain any migrations.
*/
static public function deleteOrphans() {
$query = db_select('migrate_group', 'mg');
$query->addField('mg', 'name');
$query->leftJoin('migrate_status', 'ms', 'mg.name=ms.group_name');
$query->isNull('ms.machine_name');
$result = $query->execute();
foreach ($result as $row) {
db_delete('migrate_group')
->condition('name', $row->name)
->execute();
}
}
}
sites/all/modules/contrib/migrate/migrate/includes/handler.inc
+5 -1
View File
@@ -10,12 +10,15 @@
* to implement appropriate methods (e.g., prepare, complete, or fields).
*/
abstract class MigrateHandler {
/**
* List of other handler classes which should be invoked before the current one.
* List of other handler classes which should be invoked before the current
* one.
*
* @var array
*/
protected $dependencies = array();
public function getDependencies() {
return $this->dependencies;
}
@@ -27,6 +30,7 @@ abstract class MigrateHandler {
* @var array
*/
protected $typesHandled = array();
public function getTypesHandled() {
return $this->typesHandled;
}
sites/all/modules/contrib/migrate/migrate/includes/map.inc
+36 -6
View File
@@ -10,12 +10,16 @@
* for the purpose of rollback.
*/
abstract class MigrateMap implements Iterator {
/**
* Codes reflecting the current status of a map row.
*/
const STATUS_IMPORTED = 0;
const STATUS_NEEDS_UPDATE = 1;
const STATUS_IGNORED = 2;
const STATUS_FAILED = 3;
/**
@@ -23,6 +27,7 @@ abstract class MigrateMap implements Iterator {
*
*/
const ROLLBACK_DELETE = 0;
const ROLLBACK_PRESERVE = 1;
/**
@@ -32,7 +37,9 @@ abstract class MigrateMap implements Iterator {
* @var array
*/
protected $sourceKey, $destinationKey;
abstract public function getSourceKey();
abstract public function getDestinationKey();
/**
@@ -43,6 +50,13 @@ abstract class MigrateMap implements Iterator {
*/
protected $sourceKeyMap, $destinationKeyMap;
/**
* Get the source key map.
*/
public function getSourceKeyMap() {
return $this->sourceKeyMap;
}
/**
* Boolean determining whether to track last_imported times in map tables
*
@@ -50,6 +64,16 @@ abstract class MigrateMap implements Iterator {
*/
protected $trackLastImported = FALSE;
public function getTrackLastImported() {
return $this->trackLastImported;
}
public function setTrackLastImported($trackLastImported) {
if (is_bool($trackLastImported)) {
$this->trackLastImported = $trackLastImported;
}
}
/**
* Save a mapping from the key values in the source row to the destination
* keys.
@@ -58,9 +82,11 @@ abstract class MigrateMap implements Iterator {
* @param $dest_ids
* @param $status
* @param $rollback_action
* @param $hash
*/
abstract public function saveIDMapping(stdClass $source_row, array $dest_ids,
$status = MigrateMap::STATUS_IMPORTED, $rollback_action = MigrateMap::ROLLBACK_DELETE);
$status = MigrateMap::STATUS_IMPORTED,
$rollback_action = MigrateMap::ROLLBACK_DELETE, $hash = NULL);
/**
* Record a message related to a source record
@@ -130,6 +156,7 @@ abstract class MigrateMap implements Iterator {
* Retrieve map data for a given source or destination item
*/
abstract public function getRowBySource(array $source_id);
abstract public function getRowByDestination(array $destination_id);
/**
@@ -138,29 +165,32 @@ abstract class MigrateMap implements Iterator {
abstract public function getRowsNeedingUpdate($count);
/**
* Given a (possibly multi-field) destination key, return the (possibly multi-field)
* source key mapped to it.
* Given a (possibly multi-field) destination key, return the (possibly
* multi-field) source key mapped to it.
*
* @param array $destination_id
* Array of destination key values.
*
* @return array
* Array of source key values, or NULL on failure.
*/
abstract public function lookupSourceID(array $destination_id);
/**
* Given a (possibly multi-field) source key, return the (possibly multi-field)
* destination key it is mapped to.
* Given a (possibly multi-field) source key, return the (possibly
* multi-field) destination key it is mapped to.
*
* @param array $source_id
* Array of source key values.
*
* @return array
* Array of destination key values, or NULL on failure.
*/
abstract public function lookupDestinationID(array $source_id);
/**
* Remove any persistent storage used by this map (e.g., map and message tables)
* Remove any persistent storage used by this map (e.g., map and message
* tables)
*/
abstract public function destroy();
}
sites/all/modules/contrib/migrate/migrate/includes/migration.inc
+394 -139
View File
File diff suppressed because it is too large Load Diff
sites/all/modules/contrib/migrate/migrate/includes/source.inc
+140 -30
View File
@@ -13,6 +13,7 @@
* MigrateSourceSQL for an example.
*/
abstract class MigrateSource implements Iterator {
/**
* The current row from the quey
*
@@ -26,6 +27,7 @@ abstract class MigrateSource implements Iterator {
* @var array
*/
protected $currentKey;
public function getCurrentKey() {
return $this->currentKey;
}
@@ -50,6 +52,7 @@ abstract class MigrateSource implements Iterator {
* @var int
*/
protected $numIgnored = 0;
public function getIgnored() {
return $this->numIgnored;
}
@@ -60,6 +63,7 @@ abstract class MigrateSource implements Iterator {
* @var int
*/
protected $numProcessed = 0;
public function getProcessed() {
return $this->numProcessed;
}
@@ -78,6 +82,20 @@ abstract class MigrateSource implements Iterator {
*/
protected $highwaterField;
/**
* The highwater mark at the beginning of the import operation.
*
* @var
*/
protected $originalHighwater = '';
/**
* Used in the case of multiple key sources that need to use idlist.
*
* @var string
*/
protected $multikeySeparator = ':';
/**
* List of source IDs to process.
*
@@ -116,6 +134,14 @@ abstract class MigrateSource implements Iterator {
*/
protected $skipCount = FALSE;
/**
* If TRUE, we will maintain hashed source rows to determine whether incoming
* data has changed.
*
* @var bool
*/
protected $trackChanges = FALSE;
/**
* By default, next() will directly read the map row and add it to the data
* row. A source plugin implementation may do this itself (in particular, the
@@ -138,7 +164,7 @@ abstract class MigrateSource implements Iterator {
}
if (!isset($this->cacheKey)) {
$this->cacheKey = md5((string)$this);
$this->cacheKey = md5((string) $this);
}
// If a refresh is requested, or we're not caching counts, ask the derived
@@ -186,6 +212,9 @@ abstract class MigrateSource implements Iterator {
if (!empty($options['cache_key'])) {
$this->cacheKey = $options['cache_key'];
}
if (!empty($options['track_changes'])) {
$this->trackChanges = $options['track_changes'];
}
}
/**
@@ -202,17 +231,18 @@ abstract class MigrateSource implements Iterator {
}
/**
* Implementation of Iterator::key - called when entering a loop iteration, returning
* the key of the current row. It must be a scalar - we will serialize
* to fulfill the requirement, but using getCurrentKey() is preferable.
* Implementation of Iterator::key - called when entering a loop iteration,
* returning the key of the current row. It must be a scalar - we will
* serialize to fulfill the requirement, but using getCurrentKey() is
* preferable.
*/
public function key() {
return serialize($this->currentKey);
}
/**
* Implementation of Iterator::valid() - called at the top of the loop, returning
* TRUE to process the loop and FALSE to terminate it
* Implementation of Iterator::valid() - called at the top of the loop,
* returning TRUE to process the loop and FALSE to terminate it
*/
public function valid() {
return !is_null($this->currentRow);
@@ -229,6 +259,9 @@ abstract class MigrateSource implements Iterator {
$this->numProcessed = 0;
$this->numIgnored = 0;
$this->highwaterField = $this->activeMigration->getHighwaterField();
if (!empty($this->highwaterField)) {
$this->originalHighwater = $this->activeMigration->getHighwater();
}
if ($this->activeMigration->getOption('idlist')) {
$this->idList = explode(',', $this->activeMigration->getOption('idlist'));
}
@@ -248,9 +281,11 @@ abstract class MigrateSource implements Iterator {
public function next() {
$this->currentKey = NULL;
$this->currentRow = NULL;
migrate_instrument_start(get_class($this) . ' getNextRow');
while ($row = $this->getNextRow()) {
migrate_instrument_stop(get_class($this) . ' getNextRow');
// Populate the source key for this row
$this->currentKey = $this->activeMigration->prepareKey(
$this->activeMap->getSourceKey(), $row);
@@ -262,28 +297,29 @@ abstract class MigrateSource implements Iterator {
if ($map_row) {
foreach ($map_row as $field => $value) {
$field = 'migrate_map_' . $field;
$row->$field = $value;
$row->{$field} = $value;
}
}
}
// First, determine if this row should be passed to prepareRow(), or skipped
// entirely. The rules are:
// First, determine if this row should be passed to prepareRow(), or
// skipped entirely. The rules are:
// 1. If there's an explicit idlist, that's all we care about (ignore
// highwaters and map rows).
$prepared = FALSE;
if (!empty($this->idList)) {
if (in_array(reset($this->currentKey), $this->idList)) {
// In the list, fall through.
}
else {
// Not in the list, skip it
$this->currentRow = NULL;
continue;
// Check first source key.
if (!in_array(reset($this->currentKey), $this->idList)) {
// If this is a compound source key, check the full key.
$compoundKey = implode($this->multikeySeparator, $this->currentKey);
if (count($this->currentKey) == 1 || !in_array($compoundKey, $this->idList)) {
// Could not find the key, skip.
continue;
}
}
}
// 2. If the row is not in the map (we have never tried to import it before),
// we always want to try it.
// 2. If the row is not in the map (we have never tried to import it
// before), we always want to try it.
elseif (!isset($row->migrate_map_sourceid1)) {
// Fall through
}
@@ -293,36 +329,52 @@ abstract class MigrateSource implements Iterator {
}
// 4. At this point, we have a row which has previously been imported and
// not marked for update. If we're not using highwater marks, then we
// will not take this row.
// will not take this row. Except, if we're looking for changes in the
// data, we need to go through prepareRow() before we can decide to
// skip it.
elseif (empty($this->highwaterField)) {
// No highwater, skip
$this->currentRow = NULL;
continue;
if ($this->trackChanges) {
if ($this->prepareRow($row) !== FALSE) {
if ($this->dataChanged($row)) {
// This is a keeper
$this->currentRow = $row;
break;
}
else {
// No change, skip it.
continue;
}
}
else {
// prepareRow() told us to skip it.
continue;
}
}
else {
// No highwater and not tracking changes, skip.
continue;
}
}
// 5. The initial highwater mark, before anything is migrated, is ''. We
// want to make sure we don't mistakenly skip rows with a highwater
// field value of 0, so explicitly handle '' here.
elseif ($this->activeMigration->getHighwater() === '') {
elseif ($this->originalHighwater === '') {
// Fall through
}
// 6. So, we are using highwater marks. Take the row if its highwater field
// value is greater than the saved mark, otherwise skip it.
// 6. So, we are using highwater marks. Take the row if its highwater
// field value is greater than the saved mark, otherwise skip it.
else {
// Call prepareRow() here, in case the highwaterField needs preparation
if ($this->prepareRow($row) !== FALSE) {
if ($row->{$this->highwaterField['name']} > $this->activeMigration->getHighwater()) {
if ($row->{$this->highwaterField['name']} > $this->originalHighwater) {
$this->currentRow = $row;
break;
}
else {
// Skip
$this->currentRow = NULL;
continue;
}
}
else {
$this->currentRow = NULL;
}
$prepared = TRUE;
}
@@ -358,6 +410,10 @@ abstract class MigrateSource implements Iterator {
migrate_instrument_stop(get_class($this->activeMigration) . ' prepareRow');
// We're explicitly skipping this row - keep track in the map table
if ($return === FALSE) {
// Make sure we replace any previous messages for this item with any
// new ones.
$this->activeMigration->getMap()->delete($this->currentKey, TRUE);
$this->activeMigration->saveQueuedMessages();
$this->activeMigration->getMap()->saveIDMapping($row, array(),
MigrateMap::STATUS_IGNORED, $this->activeMigration->rollbackAction);
$this->numIgnored++;
@@ -366,8 +422,62 @@ abstract class MigrateSource implements Iterator {
}
else {
$return = TRUE;
// When tracking changed data, We want to quietly skip (rather than
// "ignore") rows with changes. The caller needs to make that decision,
// so we need to provide them with the necessary information (before and
// after hashes).
if ($this->trackChanges) {
$unhashed_row = clone $row;
// Remove all map data, otherwise we'll have a false positive on the
// second import (attempt) on a row.
foreach ($unhashed_row as $field => $data) {
if (strpos($field, 'migrate_map_') === 0) {
unset($unhashed_row->{$field});
}
}
$row->migrate_map_original_hash = isset($row->migrate_map_hash) ?
$row->migrate_map_hash : '';
$row->migrate_map_hash = $this->hash($unhashed_row);
}
else {
$row->migrate_map_hash = '';
}
}
$this->numProcessed++;
return $return;
}
/**
* Determine whether this row has changed, and therefore whether it should
* be processed.
*
* @param $row
*
* @return bool
*/
protected function dataChanged($row) {
if ($row->migrate_map_original_hash != $row->migrate_map_hash) {
$return = TRUE;
}
else {
$return = FALSE;
}
return $return;
}
/**
* Generate a hash of the source row.
*
* @param $row
*
* @return string
*/
protected function hash($row) {
migrate_instrument_start('MigrateSource::hash');
$hash = md5(serialize($row));
migrate_instrument_stop('MigrateSource::hash');
return $hash;
}
}
sites/all/modules/contrib/migrate/migrate/includes/team.inc
+4
View File
@@ -7,17 +7,21 @@
*/
class MigrateTeamMember {
protected $name;
public function getName() {
return $this->name;
}
protected $emailAddress;
public function getEmailAddress() {
return $this->emailAddress;
}
protected $group;
public function getGroup() {
return $this->group;
}
sites/all/modules/contrib/migrate/migrate/migrate.api.php
+83 -1
View File
@@ -6,17 +6,99 @@
*/
/**
* Registers your module as an implementor of Migrate-based classes.
* Registers your module as an implementor of Migrate-based classes and
* provides
* default configuration for migration processes.
*
* @return
* An associative array with the following keys (of which only 'api' is
* required):
* - api: Always 2 for any module implementing the Migrate 2 API.
* - groups: An associative array, keyed by group machine name, defining one
* or more migration groups. Each value is an associative array - the
* 'title' key defines a user-visible name for the group; any other values
* are passed as arguments to all migrations in the group.
* - migrations: An associative array, keyed by migration machine name,
* defining one or more migrations. Each value is an associative array -
* any
* keys other than the following are passed as arguments to the migration
* constructor:
* - class_name (required): The name of the class implementing the
* migration.
* - group_name: The machine name of the group containing the migration.
* - disable_hooks: An associative array, keyed by hook name, listing hook
* implementations to be disabled during migration. Each value is an
* array of module names whose implementations of the hook in the key is
* to be disabled.
* - destination handlers: An array of classes implementing destination
* handlers.
* - field handlers: An array of classes implementing field handlers.
* - wizard classes: An array of classes that provide Migrate UI wizards.
* - wizard extenders: An array of classes that extend Migrate UI wizards.
* Keys are the wizard classes, values are arrays of extender classes.
*
* See system_hook_info() for all hook groups defined by Drupal core.
*
* @see hook_migrate_api_alter().
*/
function hook_migrate_api() {
$api = array(
'api' => 2,
'groups' => array(
'legacy' => array(
'title' => t('Import from legacy system'),
// Default format for all content migrations
'default_format' => 'filtered_html',
),
),
'migrations' => array(
'ExampleUser' => array(
'class_name' => 'ExampleUserMigration',
'group_name' => 'legacy',
'default_role' => 'member', // Added to constructor $arguments
),
'ExampleNode' => array(
'class_name' => 'ExampleNodeMigration',
'group_name' => 'legacy',
'default_uid' => 1, // Added to constructor $arguments
'disable_hooks' => array(
// Improve migration performance, and prevent accidental emails.
'node_insert' => array(
'expensive_module',
'email_notification_module',
),
'node_update' => array(
'expensive_module',
'email_notification_module',
),
),
),
),
);
return $api;
}
/**
* Alter information from all implementations of hook_migrate_api().
*
* @param array $info
* An array of results from hook_migrate_api(), keyed by module name.
*
* @see hook_migrate_api().
*/
function hook_migrate_api_alter(array &$info) {
// Override the class for another module's migration - say, to add some
// additional preprocessing in prepareRow().
if (isset($info['MODULE_NAME']['migrations']['ExampleNode'])) {
$info['MODULE_NAME']['migrations']['ExampleNode']['class_name'] = 'MyBetterExampleNodeMigration';
}
}
/**
* Provides text to be displayed at the top of the dashboard page (migrate_ui).
*
* @return
* Translated text for display on the dashboard page.
*/
function hook_migrate_overview() {
return t('<p>Listed below are all the migration processes defined for migration
sites/all/modules/contrib/migrate/migrate/migrate.drush.inc
+356 -163
View File
@@ -17,12 +17,15 @@ function migrate_drush_command() {
'instrument' => 'Capture performance information (timer, memory, or all)',
'force' => 'Force an operation to run, even if all dependencies are not satisfied',
'group' => 'Name of the migration group to run',
'notify' => 'Send email notification upon completion of operation',
'wildcard' => 'Process migrations that match a certain pattern. For example, Content*.',
);
$items['migrate-status'] = array(
'description' => 'List all migrations with current status.',
'options' => array(
'refresh' => 'Recognize new migrations and update counts',
'group' => 'Name of the migration group to list',
'names-only' => 'Only return names, not all the details (faster)',
),
'arguments' => array(
'migration' => 'Restrict to a single migration. Optional',
@@ -129,8 +132,6 @@ function migrate_drush_command() {
$items['migrate-rollback'] = array(
'description' => 'Roll back the destination objects from a given migration',
'options' => $migration_options,
// We will bootstrap to login from within the command callback.
'bootstrap' => DRUSH_BOOTSTRAP_DRUPAL_FULL,
'arguments' => array(
'migration' => 'Name of migration(s) to roll back. Delimit multiple using commas.',
),
@@ -144,12 +145,13 @@ function migrate_drush_command() {
'drupal dependencies' => array('migrate'),
'aliases' => array('mr'),
);
$migration_options['update'] = 'In addition to processing unimported items from the source, update previously-imported items with new data';
$migration_options['update'] = 'In addition to processing unprocessed items from the source, update previously-imported items with new data';
$migration_options['needs-update'] =
'Reimport up to 10K records where needs_update=1. This option is only needed when your Drupal DB is on a different DB server from your source data. Otherwise, these records get migrated with just migrate-import.';
$migration_options['stop'] = 'Stop specified migration(s) if applicable.';
$migration_options['rollback'] = 'Rollback specified migration(s) if applicable.';
$migration_options['file_function'] = 'Override file function to use when migrating images.';
$migration_options['ignore-highwater'] = 'Ignore the highwater field during migration';
$items['migrate-import'] = array(
'description' => 'Perform one or more migration processes',
'options' => $migration_options,
@@ -160,6 +162,7 @@ function migrate_drush_command() {
'migrate-import Article' => 'Import new articles',
'migrate-import Article --update' => 'Import new items, and also update previously-imported items',
'migrate-import Article --idlist=4,9' => 'Import two specific articles. The ids refer to the value of the primary key in base table',
'migrate-import Article --idlist=450:pasta,451' => 'Import two specific articles. A colon can be used to separate parts of compound keys; otherwise, compound keys match by the first key field.',
'migrate-import Article --limit="60 seconds" --stop --rollback' =>
'Import for up to 60 seconds after stopping and rolling back the Article migration.',
'migrate-import Article --limit="100 items"' =>
@@ -172,7 +175,10 @@ function migrate_drush_command() {
);
$items['migrate-stop'] = array(
'description' => 'Stop an active migration operation',
'options' => array('all' => 'Stop all active migration operations'),
'options' => array(
'all' => 'Stop all active migration operations',
'group' => 'Name of a specific migration group to stop',
),
'arguments' => array(
'migration' => 'Name of migration to stop',
),
@@ -198,25 +204,35 @@ function migrate_drush_command() {
);
$items['migrate-deregister'] = array(
'description' => 'Remove all tracking of a migration',
'options' => array('orphans' => 'Remove tracking for any migrations whose implementing class no longer exists'),
'options' => array(
'orphans' => 'Remove tracking for any migrations whose implementing class no longer exists',
'group' => 'Remove tracking of a migration group, and any migrations assigned to it',
),
'arguments' => array(
'migration' => 'Name of migration to deregister',
),
'examples' => array(
'migrate-deregister Article' => 'Deregister the Article migration',
'migrate-deregister --orphans' => 'Deregister any no-longer-implemented migrations',
'migrate-deregister --group=myblog' => 'Deregister the myblog group and all migrations within it',
),
'drupal dependencies' => array('migrate'),
'aliases' => array('mdreg'),
);
$items['migrate-auto-register'] = array(
'description' => 'Register any newly-defined migration classes',
'description' => 'Register any newly defined migration classes',
'drupal dependencies' => array('migrate'),
'aliases' => array('mar'),
);
$items['migrate-register'] = array(
'description' => 'Register or reregister any statically defined migrations',
'drupal dependencies' => array('migrate'),
'aliases' => array('mreg'),
);
$items['migrate-wipe'] = array(
'description' => 'Delete all nodes from specified content types.',
'examples' => array(
"migrate-wipe story article" => 'Delete all story and article nodes.',
"migrate-wipe story article" => 'Delete all story and article nodes.',
),
'arguments' => array(
'type' => 'A space delimited list of content type machine readable Ids.',
@@ -236,9 +252,11 @@ function migrate_drush_command() {
*/
function drush_migrate_get_options() {
$options = array();
$blacklist = array('stop', 'rollback', 'update', 'all');
$blacklist = array('stop', 'rollback', 'update', 'all', 'group');
$command = drush_parse_command();
foreach ($command['options'] as $key => $value) {
$global_options = drush_get_global_options();
$opts = array_merge($command['options'], $global_options);
foreach ($opts as $key => $value) {
// Strip leading --
$key = ltrim($key, '-');
if (!in_array($key, $blacklist)) {
@@ -281,6 +299,7 @@ function drush_migrate_status($name = NULL) {
try {
$refresh = drush_get_option('refresh');
$group_option = drupal_strtolower(drush_get_option('group'));
$names_only = drush_get_option('names-only');
// Validate input and load Migration(s).
if ($name) {
@@ -311,61 +330,88 @@ function drush_migrate_status($name = NULL) {
if ($group_members_count == 1) {
// An empty line and the headers.
$table[] = array('');
$table[] = array(dt('Group: !name', array('!name' => $group->getName())), dt('Total'), dt('Imported'), dt('Unimported'), dt('Status'), dt('Last imported'));
if ($names_only) {
$table[] = array(
dt('Group: !name',
array('!name' => $group->getName())),
);
}
else {
$table[] = array(
dt('Group: !name',
array('!name' => $group->getName())),
dt('Total'),
dt('Imported'),
dt('Unprocessed'),
dt('Status'),
dt('Last imported'),
);
}
}
$has_counts = TRUE;
if (method_exists($migration, 'sourceCount')) {
$total = $migration->sourceCount($refresh);
if ($total < 0) {
if (!$names_only) {
$has_counts = TRUE;
if (method_exists($migration, 'sourceCount')) {
$total = $migration->sourceCount($refresh);
if ($total < 0) {
$has_counts = FALSE;
$total = dt('N/A');
}
}
else {
$has_counts = FALSE;
$total = dt('N/A');
}
if (method_exists($migration, 'importedCount')) {
$imported = $migration->importedCount();
$processed = $migration->processedCount();
}
else {
$has_counts = FALSE;
$imported = dt('N/A');
}
if ($has_counts) {
$unimported = $total - $processed;
}
else {
$unimported = dt('N/A');
}
$status = $migration->getStatus();
switch ($status) {
case MigrationBase::STATUS_IDLE:
$status = dt('Idle');
break;
case MigrationBase::STATUS_IMPORTING:
$status = dt('Importing');
break;
case MigrationBase::STATUS_ROLLING_BACK:
$status = dt('Rolling back');
break;
case MigrationBase::STATUS_STOPPING:
$status = dt('Stopping');
break;
case MigrationBase::STATUS_DISABLED:
$status = dt('Disabled');
break;
default:
$status = dt('Unknown');
break;
}
$table[] = array(
$migration->getMachineName(),
$total,
$imported,
$unimported,
$status,
$migration->getLastImported(),
);
}
else {
$has_counts = FALSE;
$total = dt('N/A');
$table[] = array($migration->getMachineName());
}
if (method_exists($migration, 'importedCount')) {
$imported = $migration->importedCount();
$processed = $migration->processedCount();
}
else {
$has_counts = FALSE;
$imported = dt('N/A');
}
if ($has_counts) {
$unimported = $total - $processed;
}
else {
$unimported = dt('N/A');
}
$status = $migration->getStatus();
switch ($status) {
case MigrationBase::STATUS_IDLE:
$status = dt('Idle');
break;
case MigrationBase::STATUS_IMPORTING:
$status = dt('Importing');
break;
case MigrationBase::STATUS_ROLLING_BACK:
$status = dt('Rolling back');
break;
case MigrationBase::STATUS_STOPPING:
$status = dt('Stopping');
break;
case MigrationBase::STATUS_DISABLED:
$status = dt('Disabled');
break;
default:
$status = dt('Unknown');
break;
}
$table[] = array($migration->getMachineName(), $total, $imported, $unimported, $status, $migration->getLastImported());
}
}
drush_print_table($table);
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -382,7 +428,7 @@ function drush_migrate_fields_destination($args = NULL) {
if (method_exists($destination, 'fields')) {
$table = array();
foreach ($destination->fields($migration) as $machine_name => $description) {
$table[] = array($description, $machine_name);
$table[] = array(strip_tags($description), $machine_name);
}
drush_print_table($table);
}
@@ -390,8 +436,7 @@ function drush_migrate_fields_destination($args = NULL) {
drush_print(dt('No fields were found.'));
}
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -407,7 +452,7 @@ function drush_migrate_fields_source($args = NULL) {
if (method_exists($source, 'fields')) {
$table = array();
foreach ($source->fields() as $machine_name => $description) {
$table[] = array($description, $machine_name);
$table[] = array(strip_tags($description), $machine_name);
}
drush_print_table($table);
}
@@ -415,8 +460,7 @@ function drush_migrate_fields_source($args = NULL) {
drush_print(dt('No fields were found.'));
}
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -438,14 +482,20 @@ function drush_migrate_mappings($args = NULL) {
$dest_descriptions = array();
if (method_exists($destination, 'fields')) {
foreach ($destination->fields($migration) as $machine_name => $description) {
$dest_descriptions[$machine_name] = $description;
if (is_array($description)) {
$description = reset($description);
}
$dest_descriptions[$machine_name] = strip_tags($description);
}
}
$source = $migration->getSource();
$src_descriptions = array();
if (method_exists($source, 'fields')) {
foreach ($source->fields() as $machine_name => $description) {
$src_descriptions[$machine_name] = $description;
if (is_array($description)) {
$description = reset($description);
}
$src_descriptions[$machine_name] = strip_tags($description);
}
}
}
@@ -474,12 +524,22 @@ function drush_migrate_mappings($args = NULL) {
// Put out each group header
$table = array();
if ($full) {
$table[] = array(dt('Destination'), dt(''), dt('Source'), dt(''), dt('Default'),
dt('Description'));
$table[] = array(
dt('Destination'),
dt(''),
dt('Source'),
dt(''),
dt('Default'),
dt('Description'),
);
}
else {
$table[] = array(dt('Destination'), dt('Source'), dt('Default'),
dt('Description'));
$table[] = array(
dt('Destination'),
dt('Source'),
dt('Default'),
dt('Description'),
);
}
$first = TRUE;
@@ -515,12 +575,22 @@ function drush_migrate_mappings($args = NULL) {
else {
$src_description = '';
}
$table[] = array($destination, $dest_description, $source, $src_description,
$default, $mapping->getDescription());
$table[] = array(
$destination,
$dest_description,
$source,
$src_description,
$default,
$mapping->getDescription(),
);
}
else {
$table[] = array($destination, $source,
$default, $mapping->getDescription());
$table[] = array(
$destination,
$source,
$default,
$mapping->getDescription(),
);
}
}
}
@@ -534,8 +604,7 @@ function drush_migrate_mappings($args = NULL) {
}
}
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -545,14 +614,18 @@ function drush_migrate_mappings($args = NULL) {
* Display messages for a migration.
*/
function drush_migrate_messages($migration_name) {
if (!trim($migration_name)) {
drush_log(dt('You must specify a migration name'), 'status');
return;
}
try {
$migration = MigrationBase::getInstance($migration_name);
if (is_a($migration, 'Migration')) {
$map = $migration->getMap();
$message_table = $map->getMessageTable();
$result = db_select($message_table, 'msg', array('fetch' => PDO::FETCH_ASSOC))
->fields('msg')
->execute();
->fields('msg')
->execute();
$first = TRUE;
$table = array();
foreach ($result as $row) {
@@ -582,8 +655,7 @@ function drush_migrate_messages($migration_name) {
drush_print_table($table, TRUE, $widths);
}
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -602,8 +674,10 @@ function drush_migrate_analyze($args = NULL) {
if (!empty($analysis)) {
foreach ($analysis as $field_name => $details) {
if (!empty($details['description'])) {
drush_print(dt('@name (@description):', array('@name' => $field_name,
'@description' => $details['description'])));
drush_print(dt('@name (@description):', array(
'@name' => $field_name,
'@description' => $details['description'],
)));
}
else {
drush_print(dt('@name:', array('@name' => $field_name)));
@@ -616,17 +690,23 @@ function drush_migrate_analyze($args = NULL) {
}
else {
drush_print(' ' . dt('Only one value present: @value',
array('@value' => $value)));
array('@value' => $value)));
}
}
else {
if ($details['is_numeric']) {
drush_print(' ' . dt('Numeric field with a range of @min to @max',
array('@min' => $details['min_numeric'], '@max' => $details['max_numeric'])));
array(
'@min' => $details['min_numeric'],
'@max' => $details['max_numeric'],
)));
}
else {
drush_print(' ' . dt('String field with a length ranging from @min to @max',
array('@min' => $details['min_strlen'], '@max' => $details['max_strlen'])));
array(
'@min' => $details['min_strlen'],
'@max' => $details['max_strlen'],
)));
}
$values = array();
$header = NULL;
@@ -742,10 +822,16 @@ function drush_migrate_audit($args = NULL) {
}
foreach (array_diff_key($source_fields, $used_sources) as $name => $description) {
$problems['sources_unmapped'][] = array('Field' => $name, 'Description' => $description);
$problems['sources_unmapped'][] = array(
'Field' => $name,
'Description' => $description,
);
}
foreach (array_diff_key($destination_fields, $used_destinations) as $name => $description) {
$problems['destinations_unmapped'][] = array('Field' => $name, 'Description' => $description);
$problems['destinations_unmapped'][] = array(
'Field' => $name,
'Description' => $description,
);
}
$problems = array_filter($problems);
@@ -771,8 +857,7 @@ function drush_migrate_audit($args = NULL) {
}
}
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -783,6 +868,12 @@ function drush_migrate_audit($args = NULL) {
*/
function drush_migrate_rollback($args = NULL) {
try {
if (drush_get_option('notify', FALSE)) {
// Capture non-informational output for mailing
ob_start();
ob_implicit_flush(FALSE);
}
$migrations = drush_migrate_get_migrations($args);
// Rollback in reverse order
@@ -804,9 +895,9 @@ function drush_migrate_rollback($args = NULL) {
$options['limit']['unit'] = 'items';
}
elseif ($options['limit']['unit'] != 'seconds' &&
$options['limit']['unit'] != 'second' &&
$options['limit']['unit'] != 'items' &&
$options['limit']['unit'] != 'item') {
$options['limit']['unit'] != 'second' &&
$options['limit']['unit'] != 'items' &&
$options['limit']['unit'] != 'item') {
drush_set_error(NULL, dt("Invalid limit unit '!unit'",
array('!unit' => $options['limit']['unit'])));
return;
@@ -819,9 +910,9 @@ function drush_migrate_rollback($args = NULL) {
$options['feedback']['value'] = $parts[0];
$options['feedback']['unit'] = $parts[1];
if ($options['feedback']['unit'] != 'seconds' &&
$options['feedback']['unit'] != 'second' &&
$options['feedback']['unit'] != 'items' &&
$options['feedback']['unit'] != 'item') {
$options['feedback']['unit'] != 'second' &&
$options['feedback']['unit'] != 'items' &&
$options['feedback']['unit'] != 'item') {
drush_set_error(NULL, dt("Invalid feedback frequency unit '!unit'",
array('!unit' => $options['feedback']['unit'])));
return;
@@ -862,8 +953,7 @@ function drush_migrate_rollback($args = NULL) {
'warning');
}
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -873,6 +963,31 @@ function drush_migrate_rollback($args = NULL) {
if ($_migrate_track_timer && !drush_get_context('DRUSH_DEBUG')) {
drush_print_timers();
}
// Notify user
if (drush_get_option('notify')) {
_drush_migrate_notify();
}
}
/**
* Send email notification to the user running the operation.
*/
function _drush_migrate_notify() {
global $user;
if ($user->uid) {
$uid = $user->uid;
}
else {
$uid = 1;
}
$account = user_load($uid);
$params['account'] = $account;
$params['output'] = ob_get_contents();
drush_print_r(ob_get_status());
ob_end_flush();
drupal_mail('migrate_ui', 'import_complete', $account->mail,
user_preferred_language($account), $params);
}
function drush_migrate_get_migrations($args) {
@@ -883,7 +998,7 @@ function drush_migrate_get_migrations($args) {
$seen = $start === TRUE ? TRUE : FALSE;
foreach ($migration_objects as $name => $migration) {
if (!$seen && (drupal_strtolower($start) . 'migration' == drupal_strtolower($name))) {
if (!$seen && (drupal_strtolower($start) == drupal_strtolower($name))) {
// We found our starting migration. $seen is always TRUE now.
$seen = TRUE;
}
@@ -896,8 +1011,16 @@ function drush_migrate_get_migrations($args) {
elseif ($group = drush_get_option('group')) {
foreach ($migration_objects as $name => $migration) {
if (drupal_strtolower($group) !=
drupal_strtolower($migration->getGroup()->getName()) ||
!$migration->getEnabled()) {
drupal_strtolower($migration->getGroup()->getName()) ||
!$migration->getEnabled()) {
unset($migration_objects[$name]);
}
}
}
elseif ($wildcard = drush_get_option('wildcard')) {
foreach ($migration_objects as $name => $migration) {
if (!fnmatch(drupal_strtolower($wildcard), drupal_strtolower($name)) ||
!$migration->getEnabled()) {
unset($migration_objects[$name]);
}
}
@@ -974,28 +1097,33 @@ function drush_migrate_rollback_validate($args = NULL) {
function drush_migrate_validate_common($args) {
if (drush_get_option('all')) {
if (!empty($args) || drush_get_option('group')) {
return drush_set_error(NULL, dt('You must specify exactly one of a migration name, --all, or --group'));
if (!empty($args) || drush_get_option('group') || drush_get_option('wildcard')) {
return drush_set_error(NULL, dt('You must specify exactly one of a migration name, --all, or --group, or --wildcard'));
}
}
elseif (drush_get_option('group')) {
if (!empty($args) || drush_get_option('all')) {
return drush_set_error(NULL, dt('You must specify exactly one of a migration name, --all, or --group'));
if (!empty($args) || drush_get_option('all') || drush_get_option('wildcard')) {
return drush_set_error(NULL, dt('You must specify exactly one of a migration name, --all, or --group, or --wildcard'));
}
}
elseif (drush_get_option('wildcard')) {
if (!empty($args) || drush_get_option('all') || drush_get_option('group')) {
return drush_set_error(NULL, dt('You must specify exactly one of a migration name, --all, or --group, or --wildcard'));
}
}
else {
if (empty($args)) {
return drush_set_error(NULL, dt('You must specify exactly one of a migration name, --all, or --group'));
return drush_set_error(NULL, dt('You must specify exactly one of a migration name, --all, or --group, or --wildcard'));
}
$machine_names = explode(',', $args);
foreach ($machine_names as $machine_name) {
$machine_name = trim($machine_name);
$class_name = db_select('migrate_status', 'ms')
->fields('ms', array('class_name'))
->condition('machine_name', $machine_name)
->execute()
->fetchField();
->fields('ms', array('class_name'))
->condition('machine_name', $machine_name)
->execute()
->fetchField();
if (!$class_name || !class_exists($class_name)) {
drush_set_error(dt('Unrecognized migration: !name', array('!name' => $machine_name)));
}
@@ -1008,9 +1136,9 @@ function drush_migrate_validate_common($args) {
$options['feedback']['value'] = $parts[0];
$options['feedback']['unit'] = $parts[1];
if ($options['feedback']['unit'] != 'seconds' &&
$options['feedback']['unit'] != 'second' &&
$options['feedback']['unit'] != 'items' &&
$options['feedback']['unit'] != 'item') {
$options['feedback']['unit'] != 'second' &&
$options['feedback']['unit'] != 'items' &&
$options['feedback']['unit'] != 'item') {
drush_set_error(NULL, dt("Invalid feedback frequency unit '!unit'",
array('!unit' => $options['feedback']['unit'])));
return;
@@ -1024,7 +1152,28 @@ function drush_migrate_validate_common($args) {
*/
function drush_migrate_pre_migrate_import($args = NULL) {
if (drush_get_option('stop')) {
drush_invoke('migrate-stop', $args);
drush_unset_option('stop');
try {
/** @var Migration[] $migrations */
$migrations = drush_migrate_get_migrations($args);
foreach ($migrations as $migration) {
$status = $migration->getStatus();
if ($status == MigrationBase::STATUS_IMPORTING ||
$status == MigrationBase::STATUS_ROLLING_BACK) {
drush_log(dt("Stopping '!description' migration", array('!description' => $migration->getMachineName())));
$migration->stopProcess();
// Give the process a chance to stop.
$count = 0;
while ($migration->getStatus() != MigrationBase::STATUS_IDLE
&& $count++ < 5) {
sleep(1);
}
}
}
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
}
if (drush_get_option('rollback')) {
drush_unset_option('rollback');
@@ -1039,7 +1188,14 @@ function drush_migrate_pre_migrate_import($args = NULL) {
* A comma delimited list of machine names, or the special name 'all'
*/
function drush_migrate_import($args = NULL) {
$status = NULL;
try {
if (drush_get_option('notify', FALSE)) {
// Capture non-informational output for mailing
ob_start();
ob_implicit_flush(FALSE);
}
$migrations = drush_migrate_get_migrations($args);
$options = array();
if ($idlist = drush_get_option('idlist', FALSE)) {
@@ -1055,14 +1211,18 @@ function drush_migrate_import($args = NULL) {
if ($limit) {
$parts = explode(' ', $limit);
$options['limit']['value'] = $parts[0];
$options['limit']['unit'] = $parts[1];
if (!$options['limit']['unit']) {
$options['limit']['unit'] = 'items';
// Default unit.
if (!isset($parts[1])) {
$parts[1] = 'items';
}
elseif ($options['limit']['unit'] != 'seconds' &&
$options['limit']['unit'] != 'second' &&
$options['limit']['unit'] != 'items' &&
$options['limit']['unit'] != 'item') {
$options['limit']['unit'] = $parts[1];
// Validation.
if (!in_array($options['limit']['unit'], array(
'seconds',
'second',
'items',
'item',
))) {
drush_set_error(NULL, dt("Invalid limit unit '!unit'",
array('!unit' => $options['limit']['unit'])));
return;
@@ -1074,9 +1234,9 @@ function drush_migrate_import($args = NULL) {
$options['feedback']['value'] = $parts[0];
$options['feedback']['unit'] = $parts[1];
if ($options['feedback']['unit'] != 'seconds' &&
$options['feedback']['unit'] != 'second' &&
$options['feedback']['unit'] != 'items' &&
$options['feedback']['unit'] != 'item') {
$options['feedback']['unit'] != 'second' &&
$options['feedback']['unit'] != 'items' &&
$options['feedback']['unit'] != 'item') {
drush_set_error(NULL, dt("Invalid feedback frequency unit '!unit'",
array('!unit' => $options['feedback']['unit'])));
return;
@@ -1101,8 +1261,12 @@ function drush_migrate_import($args = NULL) {
foreach ($migrations as $machine_name => $migration) {
drush_log(dt("Importing '!description' migration",
array('!description' => $machine_name)));
if (drush_get_option('update')) {
if (drush_get_option('update') && !$idlist) {
$migration->prepareUpdate();
if (drush_get_option('ignore-highwater')) {
$migration->setHighwaterField(array());
}
}
if (drush_get_option('needs-update')) {
$map_rows = $migration->getMap()->getRowsNeedingUpdate(10000);
@@ -1168,8 +1332,10 @@ function drush_migrate_import($args = NULL) {
elseif ($status == MigrationBase::RESULT_INCOMPLETE) {
$stop = TRUE;
}
drush_backend_set_result(array('status' => $status,
'migrations' => implode(',', array_keys($migrations))));
drush_backend_set_result(array(
'status' => $status,
'migrations' => implode(',', array_keys($migrations)),
));
}
if ($stop) {
@@ -1177,8 +1343,7 @@ function drush_migrate_import($args = NULL) {
}
unset($migrations[$machine_name]);
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -1190,14 +1355,21 @@ function drush_migrate_import($args = NULL) {
if ($_migrate_track_timer && !drush_get_context('DRUSH_DEBUG')) {
drush_print_timers();
}
// Notify user
if (drush_get_option('notify')) {
_drush_migrate_notify();
}
return $status;
}
//**
// * Stop clearing or importing a given content set.
// *
// * @param $content_set
// * The name of the Migration
// */
/**
* Stop clearing or importing a given content set.
*
* @param $content_set
* The name of the Migration
*/
function drush_migrate_stop($args = NULL) {
try {
$migrations = drush_migrate_get_migrations($args);
@@ -1205,8 +1377,7 @@ function drush_migrate_stop($args = NULL) {
drush_log(dt("Stopping '!description' migration", array('!description' => $migration->getMachineName())));
$migration->stopProcess();
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -1223,42 +1394,49 @@ function drush_migrate_reset_status($args = NULL) {
array('!description' => $migration->getMachineName())));
$migration->resetStatus();
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
}
/**
* Deregister a given migration, or all orphaned migrations. Note that the
* migration might no longer "exist" (the class implementation might be gone),
* so we can't count on being able to instantiate it, or use migrate_migrations().
* Deregister a given migration, migration group, or all orphaned migrations.
* Note that the migration might no longer "exist" (the class implementation
* might be gone), so we can't count on being able to instantiate it, or use
* migrate_migrations().
*/
function drush_migrate_deregister($args = NULL) {
try {
$orphans = drush_get_option('orphans');
if ($orphans) {
$migrations = array();
$result = db_select('migrate_status', 'ms')
->fields('ms', array('class_name', 'machine_name'))
->execute();
foreach ($result as $row) {
if (!class_exists($row->class_name)) {
$migrations[] = $row->machine_name;
}
}
$group = drush_get_option('group');
if ($group) {
MigrateGroup::deregister($group);
drush_log(dt("Deregistered group '!description' and all its migrations",
array('!description' => $group)), 'success');
}
else {
$migrations = explode(',', $args);
if ($orphans) {
$migrations = array();
$result = db_select('migrate_status', 'ms')
->fields('ms', array('class_name', 'machine_name'))
->execute();
foreach ($result as $row) {
if (!class_exists($row->class_name)) {
$migrations[] = $row->machine_name;
}
}
}
else {
$migrations = explode(',', $args);
}
foreach ($migrations as $machine_name) {
drush_migrate_deregister_migration(drupal_strtolower($machine_name));
drush_log(dt("Deregistered '!description' migration",
array('!description' => $machine_name)), 'success');
}
}
foreach ($migrations as $machine_name) {
drush_migrate_deregister_migration($machine_name);
drush_log(dt("Deregistered '!description' migration",
array('!description' => $machine_name)), 'success');
}
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
drush_print($e->getMessage());
exit;
}
@@ -1277,13 +1455,28 @@ function drush_migrate_deregister_migration($machine_name) {
db_delete('migrate_status')
->condition('machine_name', $machine_name)
->execute();
db_delete('migrate_field_mapping')
->condition('machine_name', $machine_name)
->execute();
}
/**
* Register any previously-unrecognized non-dynamic migrations.
* Auto-registration is no longer supported. This command should be removed
* entirely in a future point release.
*
* @deprecated
*/
function drush_migrate_auto_register($args = NULL) {
migrate_autoregister();
drush_log(dt('The auto-registration feature has been removed. Migrations '
. 'must now be explicitly registered.'), 'error');
}
/**
* Register any migrations defined in hook_migrate_api().
*/
function drush_migrate_register($args = NULL) {
migrate_static_registration();
drush_log(dt('All statically defined migrations have been (re)registered.'), 'success');
}
/**
@@ -1292,10 +1485,10 @@ function drush_migrate_auto_register($args = NULL) {
function drush_migrate_wipe() {
$types = func_get_args();
$nids = db_select('node', 'n')
->fields('n', array('nid'))
->condition('type', $types, 'IN')
->execute()
->fetchCol();
->fields('n', array('nid'))
->condition('type', $types, 'IN')
->execute()
->fetchCol();
$chunks = array_chunk($nids, 50);
foreach ($chunks as $chunk) {
node_delete_multiple($chunk);
@@ -1307,7 +1500,7 @@ function drush_migrate_wipe() {
function drush_migrate_print_memory() {
global $_migrate_memory;
$temparray = array();
foreach ((array)$_migrate_memory as $name => $memoryrec) {
foreach ((array) $_migrate_memory as $name => $memoryrec) {
// We have to use timer_read() for active timers, and check the record for others
if (isset($memoryrec['start'])) {
$temparray[$name] = migrate_memory_read($name);
@@ -1325,7 +1518,7 @@ function drush_migrate_print_memory() {
foreach ($temparray as $name => $memory) {
$count = $_migrate_memory[$name]['count'];
if ($count > 0) {
$avg = round($memory/$count, 0);
$avg = round($memory / $count, 0);
}
else {
$avg = 'N/A';
sites/all/modules/contrib/migrate/migrate/migrate.info
+9 -5
View File
@@ -1,6 +1,6 @@
name = "Migrate"
description = "Import content from external sources"
package = "Development"
package = "Migration"
core = 7.x
files[] = includes/base.inc
@@ -14,6 +14,7 @@ files[] = includes/map.inc
files[] = includes/source.inc
files[] = includes/team.inc
files[] = migrate.mail.inc
files[] = plugins/destinations/block_custom.inc
files[] = plugins/destinations/entity.inc
files[] = plugins/destinations/term.inc
files[] = plugins/destinations/user.inc
@@ -28,15 +29,19 @@ files[] = plugins/destinations/table_copy.inc
files[] = plugins/destinations/menu.inc
files[] = plugins/destinations/menu_links.inc
files[] = plugins/destinations/statistics.inc
files[] = plugins/destinations/variable.inc
files[] = plugins/sources/csv.inc
files[] = plugins/sources/db2.inc
files[] = plugins/sources/files.inc
files[] = plugins/sources/json.inc
files[] = plugins/sources/list.inc
files[] = plugins/sources/mongodb.inc
files[] = plugins/sources/multiitems.inc
files[] = plugins/sources/sql.inc
files[] = plugins/sources/sqlmap.inc
files[] = plugins/sources/mssql.inc
files[] = plugins/sources/oracle.inc
files[] = plugins/sources/spreadsheet.inc
files[] = plugins/sources/xml.inc
files[] = tests/import/options.test
files[] = tests/plugins/destinations/comment.test
@@ -46,9 +51,8 @@ files[] = tests/plugins/destinations/term.test
files[] = tests/plugins/destinations/user.test
files[] = tests/plugins/sources/xml.test
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2018-06-10
version = "7.x-2.11"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1528674486"
sites/all/modules/contrib/migrate/migrate/migrate.install
+246 -52
View File
@@ -9,6 +9,8 @@ function migrate_schema() {
$schema = array();
$schema['migrate_status'] = migrate_schema_status();
$schema['migrate_log'] = migrate_schema_log();
$schema['migrate_group'] = migrate_schema_group();
$schema['migrate_field_mapping'] = migrate_schema_field_mapping();
return $schema;
}
@@ -28,6 +30,12 @@ function migrate_schema_status() {
'not null' => TRUE,
'description' => 'Name of class to instantiate for this migration',
),
'group_name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Name of group containing migration',
),
'status' => array(
'type' => 'int',
'size' => 'tiny',
@@ -115,6 +123,74 @@ function migrate_schema_log() {
);
}
function migrate_schema_group() {
return array(
'description' => 'Information on migration groups',
'fields' => array(
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Unique machine name for a migration group',
),
'title' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Display name for a migration group',
),
'arguments' => array(
'type' => 'blob',
'not null' => FALSE,
'size' => 'big',
'serialize' => TRUE,
'description' => 'A serialized array of arguments to the migration group',
),
),
'primary key' => array('name'),
);
}
function migrate_schema_field_mapping() {
return array(
'description' => 'History of migration processes',
'fields' => array(
'fmid' => array(
'type' => 'serial',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Unique ID for the field mapping row',
),
'machine_name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Parent migration for the field mapping',
),
'destination_field' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Destination field for the field mapping',
),
'source_field' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Source field for the field mapping',
),
'options' => array(
'type' => 'blob',
'not null' => FALSE,
'size' => 'big',
'serialize' => TRUE,
'description' => 'A serialized MigrateFieldMapping object holding all options',
),
),
'primary key' => array('fmid'),
);
}
/**
* Implements hook_uninstall().
* Drop map/message tables, in case implementing classes did not.
@@ -137,25 +213,29 @@ function migrate_uninstall() {
->condition('module', 'migrate')
->execute();
}
// Remove variables
variable_del('migrate_disable_autoregistration');
variable_del('migrate_disabled_handlers');
variable_del('migrate_deprecation_warnings');
}
/**
* Add highwater mark
*/
function migrate_update_7001() {
$ret = array();
if (!db_field_exists('migrate_status', 'highwater')) {
db_add_field('migrate_status', 'highwater', array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => '',
'description' => 'Highwater mark for detecting updated content',
)
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => '',
'description' => 'Highwater mark for detecting updated content',
)
);
}
$ret[] = t('Added highwater column to migrate_status table');
$ret = t('Added highwater column to migrate_status table');
return $ret;
}
@@ -163,7 +243,6 @@ function migrate_update_7001() {
* Add last_imported field to all map tables
*/
function migrate_update_7002() {
$ret = array();
foreach (db_find_tables('migrate_map_%') as $tablename) {
if (!db_field_exists($tablename, 'last_imported')) {
db_add_field($tablename, 'last_imported', array(
@@ -175,7 +254,7 @@ function migrate_update_7002() {
));
}
}
$ret[] = t('Added last_imported column to all map tables');
$ret = t('Added last_imported column to all map tables');
return $ret;
}
@@ -183,7 +262,7 @@ function migrate_update_7002() {
* Add lastthroughput column to migrate_status
*/
function migrate_update_7003() {
$ret = array();
$ret = '';
if (!db_field_exists('migrate_status', 'lastthroughput')) {
db_add_field('migrate_status', 'lastthroughput', array(
'type' => 'int',
@@ -194,7 +273,7 @@ function migrate_update_7003() {
);
}
$ret[] = t('Added lastthroughput column to migrate_status table');
$ret = t('Added lastthroughput column to migrate_status table');
return $ret;
}
@@ -202,7 +281,7 @@ function migrate_update_7003() {
* Convert lastimported datetime field to lastimportedtime int field.
*/
function migrate_update_7004() {
$ret = array();
$ret = '';
if (!db_field_exists('migrate_status', 'lastimportedtime')) {
db_add_field('migrate_status', 'lastimportedtime', array(
'type' => 'int',
@@ -212,10 +291,10 @@ function migrate_update_7004() {
)
);
if (!db_field_exists('migrate_status', 'lastimported')) {
if (db_field_exists('migrate_status', 'lastimported')) {
$result = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name', 'lastimported'))
->execute();
->fields('ms', array('machine_name', 'lastimported'))
->execute();
foreach ($result as $row) {
$lastimportedtime = strtotime($row->lastimported);
db_update('migrate_status')
@@ -226,7 +305,7 @@ function migrate_update_7004() {
db_drop_field('migrate_status', 'lastimported');
$ret[] = t('Converted lastimported datetime field to lastimportedtime int field');
$ret .= "\n" . t('Converted lastimported datetime field to lastimportedtime int field');
}
}
return $ret;
@@ -236,11 +315,11 @@ function migrate_update_7004() {
* Add support for history logging
*/
function migrate_update_7005() {
$ret = array();
$ret = '';
if (!db_table_exists('migrate_log')) {
$ret[] = t('Create migrate_log table');
$ret .= "\n" . t('Create migrate_log table');
db_create_table('migrate_log', migrate_schema_log());
$ret[] = t('Remove historic columns from migrate_status table');
$ret .= "\n" . t('Remove historic columns from migrate_status table');
db_drop_field('migrate_status', 'lastthroughput');
db_drop_field('migrate_status', 'lastimportedtime');
}
@@ -252,21 +331,21 @@ function migrate_update_7005() {
* dependencies or sourceMigration() must be changed! See CHANGELOG.txt.
*/
function migrate_update_7006() {
$ret = array();
$ret = '';
if (!db_field_exists('migrate_status', 'class_name')) {
db_add_field('migrate_status', 'class_name', array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => '',
'description' => 'Name of class to instantiate for this migration',
)
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => '',
'description' => 'Name of class to instantiate for this migration',
)
);
db_query("UPDATE {migrate_status}
SET class_name = CONCAT(machine_name, 'Migration')
");
$ret[] = t('Added class_name column to migrate_status table');
$ret = t('Added class_name column to migrate_status table');
}
return $ret;
}
@@ -275,18 +354,18 @@ function migrate_update_7006() {
* Add arguments field to migrate_status table.
*/
function migrate_update_7007() {
$ret = array();
$ret = '';
if (!db_field_exists('migrate_status', 'arguments')) {
db_add_field('migrate_status', 'arguments', array(
'type' => 'blob',
'not null' => FALSE,
'size' => 'big',
'serialize' => TRUE,
'description' => 'A serialized array of arguments to the migration constructor',
)
'type' => 'blob',
'not null' => FALSE,
'size' => 'big',
'serialize' => TRUE,
'description' => 'A serialized array of arguments to the migration constructor',
)
);
$ret[] = t('Added arguments column to migrate_status table');
$ret = t('Added arguments column to migrate_status table');
}
return $ret;
}
@@ -298,10 +377,9 @@ function migrate_update_7008() {
// Updates can be run when the module is disabled, which would mean the
// call to migrate_migrations() will fail. Just bail in that case...
if (!module_exists('migrate')) {
throw new DrupalUpdateException(t('This update cannot be run while the Migrate ' .
'module is disabled - you must enable Migrate to run this update.'));
throw new DrupalUpdateException(t('This update cannot be run while the Migrate module is disabled - you must enable Migrate to run this update.'));
}
$ret = array();
$ret = '';
foreach (migrate_migrations() as $migration) {
if (is_a($migration, 'Migration')) {
// Since we're now tracking failed/ignored rows in the map table,
@@ -316,18 +394,18 @@ function migrate_update_7008() {
$field = 'destid' . $index++;
$field_schema['not null'] = FALSE;
$map_connection->schema()->changeField($map_table, $field, $field,
$field_schema);
$ret[] = t('Changed !table.!field to be non-null',
array('!table' => $map_table, '!field' => $field));
$field_schema);
$ret .= "\n" . t('Changed !table.!field to be non-null',
array('!table' => $map_table, '!field' => $field));
}
// Add any existing failures to the map table
$msg_table = $map->getMessageTable();
$msg_marked = FALSE;
$result = $map_connection->select($msg_table, 'msg')
->fields('msg')
->condition('level', Migration::MESSAGE_INFORMATIONAL, '<>')
->execute();
->fields('msg')
->condition('level', Migration::MESSAGE_INFORMATIONAL, '<>')
->execute();
foreach ($result as $row) {
$keys = array();
$index = 1;
@@ -337,13 +415,13 @@ function migrate_update_7008() {
}
}
$map_connection->merge($map_table)
->key($keys)
->fields(array('needs_update' => MigrateMap::STATUS_FAILED))
->execute();
->key($keys)
->fields(array('needs_update' => MigrateMap::STATUS_FAILED))
->execute();
$msg_marked = TRUE;
}
if ($msg_marked) {
$ret[] = t('Marked failures in !table', array('!table' => $map_table));
$ret .= "\n" . t('Marked failures in !table', array('!table' => $map_table));
}
}
}
@@ -363,10 +441,11 @@ function migrate_update_7201() {
}
/**
* Add rollback_action field to all map tables
* Add rollback_action field to all map tables in the Drupal database.
*/
function migrate_update_7202() {
$ret = array();
// Note this won't catch any prefixed tables, or any stored in the source
// database - ensureTables() will take care of those.
foreach (db_find_tables('migrate_map_%') as $tablename) {
if (!db_field_exists($tablename, 'rollback_action')) {
db_add_field($tablename, 'rollback_action', array(
@@ -379,6 +458,121 @@ function migrate_update_7202() {
));
}
}
$ret[] = t('Added rollback_action column to all map tables');
$ret = t('Added rollback_action column to all map tables');
return $ret;
}
/**
* Add database tracking of per-group info.
*/
function migrate_update_7203() {
$ret = '';
if (!db_table_exists('migrate_group')) {
$ret .= t('Create migrate_group table') . "\n";
db_create_table('migrate_group', migrate_schema_group());
}
if (!db_field_exists('migrate_status', 'group_name')) {
$ret .= t('Add group relationship to migrate_status table') . "\n";
db_add_field('migrate_status', 'group_name', array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => 'default',
'description' => 'Name of group containing migration',
)
);
// Populate each migration's group_name field
$groups = array();
foreach (migrate_migrations() as $machine_name => $migration) {
$group_name = $migration->getGroup()->getName();
if (empty($group_name)) {
$group_name = 'default';
}
$groups[$group_name] = $group_name;
db_update('migrate_status')
->fields(array('group_name' => $group_name))
->condition('machine_name', $machine_name)
->execute();
}
// Populate the migrate_group table
foreach ($groups as $group_name) {
$title = db_select('migrate_group', 'mg')
->fields('mg', array('title'))
->condition('name', $group_name)
->execute()
->fetchField();
if (!$title) {
db_insert('migrate_group')
->fields(array(
'name' => $group_name,
'title' => $group_name,
'arguments' => serialize(array()),
))
->execute();
}
}
}
return $ret;
}
/**
* Add database tracking of field mappings.
*/
function migrate_update_7204() {
$ret = '';
if (!db_table_exists('migrate_field_mapping')) {
$ret = t('Create migrate_field_mapping table');
db_create_table('migrate_field_mapping', migrate_schema_field_mapping());
}
return $ret;
}
/**
* Remove obsolete autoregistration disablement.
*/
function migrate_update_7205() {
variable_del('migrate_disable_autoregistration');
}
/**
* Replace three-column PK with a simple serial.
*/
function migrate_update_7206() {
if (!db_field_exists('migrate_field_mapping', 'fmid')) {
db_drop_primary_key('migrate_field_mapping');
db_add_field('migrate_field_mapping', 'fmid',
array(
'type' => 'serial',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Unique ID for the field mapping row',
),
array(
'primary key' => array('fmid'),
)
);
}
}
/**
* Make sure we remove an empty 'default' group created by the previous updates.
*/
function migrate_update_7207() {
$rows = db_select('migrate_group', 'mg')
->fields('mg', array('name'))
->condition('name', 'default')
->execute()
->rowCount();
if ($rows > 0) {
$rows = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name'))
->condition('group_name', 'default')
->execute()
->rowCount();
if ($rows == 0) {
db_delete('migrate_group')
->condition('name', 'default')
->execute();
}
}
}
sites/all/modules/contrib/migrate/migrate/migrate.mail.inc
+2
View File
@@ -6,6 +6,7 @@
*/
class MigrateMailIgnore extends DefaultMailSystem {
/**
* On an email request, do nothing and say we did.
*
@@ -14,6 +15,7 @@ class MigrateMailIgnore extends DefaultMailSystem {
*
* @param $message
* A message array, as described in hook_mail_alter().
*
* @return
* TRUE if the mail was successfully accepted, otherwise FALSE.
*/
sites/all/modules/contrib/migrate/migrate/migrate.module
+239 -217
View File
@@ -15,16 +15,46 @@
define('MIGRATE_API_VERSION', 2);
define('MIGRATE_ACCESS_BASIC', 'migration information');
define('MIGRATE_ACCESS_ADVANCED', 'advanced migration information');
/**
* Implements hook_help().
*/
function migrate_help($path, $arg) {
switch ($path) {
case 'admin/help#migrate':
$output = '';
$output .= '<h3>' . t('About') . '</h3>';
$output .= '<p>' . t('The Migrate module provides a flexible framework for migrating content into Drupal from other sources (e.g., when converting a web site from another CMS to Drupal). Out-of-the-box, support for creating Drupal nodes, taxonomy terms, comments, and users are included. Plugins permit migration of other types of content. For more information, see the online documentation for the <a href="@handbook">Migrate module</a>.', array('@handbook' => 'http://drupal.org/migrate')) . '</p>';
$output .= '<h3>' . t('Uses') . '</h3>';
$output .= '<dl>';
$output .= '<dt>' . t('Getting Started') . '</dt>';
$output .= '<dd>' . t('To get started, enable the migrate_example module and browse to admin/content/migrate to see its dashboard. The code for this migration is in migrate_example/beer.inc (advanced examples are in wine.inc). Mimic that file in order to specify your own migrations.') . '</dd>';
$output .= '<dt>' . t('Migrate Support') . '</dt>';
$output .= '<dd>' . t('The Migrate module itself has support for migration into core objects. Some built-in support for migration involving contrib modules is in the migrate_extras module.') . '</dd>';
$output .= '<dt>' . t('Migrate Extended Support') . '</dt>';
$output .= '<dd>' . t('The best place to implement migration support for a contributed module is in that module, not in the Migrate or Migrate Extras modules. That way, the migration support is always self-consistent with the current module implementation - it is not practical for the migrate modules to keep up with changes to all other contrib modules.') . '</dd>';
$output .= '</dl>';
return $output;
}
}
/**
* Retrieve a list of all active migrations, ordered by dependencies. To be
* recognized, a class must be non-abstract, and derived from MigrationBase.
*
* @param $reset
* If TRUE, the static cache of migrations will be flushed before attempting to
* reinstantiate all active migrations. This can be important for script runs
* where migration classes may be dynamically registered.
*
* @return
* Array of migration objects, keyed by the machine name.
*/
function migrate_migrations() {
function migrate_migrations($reset = NULL) {
static $migrations = array();
if (!empty($migrations)) {
if (!empty($migrations) && empty($reset)) {
return $migrations;
}
@@ -32,70 +62,47 @@ function migrate_migrations() {
// make sure any dynamic migrations defined in hook_migrate_api() get registered.
migrate_get_module_apis(TRUE);
$dependencies_list = array();
$dependent_migrations = array();
$required_migrations = array();
$result = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name', 'class_name', 'arguments'))
->execute();
->fields('ms', array('machine_name', 'class_name'))
->execute();
foreach ($result as $row) {
if (class_exists($row->class_name)) {
$reflect = new ReflectionClass($row->class_name);
if (!$reflect->isAbstract() && $reflect->isSubclassOf('MigrationBase')) {
$arguments = unserialize($row->arguments);
if (!$arguments || !is_array($arguments)) {
$arguments = array();
}
$migration = MigrationBase::getInstance($row->machine_name,
$row->class_name, $arguments);
$dependencies = $migration->getDependencies();
if (count($dependencies) > 0) {
// Set classes with dependencies aside for reordering
$dependent_migrations[$row->machine_name] = $migration;
$required_migrations += $dependencies;
}
else {
// No dependencies, just add
$migrations[$row->machine_name] = $migration;
$migration = MigrationBase::getInstance($row->machine_name);
if ($migration) {
$dependencies = $migration->getDependencies();
$dependencies_list[$row->machine_name] = $dependencies;
if (count($dependencies) > 0) {
// Set classes with dependencies aside for reordering
$dependent_migrations[$row->machine_name] = $migration;
$required_migrations += $dependencies;
}
else {
// No dependencies, just add
$migrations[$row->machine_name] = $migration;
}
}
}
else {
MigrationBase::displayMessage(t('Class !class is no longer a valid concrete migration class',
array('!class' => $row->class_name)));
array('!class' => $row->class_name)));
}
}
else {
MigrationBase::displayMessage(t('Class !class no longer exists',
array('!class' => $row->class_name)));
MigrationBase::displayMessage(t('Class !class could not be found',
array('!class' => $row->class_name)));
}
}
// Scan modules with dependencies - we'll take 20 passes at it before
// giving up
// TODO: Can we share code with _migrate_class_list()?
$iterations = 0;
while (count($dependent_migrations) > 0) {
if ($iterations++ > 20) {
$migration_names = implode(',', array_keys($dependent_migrations));
throw new MigrateException(t('Failure to sort migration list - most likely due ' .
'to circular dependencies involving !migration_names',
array('!migration_names' => $migration_names)));
}
foreach ($dependent_migrations as $name => $migration) {
$ready = TRUE;
// Scan all the dependencies for this class and make sure they're all
// in the final list
foreach ($migration->getDependencies() as $dependency) {
if (!isset($migrations[$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this class to the final list
$migrations[$name] = $migration;
unset($dependent_migrations[$name]);
}
$ordered_migrations = migrate_order_dependencies($dependencies_list);
foreach ($ordered_migrations as $name) {
if (!isset($migrations[$name])) {
$migrations[$name] = $dependent_migrations[$name];
}
}
@@ -108,11 +115,19 @@ function migrate_migrations() {
$final_migrations[$name] = array();
}
// Fill in the grouped list
// Fill in the grouped list.
foreach ($migrations as $machine_name => $migration) {
$final_migrations[$migration->getGroup()->getName()][$machine_name] = $migration;
if (!method_exists($migration, 'getGroup')) {
MigrationBase::displayMessage(t('Migration !machine_name is not a valid Migration dependency.', array(
'!machine_name' => $machine_name,
)));
}
else {
$final_migrations[$migration->getGroup()
->getName()][$machine_name] = $migration;
}
}
// Then flatten the list
// Flatten the grouped list.
$migrations = array();
foreach ($final_migrations as $group_name => $group_migrations) {
foreach ($group_migrations as $machine_name => $migration) {
@@ -123,59 +138,6 @@ function migrate_migrations() {
return $migrations;
}
/**
* On request, scan the Drupal code registry for any new migration classes
* for us to register in migrate_status.
*/
function migrate_autoregister() {
// Make sure the registry is up-to-date on all available classes.
require_once 'includes/registry.inc';
_registry_update();
// Get list of modules implementing Migrate API
$modules = array_keys(migrate_get_module_apis(TRUE));
// Get list of classes we already know about
$existing_classes = db_select('migrate_status', 'ms')
->fields('ms', array('class_name'))
->execute()
->fetchCol();
// Discover class names registered with Drupal by modules implementing our API
$result = db_select('registry', 'r')
->fields('r', array('name'))
->condition('type', 'class')
->condition('module', $modules, 'IN')
->condition('filename', '%.test', 'NOT LIKE')
->execute();
foreach ($result as $record) {
$class_name = $record->name;
// If we already know about this class, skip it
if (isset($existing_classes[$class_name])) {
continue;
}
// Validate it's an implemented subclass of the parent class
// Ignore errors
try {
$class = new ReflectionClass($class_name);
}
catch (Exception $e) {
continue;
}
if (!$class->isAbstract() && $class->isSubclassOf('MigrationBase')) {
// Verify that it's not a dynamic class (the implementor will be responsible
// for registering those).
$dynamic = call_user_func(array($class_name, 'isDynamic'));
if (!$dynamic) {
// OK, this is a new non-dynamic migration class, register it
MigrationBase::registerMigration($class_name);
}
}
}
}
/**
* Invoke any available handlers attached to a given destination type.
* If any handlers have dependencies defined, they will be invoked after
@@ -185,8 +147,8 @@ function migrate_autoregister() {
* Destination type ('Node', 'User', etc.) - generally the same string as
* the destination class name without the MigrateDestination prefix.
* @param $method
* Method name such as 'prepare' (called at the beginning of an import operation)
* or 'complete' (called at the end of an import operation).
* Method name such as 'prepare' (called at the beginning of an import
* operation) or 'complete' (called at the end of an import operation).
* @param ...
* Parameters to be passed to the handler.
*/
@@ -199,7 +161,7 @@ function migrate_handler_invoke_all($destination, $method) {
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
foreach ($class_list as $class_name => $handler) {
if (!in_array($class_name, $disabled) && $handler->handlesType($destination)
&& method_exists($handler, $method)) {
&& method_exists($handler, $method)) {
migrate_instrument_start($class_name . '->' . $method);
$result = call_user_func_array(array($handler, $method), $args);
migrate_instrument_stop($class_name . '->' . $method);
@@ -216,8 +178,6 @@ function migrate_handler_invoke_all($destination, $method) {
/**
* Invoke any available handlers attached to a given field type.
* If any handlers have dependencies defined, they will be invoked after
* the specified handlers.
*
* @param $entity
* The object we are building up before calling example_save().
@@ -231,18 +191,21 @@ function migrate_handler_invoke_all($destination, $method) {
* @param $method
* Handler method to call (defaults to prepare()).
*/
function migrate_field_handler_invoke_all($entity, array $field_info, array $instance,
array $values, $method = 'prepare') {
function migrate_field_handler_invoke_all($entity, array $field_info,
array $instance, array $values, $method = 'prepare') {
$return = array();
$type = $field_info['type'];
$class_list = _migrate_class_list('MigrateFieldHandler');
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
$disabled = unserialize(variable_get('migrate_disabled_handlers',
serialize(array())));
$handler_called = FALSE;
foreach ($class_list as $class_name => $handler) {
if (!in_array($class_name, $disabled) && $handler->handlesType($type)
&& method_exists($handler, $method)) {
&& method_exists($handler, $method)) {
migrate_instrument_start($class_name . '->' . $method);
$result = call_user_func_array(array($handler, $method),
array($entity, $field_info, $instance, $values));
$handler_called = TRUE;
migrate_instrument_stop($class_name . '->' . $method);
if (isset($result) && is_array($result)) {
$return = array_merge_recursive($return, $result);
@@ -252,27 +215,42 @@ function migrate_field_handler_invoke_all($entity, array $field_info, array $ins
}
}
}
if (!$handler_called && $method == 'prepare') {
$handler = new MigrateDefaultFieldHandler();
migrate_instrument_start('MigrateDefaultFieldHandler->prepare');
$result = call_user_func_array(array($handler, 'prepare'),
array($entity, $field_info, $instance, $values));
migrate_instrument_stop('MigrateDefaultFieldHandler->prepare');
if (isset($result) && is_array($result)) {
$return = array_merge_recursive($return, $result);
}
elseif (isset($result)) {
$return[] = $result;
}
}
return $return;
}
/**
* For a given parent class, identify and instantiate objects for any non-abstract
* classes derived from the parent, returning an array of the objects indexed by
* class name. The array will be ordered such that any classes with dependencies
* are listed after the classes they are dependent on.
* For a given parent class, identify and instantiate objects for any
* non-abstract classes derived from the parent, returning an array of the
* objects indexed by class name. The array will be ordered such that any
* classes with dependencies are listed after the classes they are dependent
* on.
*
* @param $parent_class
* Name of a class from which results will be derived.
*
* @return
* Array of objects, keyed by the class name.
*/
function _migrate_class_list($parent_class) {
// Get info on modules implementing Migrate API
static $module_info;
if (!isset($modules)) {
if (!isset($module_info)) {
$module_info = migrate_get_module_apis();
}
$modules = array_keys($module_info);
static $class_lists = array();
if (!isset($class_lists[$parent_class])) {
@@ -291,90 +269,6 @@ function _migrate_class_list($parent_class) {
}
}
}
// Avoid scrounging the registry for handler classes if possible.
if (variable_get('migrate_disable_autoregistration', FALSE)) {
return $class_lists[$parent_class];
}
$dependent_classes = array();
$required_classes = array();
// Discover class names registered with Drupal by modules implementing our API
$result = db_select('registry', 'r')
->fields('r', array('name'))
->condition('type', 'class')
->condition('module', $modules, 'IN')
->condition('filename', '%.test', 'NOT LIKE')
->execute();
foreach ($result as $record) {
// Validate it's an implemented subclass of the parent class
// We can get funky errors here, ignore them (and the class that caused them)
try {
$class = new ReflectionClass($record->name);
}
catch (Exception $e) {
continue;
}
if (!$class->isAbstract() && $class->isSubclassOf($parent_class)) {
// If the constructor has required parameters, this may fail. We will
// silently ignore - it is up to the implementor of such a class to
// instantiate it in hook_migrations_alter().
try {
$object = new $record->name;
}
catch (Exception $e) {
unset($object);
}
if (isset($object)) {
$dependencies = $object->getDependencies();
if (count($dependencies) > 0) {
// Set classes with dependencies aside for reordering
$dependent_classes[$record->name] = $object;
$required_classes += $dependencies;
}
else {
// No dependencies, just add
$class_lists[$parent_class][$record->name] = $object;
}
}
}
}
// Validate that each depended-on class at least exists
foreach ($required_classes as $class_name) {
if ((!isset($dependent_classes[$class_name])) && !isset($class_lists[$parent_class][$class_name])) {
throw new MigrateException(t('Dependency on non-existent class !class - make sure ' .
'you have added the file defining !class to the .info file.',
array('!class' => $class_name)));
}
}
// Scan modules with dependencies - we'll take 20 passes at it before
// giving up
$iterations = 0;
while (count($dependent_classes) > 0) {
if ($iterations++ > 20) {
$class_names = implode(',', array_keys($dependent_classes));
throw new MigrateException(t('Failure to sort class list - most likely due ' .
'to circular dependencies involving !class_names.',
array('!class_names' => $class_names)));
}
foreach ($dependent_classes as $name => $object) {
$ready = TRUE;
// Scan all the dependencies for this class and make sure they're all
// in the final list
foreach ($object->getDependencies() as $dependency) {
if (!isset($class_lists[$parent_class][$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this class to the final list
$class_lists[$parent_class][$name] = $object;
unset($dependent_classes[$name]);
}
}
}
}
return $class_lists[$parent_class];
}
@@ -387,9 +281,26 @@ function migrate_hook_info() {
$hooks['migrate_api'] = array(
'group' => 'migrate',
);
$hooks['migrate_api_alter'] = array(
'group' => 'migrate',
);
return $hooks;
}
/**
* Implementation of hook_permission().
*/
function migrate_permission() {
return array(
MIGRATE_ACCESS_BASIC => array(
'title' => t('Access to basic migration information'),
),
MIGRATE_ACCESS_ADVANCED => array(
'title' => t('Access to advanced migration information'),
),
);
}
/**
* Get a list of modules that support the current migrate API.
*/
@@ -405,26 +316,108 @@ function migrate_get_module_apis($reset = FALSE) {
$info = $function();
if (isset($info['api']) && $info['api'] == MIGRATE_API_VERSION) {
$cache[$module] = $info;
// Register any migrations defined via the hook.
if (isset($info['migrations']) && is_array($info['migrations'])) {
foreach ($info['migrations'] as $machine_name => $arguments) {
MigrationBase::registerMigration($arguments['class_name'],
$machine_name, $arguments);
}
}
}
else {
drupal_set_message(t('%function supports Migrate API version %modversion,
Migrate module API version is %version - migration support not loaded.',
array('%function' => $function, '%modversion' => $info['api'],
'%version' => MIGRATE_API_VERSION)));
array(
'%function' => $function,
'%modversion' => $info['api'],
'%version' => MIGRATE_API_VERSION,
)));
}
}
// Allow modules to alter the migration information.
drupal_alter('migrate_api', $cache);
}
return $cache;
}
/**
* Register any migrations defined in hook_migrate_api().
*
* @param array $machine_names
* If populated, only (re)register the specified migrations.
*/
function migrate_static_registration($machine_names = array()) {
$module_info = migrate_get_module_apis(TRUE);
foreach ($module_info as $module => $info) {
// Register any groups defined via the hook.
if (isset($info['groups']) && is_array($info['groups'])) {
foreach ($info['groups'] as $name => $arguments) {
$title = $arguments['title'];
unset($arguments['title']);
MigrateGroup::register($name, $title, $arguments);
}
}
// Register any migrations defined via the hook.
if (isset($info['migrations']) && is_array($info['migrations'])) {
foreach ($info['migrations'] as $machine_name => $arguments) {
// If we have an explicit list to register, skip any not in the list.
if (!empty($machine_names) && !in_array($machine_name, $machine_names)) {
continue;
}
$class_name = $arguments['class_name'];
unset($arguments['class_name']);
// Call the right registerMigration implementation. Note that this means
// that classes that override registerMigration() must always call it
// directly, they cannot register those classes by defining them in
// hook_migrate_api() and expect their extension to be called.
if (is_subclass_of($class_name, 'Migration')) {
Migration::registerMigration($class_name, $machine_name, $arguments);
}
else {
MigrationBase::registerMigration($class_name, $machine_name, $arguments);
}
}
}
}
}
/**
* Do a topological sort on our dependencies graph.
*/
function migrate_order_dependencies($dependencies) {
$visited = array();
$list = array();
foreach (array_keys($dependencies) as $name) {
$visited[$name] = FALSE;
}
foreach (array_keys($dependencies) as $name) {
migrate_visit_dependent($dependencies, $name, $list, $visited);
}
return $list;
}
/**
* Depth-first search for independent migrations.
*/
function migrate_visit_dependent($dependencies, $name, &$list, &$visited) {
if ($visited[$name]) {
if ($list[$name]) {
return;
}
else {
throw new MigrateException(t('Failure to sort migration list due to circular dependencies involving %name.', array('%name' => $name)));
}
}
$visited[$name] = TRUE;
if (isset($dependencies[$name])) {
foreach ($dependencies[$name] as $dependent) {
migrate_visit_dependent($dependencies, $dependent, $list, $visited);
}
}
$list[$name] = $name;
}
/**
* Implements hook_watchdog().
* Find the migration that is currently running and notify it.
@@ -483,6 +476,7 @@ function migrate_memory_start($name) {
*
* @param name
* The name of the memory measurement.
*
* @return
* The change in bytes since the last start.
*/
@@ -506,9 +500,11 @@ function migrate_memory_read($name) {
*
* @param name
* The name of the memory measurement.
*
* @return
* A memory array. The array contains the number of times the memory has been
* started and stopped (count) and the accumulated memory difference value in bytes.
* started and stopped (count) and the accumulated memory difference value in
* bytes.
*/
function migrate_memory_stop($name) {
global $_migrate_memory;
@@ -531,9 +527,9 @@ function migrate_memory_stop($name) {
}
/**
* Start measuring time and (optionally) memory consumption over a section of code.
* Note that the memory consumption measurement is generally not useful in
* lower areas of the code, where data is being generated that will be freed
* Start measuring time and (optionally) memory consumption over a section of
* code. Note that the memory consumption measurement is generally not useful
* in lower areas of the code, where data is being generated that will be freed
* by the next call to the same area. For example, measuring the memory
* consumption of db_query is not going to be helpful.
*
@@ -569,7 +565,8 @@ function migrate_instrument_stop($name) {
}
/**
* Call hook_migrate_overview for overall documentation on implemented migrations.
* Call hook_migrate_overview for overall documentation on implemented
* migrations.
*/
function migrate_overview() {
$overview = '';
@@ -579,3 +576,28 @@ function migrate_overview() {
}
return $overview;
}
/**
* Implements hook_modules_enabled.
*/
function migrate_modules_enabled($modules) {
if (array_intersect($modules, module_implements('migrate_api'))) {
migrate_static_registration();
}
}
/**
* Implements hook_module_implements_alter().
*/
function migrate_module_implements_alter(&$implementation, $hook) {
// Ensure that the Migration class exists, as different bootstrap phases may
// not have included migration.inc yet.
if (class_exists('Migration') && $migration = Migration::currentMigration()) {
$disable_hooks = $migration->getDisableHooks();
if (isset($disable_hooks[$hook])) {
foreach ($disable_hooks[$hook] as $module) {
unset($implementation[$module]);
}
}
}
}
sites/all/modules/contrib/migrate/migrate/migrate_example/beer.inc
+295 -180
View File
@@ -16,34 +16,43 @@
* - Comments to be attached to the beer nodes are described in the source
* migrate_example_beer_comment table.
*
* We will use the Migrate API to import and transform this data and turn it into
* a working Drupal system.
* We will use the Migrate API to import and transform this data and turn it
* into a working Drupal site.
*/
/**
* To define a migration process from a set of source data to a particular
* kind of Drupal object (for example, a specific node type), you define
* a class derived from Migration. You must define a constructor to initialize
* your migration object. By default, your class name will be the "machine name"
* of the migration, by which you refer to it. Note that the machine name is
* case-sensitive.
* your migration object.
*
* For your classes to be instantiated so they can be used to import content,
* you must register them - look at migrate_example.migrate.inc to see how
* registration works. Right now, it's important to understand that each
* migration will have a unique "machine name", which is displayed in the UI
* and is used to reference the migration in drush commands.
*
* In any serious migration project, you will find there are some options
* which are common to the individual migrations you're implementing. You can
* define an abstract intermediate class derived from Migration, then derive your
* individual migrations from that, to share settings, utility functions, etc.
* define an abstract intermediate class derived from Migration, then derive
* your individual migrations from that, to share settings, utility functions,
* etc.
*/
abstract class BasicExampleMigration extends DynamicMigration {
public function __construct() {
// Always call the parent constructor first for basic setup
parent::__construct();
abstract class BasicExampleMigration extends Migration {
// A Migration constructor takes an array of arguments as its first parameter.
// The arguments must be passed through to the parent constructor.
public function __construct($arguments) {
parent::__construct($arguments);
// With migrate_ui enabled, migration pages will indicate people involved in
// the particular migration, with their role and contact info. We default the
// list in the shared class; it can be overridden for specific migrations.
$this->team = array(
new MigrateTeamMember('Liz Taster', 'ltaster@example.com', t('Product Owner')),
new MigrateTeamMember('Larry Brewer', 'lbrewer@example.com', t('Implementor')),
new MigrateTeamMember('Liz Taster', 'ltaster@example.com',
t('Product Owner')),
new MigrateTeamMember('Larry Brewer', 'lbrewer@example.com',
t('Implementor')),
);
// Individual mappings in a migration can be linked to a ticket or issue
@@ -62,114 +71,129 @@ abstract class BasicExampleMigration extends DynamicMigration {
* this will receive data that originated from the source and has been mapped
* by the Migration class, and create Drupal objects.
* $this->map - An instance of a class derived from MigrateMap, this will keep
* track of which source items have been imported and what destination objects
* they map to.
* Mappings - Use $this->addFieldMapping to tell the Migration class what source
* fields correspond to what destination fields, and additional information
* associated with the mappings.
* track of which source items have been imported and what destination
* objects they map to.
* Field mappings - Use $this->addFieldMapping to tell the Migration class what
* source fields correspond to what destination fields, and additional
* information associated with the mappings.
*/
class BeerTermMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
// Human-friendly description of your migration process. Be as detailed as you
// like.
$this->description = t('Migrate styles from the source database to taxonomy terms');
// Create a map object for tracking the relationships between source rows
// and their resulting Drupal objects. Usually, you'll use the MigrateSQLMap
// class, which uses database tables for tracking. Pass the machine name
// (BeerTerm) of this migration to use in generating map and message tables.
// And, pass schema definitions for the primary keys of the source and
// destination - we need to be explicit for our source, but the destination
// class knows its schema already.
$this->map = new MigrateSQLMap($this->machineName,
array(
'style' => array('type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Topic ID',
)
),
MigrateDestinationTerm::getKeySchema()
);
public function __construct($arguments) {
parent::__construct($arguments);
// Human-friendly description of your migration process. Be as detailed as
// you like.
$this->description =
t('Migrate styles from the source database to taxonomy terms');
// In this example, we're using tables that have been added to the existing
// Drupal database but which are not Drupal tables. You can examine the
// various tables (starting here with migrate_example_beer_topic) using a
// database browser like phpMyAdmin.
// database browser such as phpMyAdmin.
// First, we set up a query for this data. Note that by ordering on
// style_parent, we guarantee root terms are migrated first, so the
// parent_name mapping below will find that the parent exists.
$query = db_select('migrate_example_beer_topic', 'met')
->fields('met', array('style', 'details', 'style_parent', 'region', 'hoppiness'))
// This sort assures that parents are saved before children.
->orderBy('style_parent', 'ASC');
->fields('met', array(
'style',
'details',
'style_parent',
'region',
'hoppiness',
))
// This sort assures that parents are saved before children.
->orderBy('style_parent', 'ASC');
// Create a MigrateSource object, which manages retrieving the input data.
$this->source = new MigrateSourceSQL($query);
// Set up our destination - terms in the migrate_example_beer_styles vocabulary
$this->destination = new MigrateDestinationTerm('migrate_example_beer_styles');
// Set up our destination - terms in the migrate_example_beer_styles
// vocabulary (note that we pass the machine name of the vocabulary).
$this->destination =
new MigrateDestinationTerm('migrate_example_beer_styles');
// Create a map object for tracking the relationships between source rows
// and their resulting Drupal objects. We will use the MigrateSQLMap class,
// which uses database tables for tracking. Pass the machine name (BeerTerm)
// of this migration to use in generating map and message table names.
// And, pass schema definitions for the primary keys of the source and
// destination - we need to be explicit for our source, but the destination
// class knows its schema already.
$this->map = new MigrateSQLMap($this->machineName,
array(
'style' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Topic ID',
),
),
MigrateDestinationTerm::getKeySchema()
);
// Assign mappings TO destination fields FROM source fields. To discover
// the names used in these calls, use the drush commands
// drush migrate-fields-destination BeerTerm
// drush migrate-fields-source BeerTerm
// drush migrate-fields-destination BeerTerm
// drush migrate-fields-source BeerTerm
// or review the detail pages in the UI.
$this->addFieldMapping('name', 'style');
$this->addFieldMapping('description', 'details');
// Documenting your mappings makes it easier for the whole team to see
// exactly what the status is when developing a migration process.
$this->addFieldMapping('parent_name', 'style_parent')
->description(t('The incoming style_parent field is the name of the term parent'));
->description(t('The incoming style_parent field is the name of the term parent'));
// Mappings are assigned issue groups, by which they are grouped on the
// migration info page when the migrate_ui module is enabled. The default
// is 'Done', indicating active mappings which need no attention. A
// suggested practice is to use groups of:
// Do Not Migrate (or DNM) to indicate source fields which are not being used,
// or destination fields not to be populated by migration.
// Do Not Migrate (or DNM) to indicate source fields which are not being
// used, or destination fields not to be populated by migration.
// Client Issues to indicate input from the client is needed to determine
// how a given field is to be migrated.
// Implementor Issues to indicate that the client has provided all the
// necessary information, and now the implementor needs to complete the work.
// necessary information, and now the implementor needs to complete the
// work.
$this->addFieldMapping(NULL, 'hoppiness')
->description(t('This info will not be maintained in Drupal'))
->issueGroup(t('DNM'));
->description(t('This info will not be maintained in Drupal'))
->issueGroup(t('DNM'));
// Open mapping issues can be assigned priorities (the default is
// MigrateFieldMapping::ISSUE_PRIORITY_OK). If you're using an issue
// tracking system, and have defined issuePattern (see BasicExampleMigration
// above), you can specify a ticket/issue number in the system on the
// mapping and migrate_ui will link directory to it.
// mapping and migrate_ui will link directly to it.
$this->addFieldMapping(NULL, 'region')
->description('Will a field be added to the vocabulary for this?')
->issueGroup(t('Client Issues'))
->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_MEDIUM)
->issueNumber(770064);
->description('Will a field be added to the vocabulary for this?')
->issueGroup(t('Client Issues'))
// This priority wil cause the mapping to be highlighted in the UI.
->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_MEDIUM)
->issueNumber(770064);
// It is good practice to account for all source and destination fields
// explicitly - this makes sure that everyone understands exactly what is
// being migrated and what is not. Also, migrate_ui highlights unmapped
// fields, or mappings involving fields not in the source and destination,
// so if (for example) a new field is added to the destination field it's
// immediately visible, and you can find out if anything needs to be
// so if (for example) a new field is added to the destination typ it's
// immediately visible, and you can decide if anything needs to be
// migrated into it.
$this->addFieldMapping('format')
->issueGroup(t('DNM'));
->issueGroup(t('DNM'));
$this->addFieldMapping('weight')
->issueGroup(t('DNM'));
->issueGroup(t('DNM'));
$this->addFieldMapping('parent')
->issueGroup(t('DNM'));
->issueGroup(t('DNM'));
// We conditionally DNM these fields, so your field mappings will be clean
// whether or not you have path and or pathauto enabled
if (module_exists('path')) {
// whether or not you have path and/or pathauto enabled.
$destination_fields = $this->destination->fields();
if (isset($destination_fields['path'])) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
->issueGroup(t('DNM'));
if (isset($destination_fields['pathauto'])) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
->issueGroup(t('DNM'));
}
}
}
@@ -186,31 +210,44 @@ class BeerTermMigration extends BasicExampleMigration {
* transformations of the data.
*/
class BeerUserMigration extends BasicExampleMigration {
public function __construct() {
public function __construct($arguments) {
// The basic setup is similar to BeerTermMigraiton
parent::__construct();
parent::__construct($arguments);
$this->description = t('Beer Drinkers of the world');
$this->map = new MigrateSQLMap($this->machineName,
array('aid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Account ID.'
)
),
MigrateDestinationUser::getKeySchema()
);
$query = db_select('migrate_example_beer_account', 'mea')
->fields('mea', array('aid', 'status', 'posted', 'name', 'nickname', 'password', 'mail', 'sex', 'beers'));
->fields('mea', array(
'aid',
'status',
'posted',
'name',
'nickname',
'password',
'mail',
'sex',
'beers',
));
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationUser();
$this->map = new MigrateSQLMap($this->machineName,
array(
'aid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Account ID.',
),
),
MigrateDestinationUser::getKeySchema()
);
// One good way to organize your mappings is in three groups - mapped fields,
// unmapped source fields, and unmapped destination fields
// One good way to organize your mappings in the constructor is in three
// groups - mapped fields, unmapped source fields, and unmapped destination
// fields.
// Mapped fields
// This is a shortcut you can use when the source and destination field
// names are identical (i.e., the email address field is named 'mail' in
// names are identical (e.g., the email address field is named 'mail' in
// both the source table and in Drupal).
$this->addSimpleMappings(array('status', 'mail'));
@@ -222,9 +259,10 @@ class BeerUserMigration extends BasicExampleMigration {
// 'test_1'.
// dedupe() takes the Drupal table and column for determining uniqueness.
$this->addFieldMapping('name', 'name')
->dedupe('users', 'name');
->dedupe('users', 'name');
// The migrate module automatically converts date/time strings to UNIX timestamps.
// The migrate module automatically converts date/time strings to UNIX
// timestamps.
$this->addFieldMapping('created', 'posted');
$this->addFieldMapping('pass', 'password');
@@ -234,7 +272,7 @@ class BeerUserMigration extends BasicExampleMigration {
// value is provided in addition to a source field, the default value will
// be applied to any rows where the source field is empty or NULL.
$this->addFieldMapping('roles')
->defaultValue(DRUPAL_AUTHENTICATED_RID);
->defaultValue(DRUPAL_AUTHENTICATED_RID);
$this->addFieldMapping('field_migrate_example_gender', 'sex');
@@ -244,20 +282,34 @@ class BeerUserMigration extends BasicExampleMigration {
// user.
if (module_exists('node_reference')) {
$this->addFieldMapping('field_migrate_example_favbeers', 'beers')
->separator('|');
->separator('|');
}
else {
$this->addFieldMapping(NULL, 'beers')
->issueGroup(t('DNM'));
}
// Unmapped source fields
$this->addFieldMapping(NULL, 'nickname')
->issueGroup(t('DNM'));
->issueGroup(t('DNM'));
// Unmapped destination fields
// This is a shortcut you can use to mark several destination fields as DNM
// at once
$this->addUnmigratedDestinations(array('theme', 'signature', 'access', 'login',
'timezone', 'language', 'picture', 'is_new', 'signature_format', 'role_names',
'init', 'data'));
// at once.
$this->addUnmigratedDestinations(array(
'access',
'data',
'is_new',
'language',
'login',
'picture',
'role_names',
'signature',
'signature_format',
'theme',
'timezone',
));
// Oops, we made a typo - this should have been 'init'! If you have
// migrate_ui enabled, look at the BeerUser info page - you'll see that it
@@ -265,14 +317,15 @@ class BeerUserMigration extends BasicExampleMigration {
// list of destination fields", and also lists "1 unmapped" under Destination,
// where it highlights "init" as unmapped.
$this->addFieldMapping('int')
->issueGroup(t('DNM'));
->issueGroup(t('DNM'));
if (module_exists('path')) {
$destination_fields = $this->destination->fields();
if (isset($destination_fields['path'])) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
->issueGroup(t('DNM'));
if (isset($destination_fields['pathauto'])) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
->issueGroup(t('DNM'));
}
}
}
@@ -283,35 +336,29 @@ class BeerUserMigration extends BasicExampleMigration {
* and creates Drupal nodes of type 'Beer' as destination.
*/
class BeerNodeMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
public function __construct($arguments) {
parent::__construct($arguments);
$this->description = t('Beers of the world');
// You may optionally declare dependencies for your migration - other migrations
// which should run first. In this case, terms assigned to our nodes and
// the authors of the nodes should be migrated before the nodes themselves.
$this->dependencies = array('BeerTerm', 'BeerUser');
$this->map = new MigrateSQLMap($this->machineName,
array(
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID.',
'alias' => 'b',
)
),
MigrateDestinationNode::getKeySchema()
);
// We have a more complicated query. The Migration class fundamentally
// depends on taking a single source row and turning it into a single
// Drupal object, so how do we deal with zero or more terms attached to
// each node? One way (demonstrated for MySQL) is to pull them into a single
// each node? One way (valid for MySQL only) is to pull them into a single
// comma-separated list.
$query = db_select('migrate_example_beer_node', 'b')
->fields('b', array('bid', 'name', 'body', 'excerpt', 'aid', 'countries',
'image', 'image_alt', 'image_title', 'image_description'));
->fields('b', array(
'bid',
'name',
'body',
'excerpt',
'aid',
'countries',
'image',
'image_alt',
'image_title',
'image_description',
));
$query->leftJoin('migrate_example_beer_topic_node', 'tb', 'b.bid = tb.bid');
// Gives a single comma-separated list of related terms
$query->groupBy('tb.bid');
@@ -331,23 +378,26 @@ class BeerNodeMigration extends BasicExampleMigration {
// Set up our destination - nodes of type migrate_example_beer
$this->destination = new MigrateDestinationNode('migrate_example_beer');
$this->map = new MigrateSQLMap($this->machineName,
array(
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID.',
'alias' => 'b',
),
),
MigrateDestinationNode::getKeySchema()
);
// Mapped fields
$this->addFieldMapping('title', 'name')
->description(t('Mapping beer name in source to node title'));
->description(t('Mapping beer name in source to node title'));
$this->addFieldMapping('sticky')
->description(t('Should we default this to 0 or 1?'))
->issueGroup(t('Client questions'))
->issueNumber(765736)
->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_LOW);
// To maintain node identities between the old and new systems (i.e., have
// the same unique IDs), map the ID column from the old system to nid and
// set is_new to TRUE. This works only if we're importing into a system that
// has no existing nodes with the nids being imported.
$this->addFieldMapping('nid', 'bid')
->description(t('Preserve old beer ID as nid in Drupal'));
$this->addFieldMapping('is_new')
->defaultValue(TRUE);
->description(t('Should we default this to 0 or 1?'))
->issueGroup(t('Client questions'))
->issueNumber(765736)
->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_LOW);
// References to related objects (such as the author of the content) are
// most likely going to be identifiers from the source data, not Drupal
@@ -357,28 +407,40 @@ class BeerNodeMigration extends BasicExampleMigration {
// find a corresponding uid for the aid, the owner will be the administrative
// account.
$this->addFieldMapping('uid', 'aid')
->sourceMigration('BeerUser')
->defaultValue(1);
// Note this is the machine name of the user migration.
->sourceMigration('BeerUser')
->defaultValue(1);
// This is a multi-value text field
// This is a multi-value text field - in the source data the values are
// separated by |, so we tell migrate to split it by that character.
$this->addFieldMapping('field_migrate_example_country', 'countries')
->separator('|');
// These are related terms, which by default will be looked up by name
->separator('|');
// These are related terms, which by default will be looked up by name.
$this->addFieldMapping('migrate_example_beer_styles', 'terms')
->separator(',');
->separator(',');
// Some fields may have subfields such as text formats or summaries
// (equivalent to teasers in previous Drupal versions).
// These can be individually mapped as we see here.
// Some fields may have subfields such as text formats or summaries. These
// can be individually mapped as we see here.
$this->addFieldMapping('body', 'body');
$this->addFieldMapping('body:summary', 'excerpt');
// Copy an image file, write DB record to files table, and save in Field storage.
// We map the filename (relative to the source_dir below) to the field itself.
// File fields are more complex - the file needs to be copied, a Drupal
// file entity (file_managed table row) created, and the field populated.
// There are several different options involved. It's usually best to do
// migrate the files themselves in their own migration (see wine.inc for an
// example), but they can also be brought over through the field mapping.
// We map the filename (relative to the source_dir below) to the field
// itself.
$this->addFieldMapping('field_migrate_example_image', 'image');
// The file_class determines how the 'image' value is interpreted, and what
// other options are available. In this case, MigrateFileUri indicates that
// the 'image' value is a URI.
$this->addFieldMapping('field_migrate_example_image:file_class')
->defaultValue('MigrateFileUri');
// Here we specify the directory containing the source files.
$this->addFieldMapping('field_migrate_example_image:source_dir')
->defaultValue(drupal_get_path('module', 'migrate_example'));
->defaultValue(drupal_get_path('module', 'migrate_example'));
// And we map the alt and title values in the database to those on the image.
$this->addFieldMapping('field_migrate_example_image:alt', 'image_alt');
$this->addFieldMapping('field_migrate_example_image:title', 'image_title');
@@ -387,24 +449,44 @@ class BeerNodeMigration extends BasicExampleMigration {
$this->addUnmigratedSources(array('image_description'));
// Unmapped destination fields
$this->addUnmigratedDestinations(array('created', 'changed', 'status',
'promote', 'revision', 'language', 'revision_uid', 'log', 'tnid',
'body:format', 'body:language', 'migrate_example_beer_styles:source_type',
'migrate_example_beer_styles:create_term', 'field_migrate_example_image:destination_dir',
'field_migrate_example_image:language', 'field_migrate_example_image:file_replace',
'field_migrate_example_image:preserve_files', 'field_migrate_example_country:language', 'comment',
'field_migrate_example_image:file_class', 'field_migrate_example_image:destination_file'));
// Some conventions we use here: with a long list of fields to ignore, we
// arrange them alphabetically, one distinct field per line (although
// subfields of the same field may be grouped on the same line), and indent
// subfields to distinguish them from top-level fields.
$this->addUnmigratedDestinations(array(
'body:format',
'changed',
'comment',
'created',
'field_migrate_example_image:destination_dir',
'field_migrate_example_image:destination_file',
'field_migrate_example_image:file_replace',
'field_migrate_example_image:preserve_files',
'field_migrate_example_image:urlencode',
'is_new',
'language',
'log',
'migrate_example_beer_styles:source_type',
'migrate_example_beer_styles:create_term',
'promote',
'revision',
'revision_uid',
'status',
'tnid',
));
if (module_exists('path')) {
$destination_fields = $this->destination->fields();
if (isset($destination_fields['path'])) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
->issueGroup(t('DNM'));
if (isset($destination_fields['pathauto'])) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
->issueGroup(t('DNM'));
}
}
if (module_exists('statistics')) {
$this->addUnmigratedDestinations(array('totalcount', 'daycount', 'timestamp'));
$this->addUnmigratedDestinations(
array('totalcount', 'daycount', 'timestamp'));
}
}
}
@@ -414,46 +496,79 @@ class BeerNodeMigration extends BasicExampleMigration {
* Drupal comment objects.
*/
class BeerCommentMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
public function __construct($arguments) {
parent::__construct($arguments);
$this->description = 'Comments about beers';
$this->dependencies = array('BeerUser', 'BeerNode');
$query = db_select('migrate_example_beer_comment', 'mec')
->fields('mec', array(
'cid',
'cid_parent',
'name',
'mail',
'aid',
'body',
'bid',
'subject',
))
->orderBy('cid_parent', 'ASC');
$this->source = new MigrateSourceSQL($query);
// Note that the machine name passed for comment migrations is
// 'comment_node_' followed by the machine name of the node type these
// comments are attached to.
$this->destination =
new MigrateDestinationComment('comment_node_migrate_example_beer');
$this->map = new MigrateSQLMap($this->machineName,
array('cid' => array(
'type' => 'int',
'not null' => TRUE,
)
),
array(
'cid' => array(
'type' => 'int',
'not null' => TRUE,
),
),
MigrateDestinationComment::getKeySchema()
);
$query = db_select('migrate_example_beer_comment', 'mec')
->fields('mec', array('cid', 'cid_parent', 'name', 'mail', 'aid', 'body', 'bid', 'subject'))
->orderBy('cid_parent', 'ASC');
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationComment('comment_node_migrate_example_beer');
// Mapped fields
$this->addSimpleMappings(array('name', 'subject', 'mail'));
$this->addFieldMapping('status')
->defaultValue(COMMENT_PUBLISHED);
->defaultValue(COMMENT_PUBLISHED);
// We preserved bid => nid ids during BeerNode import so simple mapping suffices.
$this->addFieldMapping('nid', 'bid');
$this->addFieldMapping('nid', 'bid')
->sourceMigration('BeerNode');
$this->addFieldMapping('uid', 'aid')
->sourceMigration('BeerUser')
->defaultValue(0);
->sourceMigration('BeerUser')
->defaultValue(0);
$this->addFieldMapping('pid', 'cid_parent')
->sourceMigration('BeerComment')
->description('Parent comment.');
->sourceMigration('BeerComment')
->description('Parent comment.');
$this->addFieldMapping('comment_body', 'body');
// No unmapped source fields
// Unmapped destination fields
$this->addUnmigratedDestinations(array('hostname', 'created', 'changed',
'thread', 'homepage', 'language', 'comment_body:format', 'comment_body:language'));
$this->addUnmigratedDestinations(array(
'changed',
'comment_body:format',
'created',
'homepage',
'hostname',
'language',
'thread',
));
$destination_fields = $this->destination->fields();
if (isset($destination_fields['path'])) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (isset($destination_fields['pathauto'])) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
}
}
}
}
sites/all/modules/contrib/migrate/migrate/migrate_example/beer.install.inc
+202 -42
View File
@@ -47,22 +47,19 @@ function migrate_example_beer_uninstall() {
}
function migrate_example_beer_disable() {
Migration::deregisterMigration('BeerTerm');
Migration::deregisterMigration('BeerUser');
Migration::deregisterMigration('BeerNode');
Migration::deregisterMigration('BeerComment');
MigrateGroup::deregister('beer');
}
function migrate_example_beer_schema_node() {
return array(
'description' => 'Beers of the world.',
'fields' => array(
'bid' => array(
'bid' => array(
'type' => 'serial',
'not null' => TRUE,
'description' => 'Beer ID.',
),
'name' => array(
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
@@ -123,7 +120,7 @@ function migrate_example_beer_schema_topic() {
return array(
'description' => 'Categories',
'fields' => array(
'style' => array(
'style' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
@@ -160,12 +157,12 @@ function migrate_example_beer_schema_topic_node() {
return array(
'description' => 'Beers topic pairs.',
'fields' => array(
'bid' => array(
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID.',
),
'style' => array(
'style' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
@@ -180,12 +177,12 @@ function migrate_example_beer_schema_comment() {
return array(
'description' => 'Beers comments.',
'fields' => array(
'cid' => array(
'cid' => array(
'type' => 'serial',
'not null' => TRUE,
'description' => 'Comment ID.',
),
'bid' => array(
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID that is being commented upon',
@@ -233,12 +230,12 @@ function migrate_example_beer_schema_account() {
return array(
'description' => 'Beers accounts.',
'fields' => array(
'aid' => array(
'aid' => array(
'type' => 'serial',
//'not null' => TRUE,
'not null' => TRUE,
'description' => 'Account ID',
),
'status' => array(
'status' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Blocked_Allowed',
@@ -293,7 +290,7 @@ function migrate_example_beer_schema_legacy_urls() {
return array(
'description' => 'Stores legacy paths and destination ids for redirection.',
'fields' => array(
'id' => array(
'id' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Primary Key: ID.',
@@ -478,8 +475,8 @@ function migrate_example_beer_image() {
function migrate_example_beer_favs() {
if (!field_info_field('field_migrate_example_favbeers')) {
$field = array(
'field_name' => 'field_migrate_example_favbeers',
'type' => 'node_reference',
'field_name' => 'field_migrate_example_favbeers',
'type' => 'node_reference',
'cardinality' => -1,
'settings' => array(
'referenceable_types' => array('migrate_example_beer'),
@@ -509,9 +506,10 @@ function migrate_example_beer_gender() {
'field_name' => 'field_migrate_example_gender',
'type' => 'list_integer',
'settings' => array(
'allowed_values' =>
"0|Male\n" .
"1|Female\n",
'allowed_values' => array(
0 => 'Male',
1 => 'Female'
),
),
);
field_create_field($field);
@@ -560,7 +558,11 @@ function migrate_example_beer_country() {
function migrate_example_beer_content_type_delete() {
$bundle = 'migrate_example_beer';
$field_names = array('migrate_example_beer_styles', 'field_migrate_example_image', 'field_migrate_example_country');
$field_names = array(
'migrate_example_beer_styles',
'field_migrate_example_image',
'field_migrate_example_country',
);
foreach ($field_names as $field_name) {
$instance = field_info_instance('node', $field_name, $bundle);
field_delete_instance($instance);
@@ -581,15 +583,58 @@ function migrate_example_beer_content_type_delete() {
}
function migrate_example_beer_data_node() {
$fields = array('bid', 'name', 'body', 'excerpt', 'countries', 'aid', 'image',
'image_alt', 'image_title', 'image_description');
$fields = array(
'bid',
'name',
'body',
'excerpt',
'countries',
'aid',
'image',
'image_alt',
'image_title',
'image_description',
);
$query = db_insert('migrate_example_beer_node')
->fields($fields);
->fields($fields);
// Use high bid numbers to avoid overwriting an existing node id.
$data = array(
array(99999999, 'Heineken', 'Blab Blah Blah Green', 'Green', 'Netherlands|Belgium', 0, 'heineken.jpg', 'Heinekin alt', 'Heinekin title', 'Heinekin description'), // comes with migrate_example project.
array(99999998, 'Miller Lite', 'We love Miller Brewing', 'Tasteless', 'USA|Canada', 1, NULL, NULL, NULL, NULL),
array(99999997, 'Boddington', 'English occassionally get something right', 'A treat', 'United Kingdom', 1, NULL, NULL, NULL, NULL),
array(
99999999,
'Heineken',
'Blab Blah Blah Green',
'Green',
'Netherlands|Belgium',
0,
'heineken.jpg',
'Heinekin alt',
'Heinekin title',
'Heinekin description',
), // comes with migrate_example project.
array(
99999998,
'Miller Lite',
'We love Miller Brewing',
'Tasteless',
'USA|Canada',
1,
NULL,
NULL,
NULL,
NULL,
),
array(
99999997,
'Boddington',
'English occassionally get something right',
'A treat',
'United Kingdom',
1,
NULL,
NULL,
NULL,
NULL,
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
@@ -600,14 +645,59 @@ function migrate_example_beer_data_node() {
// Note that alice has duplicate username. Exercies dedupe() method.
// @TODO duplicate email also.
function migrate_example_beer_data_account() {
$fields = array('status', 'posted', 'name', 'nickname', 'password', 'mail', 'sex', 'beers');
$fields = array(
'status',
'posted',
'name',
'nickname',
'password',
'mail',
'sex',
'beers',
);
$query = db_insert('migrate_example_beer_account')
->fields($fields);
$data = array(
array(1, '2010-03-30 10:31:05', 'alice', 'alice hot pants', 'alicepass', 'alice@example.com', '1', '99999999|99999998|99999997'),
array(1, '2010-04-04 10:31:05', 'alice', 'alice dupe pants', 'alicepass', 'alice2@example.com', '1', '99999999|99999998|99999997'),
array(0, '2007-03-15 10:31:05', 'bob', 'rebob', 'bobpass', 'bob@example.com', '1', '99999999|99999997'),
array(1, '2004-02-29 10:31:05', 'charlie', 'charlie chocolate', 'mykids', 'charlie@example.com', '0', '99999999|99999998'),
array(
1,
'2010-03-30 10:31:05',
'alice',
'alice hot pants',
'alicepass',
'alice@example.com',
'1',
'99999999|99999998|99999997',
),
array(
1,
'2010-04-04 10:31:05',
'alice',
'alice dupe pants',
'alicepass',
'alice2@example.com',
'1',
'99999999|99999998|99999997',
),
array(
0,
'2007-03-15 10:31:05',
'bob',
'rebob',
'bobpass',
'bob@example.com',
'1',
'99999999|99999997',
),
array(
1,
'2004-02-29 10:31:05',
'charlie',
'charlie chocolate',
'mykids',
'charlie@example.com',
'0',
'99999999|99999998',
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
@@ -616,15 +706,55 @@ function migrate_example_beer_data_account() {
}
function migrate_example_beer_data_comment() {
$fields = array('bid', 'cid_parent', 'subject', 'body', 'name', 'mail', 'aid');
$fields = array(
'bid',
'cid_parent',
'subject',
'body',
'name',
'mail',
'aid',
);
$query = db_insert('migrate_example_beer_comment')
->fields($fields);
$data = array(
array(99999998, NULL, 'im first', 'hot body', 'alice', 'alice@example.com', 0),
array(99999998, NULL, 'im second', 'hot body', 'alice', 'alice@example.com', 0),
array(99999999, NULL, 'im parent', 'hot body', 'alice', 'alice@example.com', 0),
array(
99999998,
NULL,
'im first',
'hot body',
'alice',
'alice@example.com',
0,
),
array(
99999998,
NULL,
'im second',
'hot body',
'alice',
'alice@example.com',
0,
),
array(
99999999,
NULL,
'im parent',
'hot body',
'alice',
'alice@example.com',
0,
),
array(99999999, 1, 'im child', 'cold body', 'bob', NULL, 1),
array(99999999, 2, 'im grandchild', 'bitter body', 'charlie@example.com', NULL, 1),
array(
99999999,
2,
'im grandchild',
'bitter body',
'charlie@example.com',
NULL,
1,
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
@@ -639,7 +769,13 @@ function migrate_example_beer_data_topic() {
$data = array(
array('ale', 'traditional', NULL, 'Medieval British Isles', 'Medium'),
array('red ale', 'colorful', 'ale', NULL, NULL),
array('pilsner', 'refreshing', NULL, 'Pilsen, Bohemia (now Czech Republic)', 'Low'),
array(
'pilsner',
'refreshing',
NULL,
'Pilsen, Bohemia (now Czech Republic)',
'Low',
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
@@ -663,13 +799,37 @@ function migrate_example_beer_data_topic_node() {
}
function migrate_example_beer_data_urls() {
$fields = array('id', 'migration_name', 'source_id', 'source_uri', 'modificationdatetime');
$fields = array(
'id',
'migration_name',
'source_id',
'source_uri',
'modificationdatetime',
);
$query = db_insert('migrate_example_beer_legacy_urls')
->fields($fields);
$data = array(
array(1, 'BeerNode', 99999997, 'the_boddington/main', strtotime('2010-04-12 08:32:06')),
array(2, 'BeerNode', 99999998, 'Miller Lite taste', strtotime('2010-04-12 08:32:05')),
array(3, 'BeerNode', 99999999, 'green wonder', strtotime('2010-04-12 08:32:03')),
array(
1,
'BeerNode',
99999997,
'the_boddington/main',
strtotime('2010-04-12 08:32:06'),
),
array(
2,
'BeerNode',
99999998,
'Miller Lite taste',
strtotime('2010-04-12 08:32:05'),
),
array(
3,
'BeerNode',
99999999,
'green wonder',
strtotime('2010-04-12 08:32:03'),
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
sites/all/modules/contrib/migrate/migrate/migrate_example/example.table_copy.inc
+6 -4
View File
@@ -7,6 +7,7 @@
*/
class RoleTableMigration extends Migration {
public function __construct() {
parent::__construct();
$this->dependencies = array();
@@ -23,14 +24,15 @@ class RoleTableMigration extends Migration {
$this->destination = new MigrateDestinationTableCopy('role_copy', $destination_key);
$this->map = new MigrateSQLMap($this->machineName,
array('rid' => array(
array(
'rid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'alias' => 'r',
)
),
$destination_key
),
),
$destination_key
);
}
}
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example.info
+4 -6
View File
@@ -1,6 +1,6 @@
name = "Migrate Example"
description = "Example migration data."
package = "Development"
package = "Migration"
core = 7.x
dependencies[] = taxonomy
dependencies[] = image
@@ -12,16 +12,14 @@ dependencies[] = number
;node_reference is useful but not required
;dependencies[] = node_reference
files[] = migrate_example.module
files[] = beer.inc
files[] = wine.inc
; For testing table_copy plugin. Since is infrequently used, we comment it out.
; files[] = example.table_copy.inc
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2018-06-10
version = "7.x-2.11"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1528674486"
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example.install
+19 -18
View File
@@ -34,6 +34,7 @@ function migrate_example_install() {
);
$example_format = (object) $example_format;
filter_format_save($example_format);
migrate_static_registration();
}
function migrate_example_uninstall() {
@@ -62,8 +63,8 @@ function migrate_example_update_7001() {
);
$result = db_select('migrate_example_beer_legacy_urls', 'ms')
->fields('ms', array('machine_name', 'modificationdate'))
->execute();
->fields('ms', array('machine_name', 'modificationdate'))
->execute();
foreach ($result as $row) {
$modificationdatetime = strtotime($row->modificationdate);
db_update('migrate_example_beer_legacy_urls')
@@ -84,24 +85,24 @@ function migrate_example_update_7001() {
function migrate_example_update_7002() {
$ret = array();
db_add_field('migrate_example_beer_node', 'image_alt', array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image ALT',
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image ALT',
)
);
db_add_field('migrate_example_beer_node', 'image_title', array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image title',
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image title',
)
);
db_add_field('migrate_example_beer_node', 'image_description', array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image description',
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image description',
)
);
db_update('migrate_example_beer_node')
@@ -206,10 +207,10 @@ function migrate_example_update_7006() {
function migrate_example_update_7007() {
$ret = array();
db_add_field('migrate_example_wine_account', 'positions', array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Positions held',
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Positions held',
)
);
$query = db_update('migrate_example_wine_account')
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example.migrate.inc
+227 -28
View File
@@ -9,39 +9,238 @@
/*
* You must implement hook_migrate_api(), setting the API level to 2, if you are
* implementing any migration classes. As of Migrate 2.5, you should also
* register your migration and handler classes explicitly here - the former
* method of letting them get automatically registered on a cache clear will
* break in certain environments (see http://drupal.org/node/1778952).
* implementing any migration classes. If your migration application is static -
* that is, you know at implementation time exactly what migrations must be
* instantiated - then you should register your migrations here. If your
* application is more dynamic (for example, if selections in the UI determine
* exactly what migrations need to be instantiated), then you would register
* your migrations using registerMigration() - see migrate_example_baseball for
* more information.
*/
function migrate_example_migrate_api() {
// Usually field mappings are established by code in the migration constructor -
// a call to addFieldMapping(). They may also be passed as arguments when
// registering a migration - in this case, they are stored in the database
// and override any mappings for the same field in the code. To do this,
// construct the field mapping object and configure it similarly to when
// you call addFieldMapping, and pass your mappings as an array below.
$translate_mapping = new MigrateFieldMapping('translate', NULL);
$translate_mapping->defaultValue(0);
$ignore_mapping = new MigrateFieldMapping('migrate_example_beer_styles:ignore_case', NULL);
$ignore_mapping->defaultValue(1);
$api = array(
// Required - tells the Migrate module that you are implementing version 2
// of the Migrate API.
'api' => 2,
// Migrations can be organized into groups. The key used here will be the
// machine name of the group, which can be used in Drush:
// drush migrate-import --group=wine
// The title is a required argument which is displayed for the group in the
// UI. You may also have additional arguments for any other data which is
// common to all migrations in the group.
'groups' => array(
'beer' => array(
'title' => t('Beer Imports'),
),
'wine' => array(
'title' => t('Wine Imports'),
),
),
// Here we register the individual migrations. The keys (BeerTerm, BeerUser,
// etc.) are the machine names of the migrations, and the class_name
// argument is required. The group_name is optional (defaulting to 'default')
// but specifying it is a best practice.
'migrations' => array(
'BeerTerm' => array('class_name' => 'BeerTermMigration'),
'BeerUser' => array('class_name' => 'BeerUserMigration'),
'BeerNode' => array('class_name' => 'BeerNodeMigration'),
'BeerComment' => array('class_name' => 'BeerCommentMigration'),
'WinePrep' => array('class_name' => 'WinePrepMigration'),
'WineVariety' => array('class_name' => 'WineVarietyMigration'),
'WineRegion' => array('class_name' => 'WineRegionMigration'),
'WineBestWith' => array('class_name' => 'WineBestWithMigration'),
'WineFileCopy' => array('class_name' => 'WineFileCopyMigration'),
'WineFileBlob' => array('class_name' => 'WineFileBlobMigration'),
'WineRole' => array('class_name' => 'WineRoleMigration'),
'WineUser' => array('class_name' => 'WineUserMigration'),
'WineProducer' => array('class_name' => 'WineProducerMigration'),
'WineProducerXML' => array('class_name' => 'WineProducerXMLMigration'),
'WineProducerMultiXML' => array('class_name' => 'WineProducerMultiXMLMigration'),
'WineProducerXMLPull' => array('class_name' => 'WineProducerXMLPullMigration'),
'WineWine' => array('class_name' => 'WineWineMigration'),
'WineComment' => array('class_name' => 'WineCommentMigration'),
'WineTable' => array('class_name' => 'WineTableMigration'),
'WineFinish' => array('class_name' => 'WineFinishMigration'),
'WineUpdates' => array('class_name' => 'WineUpdatesMigration'),
'WineCommentUpdates' => array('class_name' => 'WineCommentUpdatesMigration'),
'WineVarietyUpdates' => array('class_name' => 'WineVarietyUpdatesMigration'),
'WineUserUpdates' => array('class_name' => 'WineUserUpdatesMigration'),
'BeerTerm' => array(
'class_name' => 'BeerTermMigration',
'group_name' => 'beer',
),
'BeerUser' => array(
'class_name' => 'BeerUserMigration',
'group_name' => 'beer',
),
'BeerNode' => array(
'class_name' => 'BeerNodeMigration',
'group_name' => 'beer',
// You may optionally declare dependencies for your migration - other
// migrations which should run first. In this case, terms assigned to our
// nodes and the authors of the nodes should be migrated before the nodes
// themselves.
'dependencies' => array(
'BeerTerm',
'BeerUser',
),
// Here is where we add field mappings which may override those
// specified in the group constructor.
'field_mappings' => array(
$translate_mapping,
$ignore_mapping,
),
),
'BeerComment' => array(
'class_name' => 'BeerCommentMigration',
'group_name' => 'beer',
'dependencies' => array(
'BeerUser',
'BeerNode',
),
),
'WinePrep' => array(
'class_name' => 'WinePrepMigration',
'group_name' => 'wine',
),
'WineVariety' => array(
'class_name' => 'WineVarietyMigration',
'group_name' => 'wine',
),
'WineRegion' => array(
'class_name' => 'WineRegionMigration',
'group_name' => 'wine',
),
'WineBestWith' => array(
'class_name' => 'WineBestWithMigration',
'group_name' => 'wine',
),
'WineFileCopy' => array(
'class_name' => 'WineFileCopyMigration',
'group_name' => 'wine',
'dependencies' => array('WinePrep'),
),
'WineFileBlob' => array(
'class_name' => 'WineFileBlobMigration',
'group_name' => 'wine',
'dependencies' => array('WinePrep'),
),
'WineRole' => array(
'class_name' => 'WineRoleMigration',
'group_name' => 'wine',
// TIP: Regular dependencies, besides enforcing (in the absence of
// --force) the run order of migrations, affect the sorting of
// migrations on display. You can use soft dependencies to affect just
// the display order when the migrations aren't technically required to
// run in a certain order. In this case, we want the role migration to
// appear after the file migrations.
'soft_dependencies' => array('WineFileCopy'),
),
'WineUser' => array(
'class_name' => 'WineUserMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineFileCopy',
'WineRole',
),
),
'WineProducer' => array(
'class_name' => 'WineProducerMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerXML' => array(
'class_name' => 'WineProducerXMLMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerNamespaceXML' => array(
'class_name' => 'WineProducerNamespaceXMLMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerMultiXML' => array(
'class_name' => 'WineProducerMultiXMLMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerMultiNamespaceXML' => array(
'class_name' => 'WineProducerMultiNamespaceXMLMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerXMLPull' => array(
'class_name' => 'WineProducerXMLPullMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerNamespaceXMLPull' => array(
'class_name' => 'WineProducerNamespaceXMLPullMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineWine' => array(
'class_name' => 'WineWineMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineVariety',
'WineBestWith',
'WineUser',
'WineProducer',
),
),
'WineComment' => array(
'class_name' => 'WineCommentMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineUser',
'WineWine',
),
),
'WineTable' => array(
'class_name' => 'WineTableMigration',
'group_name' => 'wine',
'soft_dependencies' => array('WineComment'),
),
'WineFinish' => array(
'class_name' => 'WineFinishMigration',
'group_name' => 'wine',
'dependencies' => array('WineComment'),
),
'WineUpdates' => array(
'class_name' => 'WineUpdatesMigration',
'group_name' => 'wine',
'dependencies' => array('WineWine'),
'soft_dependencies' => array('WineFinish'),
),
'WineCommentUpdates' => array(
'class_name' => 'WineCommentUpdatesMigration',
'group_name' => 'wine',
'dependencies' => array('WineComment'),
'soft_dependencies' => array('WineUpdates'),
),
'WineVarietyUpdates' => array(
'class_name' => 'WineVarietyUpdatesMigration',
'group_name' => 'wine',
'dependencies' => array('WineVariety'),
'soft_dependencies' => array('WineUpdates'),
),
'WineUserUpdates' => array(
'class_name' => 'WineUserUpdatesMigration',
'group_name' => 'wine',
'dependencies' => array('WineUser'),
'soft_dependencies' => array('WineUpdates'),
),
),
);
return $api;
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example.module
+4 -4
View File
@@ -2,13 +2,13 @@
/**
* @file
* THIS SPACE INTENTIONALLY LEFT BLANK.
* THIS FILE INTENTIONALLY LEFT BLANK.
*
* Yes, there is no code in the .module file. Migrate operates almost entirely
* through classes, and by adding any files containing class definitions to the
* .info file, those files are automatically included only when the classes they
* contain are referenced. The one non-class piece you need to implement is
* hook_migrate_api(), but because .migrate.inc is registered using hook_hook_info
* by defining your implementation of that hook in mymodule.migrate.inc, it is
* automatically invoked only when needed.
* hook_migrate_api(), but because .migrate.inc is registered using
* hook_hook_info, by defining your implementation of that hook in
* example.migrate.inc, it is automatically invoked only when needed.
*/
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.info
+4 -5
View File
@@ -8,12 +8,11 @@ features[field][] = "node-migrate_example_oracle-field_mainimage"
features[node][] = "migrate_example_oracle"
files[] = "migrate_example_oracle.migrate.inc"
name = "Migrate example - Oracle"
package = "Migrate Examples"
package = "Migration"
project = "migrate_example_oracle"
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2018-06-10
version = "7.x-2.11"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1528674486"
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.install
+30 -28
View File
@@ -123,7 +123,7 @@ function migrate_example_oracle_requirements($phase) {
}
$sample_setting =
'<pre>
'<pre>
$conf[\'oracle_db\'] = array(
\'username\' => \'Oracle_username\',
\'password\' => \'Oracle_password\',
@@ -134,7 +134,7 @@ $conf[\'oracle_db\'] = array(
$requirements['oracle_db'] = array('title' => $t('Oracle configuration'));
global $conf;
if (empty($conf['oracle_db']) || empty($conf['oracle_db']['username']) ||
empty($conf['oracle_db']['password']) || empty($conf['oracle_db']['connection_string'])) {
empty($conf['oracle_db']['password']) || empty($conf['oracle_db']['connection_string'])) {
$requirements['oracle_db']['description'] = $t('You must define $conf[\'oracle_db\']
(in your site\'s settings.php file) to point to the Oracle database where
you want test data to be stored: ' . $sample_setting);
@@ -150,8 +150,10 @@ $conf[\'oracle_db\'] = array(
$e = oci_error();
$requirements['oracle_connect']['description'] = $t('Could not connect to configured
Oracle database at !conn_string. Oracle error message: !message',
array('!conn_string' => $conf['oracle_db']['connection_string'],
'!message' => $e['message']));
array(
'!conn_string' => $conf['oracle_db']['connection_string'],
'!message' => $e['message'],
));
$requirements['oracle_connect']['severity'] = REQUIREMENT_ERROR;
break;
}
@@ -159,35 +161,35 @@ $conf[\'oracle_db\'] = array(
// Check for necessary privileges
$requirements['oracle_privs'] = array('title' => $t('Necessary Oracle privileges are assigned'));
$statement = oci_parse($connection, 'SELECT * FROM SESSION_PRIVS');
if (!$statement) {
$e = oci_error($connection);
$requirements['oracle_connect']['description'] = $e['message'];
$requirements['oracle_connect']['severity'] = REQUIREMENT_ERROR;
if (!$statement) {
$e = oci_error($connection);
$requirements['oracle_connect']['description'] = $e['message'];
$requirements['oracle_connect']['severity'] = REQUIREMENT_ERROR;
break;
}
$result = oci_execute($statement);
if (!$result) {
$e = oci_error($statement);
$requirements['oracle_connect']['description'] = $e['message'];
$requirements['oracle_connect']['severity'] = REQUIREMENT_ERROR;
break;
}
$ok = FALSE;
while ($row = oci_fetch_object($statement)) {
if ($row->PRIVILEGE == 'CREATE TABLE') {
$ok = TRUE;
break;
}
$result = oci_execute($statement);
if (!$result) {
$e = oci_error($statement);
$requirements['oracle_connect']['description'] = $e['message'];
$requirements['oracle_connect']['severity'] = REQUIREMENT_ERROR;
break;
}
$ok = FALSE;
while ($row = oci_fetch_object($statement)) {
if ($row->PRIVILEGE == 'CREATE TABLE') {
$ok = TRUE;
break;
}
}
}
if (!$ok) {
$requirements['oracle_privs']['description'] = $t('The specified
if (!$ok) {
$requirements['oracle_privs']['description'] = $t('The specified
username !username does not have the CREATE TABLE privilege. This privilege
is necessary to create test tables in the Oracle database.',
array('!username' => $conf['oracle_db']['username']));
$requirements['oracle_privs']['severity'] = REQUIREMENT_ERROR;
break;
}
array('!username' => $conf['oracle_db']['username']));
$requirements['oracle_privs']['severity'] = REQUIREMENT_ERROR;
break;
}
break;
case 'update':
break;
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.migrate.inc
+28 -18
View File
@@ -1,9 +1,10 @@
<?php
/**
* @file
* Examples and test fodder for migration from Oracle sources. To use this example
* (and to run the corresponding tests) you must define a connection to an Oracle database
* in your settings.php. E.g.,
* Examples and test fodder for migration from Oracle sources. To use this
* example
* (and to run the corresponding tests) you must define a connection to an
* Oracle database in your settings.php. E.g.,
*
* $conf['oracle_db'] = array(
* 'username' => 'DRUPAL',
@@ -11,17 +12,18 @@
* 'connection_string' => '//oracledb/orcl',
* );
*
* The username must have the CREATE TABLE privilege, so test data can be stored for the
* example to import.
* The username must have the CREATE TABLE privilege, so test data can be
* stored for the example to import.
*
* See http://us.php.net/manual/en/function.oci-connect.php for more information on
* connection_string.
* See http://us.php.net/manual/en/function.oci-connect.php for more
* information on connection_string.
*/
/**
* Migration class to test importing from Oracle into nodes.
*/
class MigrateExampleOracleNode extends Migration {
public function __construct() {
parent::__construct();
$this->description = t('Example migration from Oracle into nodes.');
@@ -35,7 +37,7 @@ class MigrateExampleOracleNode extends Migration {
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Content ID',
)
),
),
MigrateDestinationNode::getKeySchema()
);
@@ -67,23 +69,31 @@ class MigrateExampleOracleNode extends Migration {
// Basic fields
$this->addFieldMapping('title', 'TITLE');
$this->addFieldMapping('uid')
->defaultValue(1);
->defaultValue(1);
$this->addFieldMapping('body', 'BODY');
$this->addFieldMapping('field_mainimage', 'MAINIMAGE')
->description('An image blob in the DB')
->arguments(array(
'file_function' => 'file_blob',
// Alternatively, specify a column here for dynamic file name.
'source_path' => 'druplicon.png',
->description('An image blob in the DB')
->arguments(array(
'file_function' => 'file_blob',
// Alternatively, specify a column here for dynamic file name.
'source_path' => 'druplicon.png',
));
));
$this->addFieldMapping('created', 'CREATED');
$this->addFieldMapping('changed', 'UPDATED');
// Unmapped destination fields
$this->addUnmigratedDestinations(array('is_new', 'status', 'promote',
'revision', 'language', 'sticky', 'revision_uid', 'path'));
$this->addUnmigratedDestinations(array(
'is_new',
'status',
'promote',
'revision',
'language',
'sticky',
'revision_uid',
'path',
));
}
}
@@ -95,7 +105,7 @@ function migrate_example_oracle_migrate_api() {
'api' => 2,
'migrations' => array(
'MigrateExampleOracle' => array('class_name' => 'MigrateExampleOracleNode'),
)
),
);
return $api;
}
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.module
+27 -11
View File
@@ -20,20 +20,36 @@ function migrate_example_oracle_generate($length) {
}
/**
* Return an array of data rows for testing Oracle import. Note that 4000 is a magic
* number for Oracle LOB datatypes, so we testing lengths above and below that limit.
* Return an array of data rows for testing Oracle import. Note that 4000 is a
* magic number for Oracle LOB datatypes, so we testing lengths above and below
* that limit.
*/
function migrate_example_oracle_sample_data() {
$image = file_get_contents('misc/druplicon.png');
return array(
array('oid' => 3, 'title' => 'Sample title', 'body' => 'Sample body',
'mainimage' => $image, 'created' => '2011/05/01 01:02:03',
'updated' => '2011/06/30 04:05:06'),
array('oid' => 5, 'title' => 'Another title', 'body' => migrate_example_oracle_generate(3900),
'mainimage' => $image, 'created' => '2011/08/12 07:08:09',
'updated' => '2011/12/25 10:11:12'),
array('oid' => 7, 'title' => 'Yet another title', 'body' => migrate_example_oracle_generate(4500),
'mainimage' => $image, 'created' => '2012/01/01 13:14:15',
'updated' => '2012/03/14 16:17:18'),
array(
'oid' => 3,
'title' => 'Sample title',
'body' => 'Sample body',
'mainimage' => $image,
'created' => '2011/05/01 01:02:03',
'updated' => '2011/06/30 04:05:06',
),
array(
'oid' => 5,
'title' => 'Another title',
'body' => migrate_example_oracle_generate(3900),
'mainimage' => $image,
'created' => '2011/08/12 07:08:09',
'updated' => '2011/12/25 10:11:12',
),
array(
'oid' => 7,
'title' => 'Yet another title',
'body' => migrate_example_oracle_generate(4500),
'mainimage' => $image,
'created' => '2012/01/01 13:14:15',
'updated' => '2012/03/14 16:17:18',
),
);
}
sites/all/modules/contrib/migrate/migrate/migrate_example/wine.inc
+1200 -607
View File
File diff suppressed because it is too large Load Diff
sites/all/modules/contrib/migrate/migrate/migrate_example/wine.install.inc
+272 -103
View File
@@ -65,39 +65,20 @@ function migrate_example_wine_uninstall() {
}
function migrate_example_wine_disable() {
MigrationBase::deregisterMigration('WinePrep');
Migration::deregisterMigration('WineFileCopy');
Migration::deregisterMigration('WineFileBlob');
Migration::deregisterMigration('WineRegion');
Migration::deregisterMigration('WineUser');
Migration::deregisterMigration('WineVariety');
Migration::deregisterMigration('WineBestWith');
Migration::deregisterMigration('WineProducer');
Migration::deregisterMigration('WineProducerXML');
Migration::deregisterMigration('WineProducerXMLPull');
Migration::deregisterMigration('WineProducerMultiXML');
Migration::deregisterMigration('WineWine');
Migration::deregisterMigration('WineComment');
MigrationBase::deregisterMigration('WineFinish');
Migration::deregisterMigration('WineUpdates');
Migration::deregisterMigration('WineUserUpdates');
Migration::deregisterMigration('WineVarietyUpdates');
Migration::deregisterMigration('WineCommentUpdates');
Migration::deregisterMigration('WineRole');
Migration::deregisterMigration('WineTable');
MigrateGroup::deregister('wine');
}
function migrate_example_wine_schema_wine() {
return array(
'description' => 'Wines of the world',
'fields' => array(
'wineid' => array(
'wineid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Wine ID',
),
'name' => array(
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
@@ -159,7 +140,7 @@ function migrate_example_wine_schema_updates() {
return array(
'description' => 'Updated wine ratings',
'fields' => array(
'wineid' => array(
'wineid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -180,13 +161,13 @@ function migrate_example_wine_schema_producer() {
return array(
'description' => 'Wine producers of the world',
'fields' => array(
'producerid' => array(
'producerid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Producer ID',
),
'name' => array(
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
@@ -230,7 +211,7 @@ function migrate_example_wine_schema_categories() {
'not null' => TRUE,
'description' => 'Type of category: variety, region, best_with',
),
'name' => array(
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
@@ -261,12 +242,12 @@ function migrate_example_wine_schema_vintages() {
return array(
'description' => 'Wine vintages',
'fields' => array(
'wineid' => array(
'wineid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Wine ID',
),
'vintage' => array(
'vintage' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -301,12 +282,12 @@ function migrate_example_wine_schema_category_wine() {
return array(
'description' => 'Wine category assignments',
'fields' => array(
'wineid' => array(
'wineid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Wine ID',
),
'categoryid' => array(
'categoryid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -321,12 +302,12 @@ function migrate_example_wine_schema_category_producer() {
return array(
'description' => 'Producer category assignments',
'fields' => array(
'producerid' => array(
'producerid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Producer ID',
),
'categoryid' => array(
'categoryid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -341,13 +322,13 @@ function migrate_example_wine_schema_comment() {
return array(
'description' => 'Wine comments',
'fields' => array(
'commentid' => array(
'commentid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Comment ID',
),
'wineid' => array(
'wineid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -422,7 +403,7 @@ function migrate_example_wine_schema_comment_updates() {
return array(
'description' => 'Wine comment updates',
'fields' => array(
'commentid' => array(
'commentid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -443,12 +424,12 @@ function migrate_example_wine_schema_account() {
return array(
'description' => 'Wine accounts.',
'fields' => array(
'accountid' => array(
'accountid' => array(
'type' => 'serial',
'not null' => TRUE,
'description' => 'Account ID',
),
'status' => array(
'status' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Blocked_Allowed',
@@ -507,7 +488,7 @@ function migrate_example_wine_schema_account() {
'not null' => TRUE,
'description' => 'Signature for comments',
),
'imageid' => array(
'imageid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => FALSE,
@@ -528,7 +509,7 @@ function migrate_example_wine_schema_account_updates() {
return array(
'description' => 'Wine account updates',
'fields' => array(
'accountid' => array(
'accountid' => array(
'type' => 'serial',
'not null' => TRUE,
'description' => 'Account ID',
@@ -548,7 +529,7 @@ function migrate_example_wine_schema_blobs() {
return array(
'description' => 'Wine blobs to be migrated to file entities',
'fields' => array(
'imageid' => array(
'imageid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -568,31 +549,31 @@ function migrate_example_wine_schema_files() {
return array(
'description' => 'Wine and account files',
'fields' => array(
'imageid' => array(
'imageid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Image ID',
),
'url' => array(
'url' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Image URL',
),
'image_alt' => array(
'image_alt' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image alt',
),
'image_title' => array(
'image_title' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image title',
),
'wineid' => array(
'wineid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => FALSE,
@@ -607,19 +588,19 @@ function migrate_example_wine_schema_table_source() {
return array(
'description' => 'Source data to go into a custom Drupal table',
'fields' => array(
'fooid' => array(
'fooid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Primary key',
),
'field1' => array(
'field1' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'First field',
),
'field2' => array(
'field2' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -634,19 +615,19 @@ function migrate_example_wine_schema_table_dest() {
return array(
'description' => 'Custom Drupal table to receive source data directly',
'fields' => array(
'recordid' => array(
'recordid' => array(
'type' => 'serial',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Primary key',
),
'drupal_text' => array(
'drupal_text' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'First field',
),
'drupal_int' => array(
'drupal_int' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
@@ -963,9 +944,12 @@ function migrate_example_wine_fields() {
function migrate_example_wine_content_type_delete() {
$bundle = 'migrate_example_wine';
$field_names = array('migrate_example_wine_varieties',
'migrate_example_wine_regions', 'migrate_example_wine_best_with',
'field_migrate_example_image');
$field_names = array(
'migrate_example_wine_varieties',
'migrate_example_wine_regions',
'migrate_example_wine_best_with',
'field_migrate_example_image',
);
foreach ($field_names as $field_name) {
$instance = field_info_instance('node', $field_name, $bundle);
field_delete_instance($instance);
@@ -983,15 +967,45 @@ function migrate_example_wine_content_type_delete() {
}
function migrate_example_wine_data_wine() {
$fields = array('wineid', 'name', 'body', 'excerpt', 'accountid',
'posted', 'last_changed', 'variety', 'region', 'rating');
$fields = array(
'wineid',
'name',
'body',
'excerpt',
'accountid',
'posted',
'last_changed',
'variety',
'region',
'rating',
);
$query = db_insert('migrate_example_wine')
->fields($fields);
->fields($fields);
$data = array(
array(1, 'Montes Classic Cabernet Sauvignon', 'Intense ruby-red color', 'Great!', 9,
strtotime('2010-01-02 03:04:05'), strtotime('2010-03-04 05:06:07'), 25, 17, 95),
array(2, 'Archeo Ruggero di Tasso Nero d\'Avola', 'Lots of berry character', 'Pair with red sauced dishes', 3,
strtotime('2010-09-03 18:23:58'), strtotime('2010-09-03 18:23:58'), 26, 2, 85),
array(
1,
'Montes Classic Cabernet Sauvignon',
'Intense ruby-red color',
'Great!',
9,
strtotime('2010-01-02 03:04:05'),
strtotime('2010-03-04 05:06:07'),
25,
17,
95,
),
array(
2,
'Archeo Ruggero di Tasso Nero d\'Avola',
'Lots of berry character',
'Pair with red sauced dishes',
3,
strtotime('2010-09-03 18:23:58'),
strtotime('2010-09-03 18:23:58'),
26,
2,
85,
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
@@ -1002,7 +1016,7 @@ function migrate_example_wine_data_wine() {
function migrate_example_wine_data_updates() {
$fields = array('wineid', 'rating');
$query = db_insert('migrate_example_wine_updates')
->fields($fields);
->fields($fields);
$data = array(
array(1, 93),
array(2, NULL),
@@ -1016,7 +1030,7 @@ function migrate_example_wine_data_updates() {
function migrate_example_wine_data_producer() {
$fields = array('producerid', 'name', 'body', 'excerpt', 'accountid');
$query = db_insert('migrate_example_wine_producer')
->fields($fields);
->fields($fields);
$data = array(
array(1, 'Montes', 'Fine Chilean winery', 'Great!', 9),
array(2, 'Archeo', 'Sicilia!', NULL, 3),
@@ -1028,20 +1042,69 @@ function migrate_example_wine_data_producer() {
}
function migrate_example_wine_data_account() {
$fields = array('accountid', 'status', 'posted', 'last_access', 'last_login',
'name', 'sex', 'password', 'mail', 'original_mail', 'sig', 'imageid', 'positions');
$fields = array(
'accountid',
'status',
'posted',
'last_access',
'last_login',
'name',
'sex',
'password',
'mail',
'original_mail',
'sig',
'imageid',
'positions',
);
$query = db_insert('migrate_example_wine_account')
->fields($fields);
$data = array(
array(1, 1, '2010-03-30 10:31:05', '2010-04-30 18:25:24', '2010-04-30 14:01:02',
'darren', 'M', 'dpass', 'ddarren@example.com', 'darren@example.com',
'All about the Australians', NULL, '5'),
array(3, 0, '2007-03-15 10:31:05', '2007-06-10 04:11:38', '2007-06-10 04:11:38',
'emily', 'F', 'insecure', 'emily@example.com', 'emily@example.com',
'Sommelier to the stars', NULL, '18'),
array(9, 1, '2004-02-29 10:31:05', '2004-02-29 10:31:05', '2004-02-29 10:31:05',
'fonzie', NULL, 'bike', 'thefonz@example.com', 'arthur@example.com',
'Aaay!', 1, '5,18'),
array(
1,
1,
'2010-03-30 10:31:05',
'2010-04-30 18:25:24',
'2010-04-30 14:01:02',
'darren',
'M',
'dpass',
'ddarren@example.com',
'darren@example.com',
'All about the Australians',
NULL,
'5',
),
array(
3,
0,
'2007-03-15 10:31:05',
'2007-06-10 04:11:38',
'2007-06-10 04:11:38',
'emily',
'F',
'insecure',
'emily@example.com',
'emily@example.com',
'Sommelier to the stars',
NULL,
'18',
),
array(
9,
1,
'2004-02-29 10:31:05',
'2004-02-29 10:31:05',
'2004-02-29 10:31:05',
'fonzie',
NULL,
'bike',
'thefonz@example.com',
'arthur@example.com',
'Aaay!',
1,
'5,18',
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
@@ -1052,7 +1115,7 @@ function migrate_example_wine_data_account() {
function migrate_example_wine_data_account_updates() {
$fields = array('accountid', 'sex');
$query = db_insert('migrate_example_wine_account_updates')
->fields($fields);
->fields($fields);
$data = array(
array(1, NULL),
array(3, 'M'),
@@ -1065,26 +1128,93 @@ function migrate_example_wine_data_account_updates() {
}
function migrate_example_wine_data_comment() {
$fields = array('commentid', 'wineid', 'comment_parent', 'subject', 'body',
'name', 'mail', 'accountid', 'commenthost', 'userpage', 'posted', 'lastchanged');
$fields = array(
'commentid',
'wineid',
'comment_parent',
'subject',
'body',
'name',
'mail',
'accountid',
'commenthost',
'userpage',
'posted',
'lastchanged',
);
$query = db_insert('migrate_example_wine_comment')
->fields($fields);
$data = array(
array(1, 1, NULL, 'im first', 'Tasty', 'grace', 'grace@example.com', 0,
'123.456.78.9', 'http:://grace.example.com/',
strtotime('2010-01-02 03:04:05'), strtotime('2010-04-05 06:07:08')),
array(2, 1, NULL, 'im second', 'Delicious', 'horace', 'horace@example.com', 0,
'example.com', NULL,
strtotime('2010-02-02 03:04:05'), strtotime('2010-05-05 06:07:08')),
array(3, 1, NULL, 'im parent', 'Don\'t care for it', 'irene', 'irene@example.com', 0,
'254.0.2.5', 'http:://www.example.com/irene',
strtotime('2010-03-02 03:04:05'), strtotime('2010-03-02 03:04:05')),
array(4, 1, 3, 'im child', 'But it\'s so good!', 'emily', NULL, 3,
'58.29.126.1', 'http:://www.wine.com/',
strtotime('2010-01-02 03:04:05'), strtotime('2010-01-02 03:04:05')),
array(5, 1, 4, 'im grandchild', 'Right on, Emily!', 'thefonz@example.com', NULL, 9,
'123.456.78.9', NULL,
strtotime('2010-06-02 03:04:05'), strtotime('2010-06-02 03:04:05')),
array(
1,
1,
NULL,
'im first',
'Tasty',
'grace',
'grace@example.com',
0,
'123.456.78.9',
'http:://grace.example.com/',
strtotime('2010-01-02 03:04:05'),
strtotime('2010-04-05 06:07:08'),
),
array(
2,
1,
NULL,
'im second',
'Delicious',
'horace',
'horace@example.com',
0,
'example.com',
NULL,
strtotime('2010-02-02 03:04:05'),
strtotime('2010-05-05 06:07:08'),
),
array(
3,
1,
NULL,
'im parent',
'Don\'t care for it',
'irene',
'irene@example.com',
0,
'254.0.2.5',
'http:://www.example.com/irene',
strtotime('2010-03-02 03:04:05'),
strtotime('2010-03-02 03:04:05'),
),
array(
4,
1,
3,
'im child',
'But it\'s so good!',
'emily',
NULL,
3,
'58.29.126.1',
'http:://www.wine.com/',
strtotime('2010-01-02 03:04:05'),
strtotime('2010-01-02 03:04:05'),
),
array(
5,
1,
4,
'im grandchild',
'Right on, Emily!',
'thefonz@example.com',
NULL,
9,
'123.456.78.9',
NULL,
strtotime('2010-06-02 03:04:05'),
strtotime('2010-06-02 03:04:05'),
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
@@ -1095,7 +1225,7 @@ function migrate_example_wine_data_comment() {
function migrate_example_wine_data_comment_updates() {
$fields = array('commentid', 'subject');
$query = db_insert('migrate_example_wine_comment_updates')
->fields($fields);
->fields($fields);
$data = array(
array(1, 'I am first'),
array(2, 'I am second'),
@@ -1110,12 +1240,33 @@ function migrate_example_wine_data_comment_updates() {
}
function migrate_example_wine_data_categories() {
$fields = array('categoryid', 'type', 'name', 'category_parent', 'details', 'ordering');
$fields = array(
'categoryid',
'type',
'name',
'category_parent',
'details',
'ordering',
);
$query = db_insert('migrate_example_wine_categories')
->fields($fields);
->fields($fields);
$data = array(
array(1, 'variety', 'White wine', NULL, 'White wines are generally simpler and sweeter than red', 3),
array(3, 'variety', 'Red wine', NULL, 'Red wines are generally more complex and "dry" than white', 1),
array(
1,
'variety',
'White wine',
NULL,
'White wines are generally simpler and sweeter than red',
3,
),
array(
3,
'variety',
'Red wine',
NULL,
'Red wines are generally more complex and "dry" than white',
1,
),
array(8, 'variety', 'Riesling', 1, 'Associated with Germany', 2),
array(9, 'variety', 'Chardonnay', 1, 'One of the most popular whites', 1),
array(13, 'variety', 'Merlot', 3, 'Very drinkable', 4),
@@ -1147,7 +1298,7 @@ function migrate_example_wine_data_categories() {
function migrate_example_wine_data_vintages() {
$fields = array('wineid', 'vintage');
$query = db_insert('migrate_example_wine_vintages')
->fields($fields);
->fields($fields);
$data = array(
array(1, 2006),
array(1, 2007),
@@ -1162,7 +1313,7 @@ function migrate_example_wine_data_vintages() {
function migrate_example_wine_data_variety_updates() {
$fields = array('categoryid', 'details');
$query = db_insert('migrate_example_wine_variety_updates')
->fields($fields);
->fields($fields);
$data = array(
array(1, 'White wines are simpler and sweeter than red'),
array(3, 'Red wines are generally more complex and dry than white'),
@@ -1212,10 +1363,28 @@ function migrate_example_wine_data_files() {
$query = db_insert('migrate_example_wine_files')
->fields($fields);
$data = array(
array(1, 'http://drupal.org/sites/all/modules/drupalorg/drupalorg/images/association-individual.png', NULL, NULL, NULL),
array(2, 'http://cyrve.com/files/penguin.jpeg', 'Penguin alt', 'Penguin title', 1),
array(3, 'http://cyrve.com/files/rioja.jpeg', 'Rioja alt', 'Rioja title', 2),
array(4, 'http://cyrve.com/files/boutisse_0.jpeg', 'Boutisse alt', 'Boutisse title', 2),
array(1, 'http://placekitten.com/200/200', NULL, NULL, NULL),
array(
2,
'http://cyrve.com/files/penguin.jpeg',
'Penguin alt',
'Penguin title',
1,
),
array(
3,
'http://cyrve.com/files/rioja.jpeg',
'Rioja alt',
'Rioja title',
2,
),
array(
4,
'http://cyrve.com/files/boutisse_0.jpeg',
'Boutisse alt',
'Boutisse title',
2,
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
sites/all/modules/contrib/migrate/migrate/migrate_example/xml/0002.xml
+7
View File
@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<pr:producer xmlns:pr="http://www.wine.org/wine-producers">
<pr:name>Château Latour</pr:name>
<pr:description>Makers of grand vin Chateau Latour, Les Forts de Latour and Pauillac</pr:description>
<pr:authorid>3</pr:authorid>
<pr:region>Bordeaux</pr:region>
</pr:producer>
sites/all/modules/contrib/migrate/migrate/migrate_example/xml/index2.xml
+4
View File
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
<wn:content xmlns:wn="http://www.wine.org/wine">
<wn:sourceid>0002</wn:sourceid>
</wn:content>
sites/all/modules/contrib/migrate/migrate/migrate_example/xml/producers3.xml
+17
View File
@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<pr:producers xmlns:pr="http://www.wine.org/wine-producers">
<pr:producer>
<pr:sourceid>0009</pr:sourceid>
<pr:name>Château Feytit Clinet</pr:name>
<pr:description>Good things, they say, come in small packages; this is certainly the case at Château Feyit Clinet, the maker of thelabel Pomerol.</pr:description>
<pr:authorid>1</pr:authorid>
<pr:region>Pomerol</pr:region>
</pr:producer>
<pr:producer>
<pr:sourceid>0010</pr:sourceid>
<pr:name>Château Doisy-Védrines</pr:name>
<pr:description>Blessed with ancient vines, the fruit here is often subject to high levels of Botrytis (or noble rot), which shrinks the grapes and concentrates sugar levels in the remaining juice.</pr:description>
<pr:authorid>3</pr:authorid>
<pr:region>Barsac</pr:region>
</pr:producer>
</pr:producers>
sites/all/modules/contrib/migrate/migrate/migrate_example/xml/producers4.xml
+17
View File
@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<pr:producers xmlns:pr="http://www.wine.org/wine-producers">
<pr:producer>
<pr:sourceid>0009</pr:sourceid>
<pr:name>Château Bourgneuf</pr:name>
<pr:description>Showing attractive red and dark fruits, cherry and plum, it is rich and even slightly racy, before concluding with classic Pomerol concentration and focus.</pr:description>
<pr:authorid>3</pr:authorid>
<pr:region>Pomerol</pr:region>
</pr:producer>
<pr:producer>
<pr:sourceid>0010</pr:sourceid>
<pr:name>Château Doisy-Daëne</pr:name>
<pr:description>Medium bodied, with elegance rather than density there is a wide spectrum of flavours; apples, peaches, lemongrass and a touch of white spice. Delicious now.</pr:description>
<pr:authorid>9</pr:authorid>
<pr:region>Barsac</pr:region>
</pr:producer>
</pr:producers>
sites/all/modules/contrib/migrate/migrate/migrate_example_baseball/migrate_example_baseball.features.inc
+1 -1
View File
@@ -6,7 +6,7 @@
function migrate_example_baseball_node_info() {
$items = array(
'migrate_example_baseball' => array(
'name' => t('migrate_example_baseball'),
'name' => t('Migrate example - Baseball'),
'base' => 'node_content',
'description' => t('A baseball box score'),
'has_title' => '1',
sites/all/modules/contrib/migrate/migrate/migrate_example_baseball/migrate_example_baseball.info
+4 -5
View File
@@ -21,12 +21,11 @@ features[field][] = "node-migrate_example_baseball-field_visiting_team"
features[node][] = "migrate_example_baseball"
files[] = "migrate_example_baseball.migrate.inc"
name = "migrate_example_baseball"
package = "Migrate Examples"
package = "Migration"
php = "5.2.4"
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2018-06-10
version = "7.x-2.11"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1528674486"
sites/all/modules/contrib/migrate/migrate/migrate_example_baseball/migrate_example_baseball.install
+21 -17
View File
@@ -8,11 +8,11 @@
function migrate_example_baseball_enable() {
$path = dirname(__FILE__) . '/data';
migrate_example_baseball_get_files($path);
for ($i=0; $i<=9; $i++) {
$file = 'gl200' . $i . '.txt';
for ($i = 0; $i <= 9; $i++) {
$file = 'GL200' . $i . '.TXT';
Migration::registerMigration('GameBaseball',
pathinfo($file, PATHINFO_FILENAME),
array('source_file' => $path . '/' . $file));
array('source_file' => $path . '/' . $file, 'group_name' => 'baseball'));
}
}
@@ -26,7 +26,7 @@ function migrate_example_baseball_get_files($path) {
if (!file_exists("$path/GL2000.TXT") && !file_exists("$path/gl2000.txt")) {
file_prepare_directory($path, FILE_CREATE_DIRECTORY | FILE_MODIFY_PERMISSIONS);
$result = copy('http://www.retrosheet.org/gamelogs/gl2000_09.zip',
$path . '/gl2000_09.zip');
$path . '/gl2000_09.zip');
if ($result) {
$zip = new ZipArchive();
$zip->open($path . '/gl2000_09.zip');
@@ -39,11 +39,22 @@ function migrate_example_baseball_get_files($path) {
function migrate_example_baseball_uninstall() {
$bundle = 'migrate_example_baseball';
$field_names = array('field_park', 'field_home_team', 'field_home_game_number',
'field_home_score', 'field_visiting_score', 'field_outs', 'field_attendance',
'field_duration', 'field_home_batters', 'field_visiting_batters',
'field_home_pitcher', 'field_visiting_pitcher', 'field_visiting_team',
'field_start_date');
$field_names = array(
'field_park',
'field_home_team',
'field_home_game_number',
'field_home_score',
'field_visiting_score',
'field_outs',
'field_attendance',
'field_duration',
'field_home_batters',
'field_visiting_batters',
'field_home_pitcher',
'field_visiting_pitcher',
'field_visiting_team',
'field_start_date',
);
foreach ($field_names as $field_name) {
$instance = field_info_instance('node', $field_name, $bundle);
field_delete_instance($instance);
@@ -53,14 +64,7 @@ function migrate_example_baseball_uninstall() {
}
function migrate_example_baseball_disable() {
for ($i=0; $i<=9; $i++) {
$file = 'gl200' . $i . '.txt';
Migration::deregisterMigration(pathinfo($file, PATHINFO_FILENAME));
$filename = dirname(__FILE__) . '/data/' . $file;
if (file_exists($filename)) {
unlink($filename);
}
}
MigrateGroup::deregister('baseball');
}
/**
sites/all/modules/contrib/migrate/migrate/migrate_example_baseball/migrate_example_baseball.migrate.inc
+53 -42
View File
@@ -13,39 +13,47 @@
function migrate_example_baseball_migrate_api() {
$api = array(
'api' => 2,
'groups' => array(
'baseball' => array(
'title' => t('Baseball'),
),
),
);
return $api;
}
/**
* A dynamic migration that is reused for each source CSV file.
* A migration that is reused for each source CSV file.
*/
class GameBaseball extends DynamicMigration {
public function __construct(array $arguments) {
$this->arguments = $arguments;
parent::__construct();
class GameBaseball extends Migration {
public function __construct($arguments) {
parent::__construct($arguments);
$this->description = t('Import box scores from CSV file.');
// Create a map object for tracking the relationships between source rows
$this->map = new MigrateSQLMap($this->machineName,
array(
'start_date' => array('type' => 'varchar',
'length' => 8,
'not null' => TRUE,
'description' => 'Start date',
),
'home_team' => array('type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Home team',
),
'home_game_number' => array('type' => 'int',
'not null' => TRUE,
'description' => 'Home team game number',
),
array(
'start_date' => array(
'type' => 'varchar',
'length' => 8,
'not null' => TRUE,
'description' => 'Start date',
),
MigrateDestinationNode::getKeySchema()
);
'home_team' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Home team',
),
'home_game_number' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Home team game number',
),
),
MigrateDestinationNode::getKeySchema()
);
// Create a MigrateSource object, which manages retrieving the input data.
$this->source = new MigrateSourceCSV($arguments['source_file'], $this->csvcolumns(), array(), $this->fields());
@@ -53,7 +61,7 @@ class GameBaseball extends DynamicMigration {
$this->destination = new MigrateDestinationNode('migrate_example_baseball');
$this->addFieldMapping('title', 'title')
->description('See prepareRow().');
->description('See prepareRow().');
$this->addFieldMapping('field_start_date', 'start_date');
$this->addFieldMapping('field_park', 'park_id');
$this->addFieldMapping('field_visiting_team', 'visiting_team');
@@ -64,24 +72,24 @@ class GameBaseball extends DynamicMigration {
$this->addFieldMapping('field_outs', 'outs');
$this->addFieldMapping('field_attendance', 'attendance');
$this->addFieldMapping('field_duration', 'duration')
->defaultValue(NULL);
->defaultValue(NULL);
$this->addFieldMapping('field_home_pitcher', 'home_pitcher');
$this->addFieldMapping('field_visiting_pitcher', 'visiting_pitcher');
$this->addFieldMapping('field_home_batters', 'home_batters')
->separator(',')
->description('See prepareRow().');
->separator(',')
->description('See prepareRow().');
$this->addFieldMapping('field_visiting_batters', 'visiting_batters')
->separator(',')
->description('See prepareRow().');
for ($i=1; $i <= 9; $i++ ) {
->separator(',')
->description('See prepareRow().');
for ($i = 1; $i <= 9; $i++) {
$this->addFieldMapping(NULL, "visiting_batter_$i")
->description('Not needed since we use the multi-value field: visiting_batters.');
->description('Not needed since we use the multi-value field: visiting_batters.');
$this->addFieldMapping(NULL, "home_batter_$i")
->description('Not needed since we use the multi-value field: home_batters.');
->description('Not needed since we use the multi-value field: home_batters.');
}
}
function csvcolumns() {
protected function csvcolumns() {
// Note: Remember to subtract 1 from column number at http://www.retrosheet.org/gamelogs/glfields.txt
$columns[0] = array('start_date', 'Date of game');
$columns[3] = array('visiting_team', 'Visiting team');
@@ -93,22 +101,25 @@ class GameBaseball extends DynamicMigration {
$columns[16] = array('park_id', 'Ballpark ID');
$columns[17] = array('attendance', 'Attendance');
$columns[18] = array('duration', 'Duration in minutes');
for ($i=1; $i <= 9; $i++ ) {
$columns[103+3*$i] = array("visiting_batter_$i", "Visiting batter $i");
$columns[130+3*$i] = array("home_batter_$i", "Home batter $i");
for ($i = 1; $i <= 9; $i++) {
$columns[103 + 3 * $i] = array(
"visiting_batter_$i",
"Visiting batter $i",
);
$columns[130 + 3 * $i] = array("home_batter_$i", "Home batter $i");
}
$columns[102] = array('visiting_pitcher', 'Visiting starting pitcher');
$columns[104] = array('home_pitcher', 'Home starting pitcher');
return $columns;
}
function prepareRow($row) {
public function prepareRow($row) {
// Collect all the batters into one multi-value field.
for ($i=1; $i <= 9; $i++ ) {
for ($i = 1; $i <= 9; $i++) {
$key = "visiting_batter_$i";
$visiting_batters[] = $row->$key;
$visiting_batters[] = $row->{$key};
$key = "home_batter_$i";
$home_batters[] = $row->$key;
$home_batters[] = $row->{$key};
}
$row->visiting_batters = implode(',', $visiting_batters);
$row->home_batters = implode(',', $home_batters);
@@ -120,14 +131,14 @@ class GameBaseball extends DynamicMigration {
*/
protected function generateMachineName($class_name = NULL) {
return drupal_strtolower(pathinfo($this->arguments['source_file'],
PATHINFO_FILENAME));
PATHINFO_FILENAME));
}
function fields() {
public function fields() {
return array(
'title' => 'A composite field made during prepareRow().',
'home_batters' => 'An array of batters, populated during prepareRow().',
'visiting_batters' => 'An array of batters, populated during prepareRow().',
);
}
}
}
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.css
+3 -6
View File
@@ -1,18 +1,15 @@
table.migrate-dashboard tr.migrate-running {
background-color: #CFC;
background-color: #cfc;
}
.migrate-running {
background-color: #CFC;
background-color: #cfc;
}
.migrate-option-separator {
margin-bottom:0.2em;
margin-bottom: 0.2em;
padding-bottom: 0.2em;
border-bottom: 1px solid #aaa;
}
#migrate-migration-info td.migrate-error {
background: #ffc9c9;
}
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.info
+6 -7
View File
@@ -1,14 +1,13 @@
name = "Migrate UI"
description = "UI for managing migration processes"
package = "Development"
;configure = admin/config/development/migrate
package = "Migration"
configure = admin/content/migrate/configure
core = 7.x
dependencies[] = migrate
files[] = migrate_ui.module
files[] = migrate_ui.wizard.inc
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2018-06-10
version = "7.x-2.11"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1528674486"
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.install
+71
View File
@@ -0,0 +1,71 @@
<?php
/**
* @file
* Install/update function for migrate_ui.
*/
/**
* Implements hook_install().
*/
function migrate_ui_install() {
migrate_ui_set_weight();
}
/**
* Implements hook_uninstall().
*/
function migrate_ui_uninstall() {
variable_del('migrate_import_method');
variable_del('migrate_drush_path');
variable_del('migrate_drush_mail');
variable_del('migrate_drush_mail_subject');
variable_del('migrate_drush_mail_body');
}
/**
* Make sure we have a higher weight than node.
*/
function migrate_ui_update_7201() {
migrate_ui_set_weight();
}
/**
* Sets the weight of migrate_ui higher than node, so Import links come after
* "Add content" at admin/content.
*/
function migrate_ui_set_weight() {
$node_weight = db_select('system', 's')
->fields('s', array('weight'))
->condition('name', 'node')
->execute()
->fetchField();
db_update('system')
->fields(array('weight' => $node_weight + 1))
->condition('name', 'migrate_ui')
->execute();
}
/**
* If WordPress Migrate has background imports via drush enabled, copy the
* configuration to the new general Migrate support.
*/
function migrate_ui_update_7202() {
$drush_command = variable_get('wordpress_migrate_drush', '');
if ($drush_command) {
variable_set('migrate_drush_path', $drush_command);
// Consolidate these two variables into import method - 0 means immediate
// only, 1 means drush only, 2 means offer both options.
$import_method = variable_get('wordpress_migrate_import_method', 0);
$force_drush = variable_get('wordpress_migrate_force_drush', FALSE);
if (!$force_drush) {
$import_method = 2;
}
variable_set('migrate_import_method', $import_method);
variable_set('migrate_drush_mail',
variable_get('wordpress_migrate_notification', 0));
variable_set('migrate_drush_mail_subject',
variable_get('wordpress_migrate_notification_subject', ''));
variable_set('migrate_drush_mail_body',
variable_get('wordpress_migrate_notification_body', ''));
}
}
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.module
+153 -62
View File
@@ -1,25 +1,12 @@
<?php
define('MIGRATE_ACCESS_BASIC', 'migration information');
function migrate_ui_help($path, $arg) {
switch ($path) {
case 'admin/migrate':
return t('The current status of each migration defined in this system. Click on a migration name for details on its configuration.');
case 'admin/content/migrate':
return t('The current status of each migration group defined in this system. Click on a group name for details on its configuration.');
}
}
/**
* Implementation of hook_permission().
*/
function migrate_ui_permission() {
return array(
MIGRATE_ACCESS_BASIC => array(
'title' => t('Basic access to migration information'),
),
);
}
/**
* Implementation of hook_menu().
*/
@@ -27,67 +14,171 @@ function migrate_ui_menu() {
$items = array();
$items['admin/content/migrate'] = array(
'title' => 'Migrate',
'type' => MENU_LOCAL_TASK | MENU_NORMAL_ITEM,
'description' => 'Monitor the creation of Drupal content from source data',
'page callback' => 'migrate_ui_dashboard',
'title' => 'Migrate',
'description' => 'Manage importing of data into your Drupal site',
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_ui_migrate_dashboard'),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'access callback' => 'user_access',
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/dashboard'] = array(
'title' => 'Migrate',
$items['admin/content/migrate/groups'] = array(
'title' => 'Dashboard',
'type' => MENU_DEFAULT_LOCAL_TASK,
'weight' => -10,
);
$items['admin/content/migrate/registration'] = array(
'title' => 'Registration',
$items['admin/content/migrate/configure'] = array(
'title' => 'Configuration',
'type' => MENU_LOCAL_TASK,
'description' => 'Configure class registration',
'page callback' => 'migrate_ui_registration',
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
'weight' => 5,
);
$items['admin/content/migrate/handlers'] = array(
'title' => 'Handlers',
'type' => MENU_LOCAL_TASK,
'description' => 'Configure migration handlers',
'page callback' => 'migrate_ui_handlers',
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
'weight' => 10,
);
$items['admin/content/migrate/messages/%migration'] = array(
'title callback' => '_migrate_ui_title',
'title arguments' => array(4),
'description' => 'View messages from a migration',
'page callback' => 'migrate_ui_messages',
'page arguments' => array(4),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/%migration'] = array(
'title callback' => '_migrate_ui_title',
'title arguments' => array(3),
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_migration_info', 3),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'description' => 'Configure migration settings',
'page callback' => 'migrate_ui_configure',
'access arguments' => array(MIGRATE_ACCESS_ADVANCED),
'file' => 'migrate_ui.pages.inc',
'weight' => 100,
);
// Add tabs for each implemented migration wizard.
$wizards = migrate_ui_wizards();
foreach ($wizards as $wizard_class => $wizard) {
$items["admin/content/migrate/new/$wizard_class"] = array(
'type' => MENU_LOCAL_TASK,
'title' => 'Import from @source_title',
'title arguments' => array('@source_title' => $wizard->getSourceName()),
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_ui_wizard', $wizard_class),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.wizard.inc',
);
}
$items['admin/content/migrate/groups/%'] =
array(
'title callback' => 'migrate_ui_migrate_group_title',
'title arguments' => array(4),
'type' => MENU_NORMAL_ITEM,
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_ui_migrate_group', 4),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/groups/%/%'] =
array(
'title callback' => 'migrate_ui_migrate_migration_title',
'title arguments' => array(5),
'type' => MENU_NORMAL_ITEM,
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_migration_info', 4, 5),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/groups/%/%/view'] =
array(
'title' => 'View',
'type' => MENU_DEFAULT_LOCAL_TASK,
'weight' => -10,
);
$items['admin/content/migrate/groups/%/%/edit'] =
array(
'type' => MENU_LOCAL_TASK,
'title' => 'Edit',
'description' => 'Edit migration mappings',
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_ui_edit_mappings', 4, 5),
'access arguments' => array(MIGRATE_ACCESS_ADVANCED),
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/groups/%/%/messages'] =
array(
'type' => MENU_LOCAL_TASK,
'title' => 'Messages',
'description' => 'View messages from a migration',
'page callback' => 'migrate_ui_messages',
'page arguments' => array(4, 5),
'access arguments' => array(MIGRATE_ACCESS_ADVANCED),
'file' => 'migrate_ui.pages.inc',
);
return $items;
}
// A menu load callback.
function migration_load($machine_name) {
if ($machine_name) {
return Migration::getInstance($machine_name);
}
/**
* Title callback for the migrate group view page.
*/
function migrate_ui_migrate_group_title($group_name) {
$group = MigrateGroup::getInstance($group_name);
return $group->getTitle();
}
function _migrate_ui_title($migration) {
if (is_string($migration)) {
$migration = migration_load($migration);
}
return $migration->getMachineName();
/**
* Title callback for the migration view page.
*/
function migrate_ui_migrate_migration_title($migration_name) {
return $migration_name;
}
/**
* Implements hook_theme()
*
* @return array
*/
function migrate_ui_theme() {
return array(
'migrate_ui_field_mapping_form' => array(
'arguments' => array('field_mappings' => NULL),
'render element' => 'field_mappings',
'file' => '/migrate_ui.pages.inc',
),
'migrate_ui_field_mapping_dependencies' => array(
'arguments' => array('dependencies' => NULL),
'render element' => 'dependencies',
'file' => '/migrate_ui.pages.inc',
),
);
}
/**
* Implementation of hook_mail().
*
* @param $key
* @param $message
* @param $params
*/
function migrate_ui_mail($key, &$message, $params) {
$options['language'] = $message['language'];
user_mail_tokens($variables, array(), $options);
$langcode = $message['language']->language;
$subject = variable_get('migrate_drush_mail_subject', '');
$message['subject'] = t($subject, array(), array('langcode' => $langcode));
$body = variable_get('migrate_drush_mail_body', '');
$body .= "\n" . $params['output'];
$message['body'][] = t($body, array(), array('langcode' => $langcode));
}
/**
* Get info on all modules supporting a migration wizard.
*
* @return array
* key: machine_name for a particular wizard implementation. Used in the menu
* link.
* value: Wizard configuration array containing:
* source_title -
*/
function migrate_ui_wizards() {
$module_apis = migrate_get_module_apis();
$wizards = array();
foreach ($module_apis as $info) {
if (isset($info['wizard classes']) && is_array($info['wizard classes'])) {
foreach ($info['wizard classes'] as $wizard_class) {
$wizard_class = strtolower($wizard_class);
$wizards[$wizard_class] = new $wizard_class;
}
}
}
return $wizards;
}
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.pages.inc
+1433 -282
View File
File diff suppressed because it is too large Load Diff
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.wizard.inc
+679
View File
@@ -0,0 +1,679 @@
<?php
/**
* @file
* Migration wizard framework.
*/
/**
* The primary formbuilder function for the wizard form.
*
* This form has two defined submit handlers to process the different steps:
* - Previous: handles the way to get back one step in the wizard.
* - Next: handles each step form submission,
*
* The third handler, the finish button handler, is the default form _submit
* handler used to process the information.
*
* @param string $class_name
* Name of the MigrateUIWizard clsas for this wizard.
*/
function migrate_ui_wizard($form, &$form_state, $class_name = '') {
// Rather than track state in $form_state, we simply keep our wizard
// instance there, and it encapsulates all the state. We just need
// to create the instance the first time in, and it will be serialized
// between steps.
/** @var MigrateUIWizard $wizard */
if (empty($form_state['wizard'])) {
$wizard = new $class_name();
// Add any extenders.
$module_apis = migrate_get_module_apis();
// Need a second pass at this to add wizard extenders.
foreach ($module_apis as $module => $info) {
// Add any extenders.
// @todo: consider allowing extender classes to declare dependencies on
// other extender classes, to ensure they work in the correct order?
if (isset($info['wizard extenders'])) {
foreach ($info['wizard extenders'] as $wizard_class => $extender_classes) {
// Note that $class_name is in lower case, so we can't just use isset()
// to find our wizard.
if (strtolower($wizard_class) == $class_name) {
foreach ($extender_classes as $extender_class) {
$wizard->addExtender($extender_class);
}
}
}
}
}
$form_state['wizard'] = $wizard;
}
else {
$wizard = $form_state['wizard'];
}
// Fetch the form for the wizard's current step.
$form = $wizard->form($form_state);
return $form;
}
/**
* Submit handler for the "previous" button. Moves the wizard back to the
* previous step, and retrieves the values that were submitted on that step.
*
* @todo: Can we remove steps that were dynamically added?
*/
function migrate_ui_wizard_previous_submit($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->gotoPreviousStep($form_state);
}
/**
* Validate handler for the 'next' button. Dispatches to the wizard's current
* step for validation.
*/
function migrate_ui_wizard_next_validate($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->formValidate($form_state);
}
/**
* Submit handler for the 'next' button. Saves the form values for the step
* we're leaving, so Previous can pick them up, and moves the wizard to the
* next step.
*/
function migrate_ui_wizard_next_submit($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->gotoNextStep($form_state);
}
/**
* Submit handler for the Save settings button. Register the migrations that
* were
* (implicitly) defined along the way and redirect to the Migrate dashboard.
*/
function migrate_ui_wizard_submit($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->formSaveSettings();
$form_state['redirect'] = 'admin/content/migrate/groups/' .
$wizard->getGroupName();
}
/**
* Submit handler for the "Save settings and import" button. Register the
* migrations that were (implicitly) defined along the way, run the import, and
* redirect to the Migrate dashboard.
*/
function migrate_ui_wizard_migrate_submit($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->formSaveSettings();
$wizard->formPerformImport();
$form_state['redirect'] = 'admin/content/migrate/groups/' .
$wizard->getGroupName();
}
/**
* The base class for migration wizards. Extend this class to implement a
* wizard UI for importing into Drupal from a given source format (Drupal,
* WordPress, etc.).
*/
abstract class MigrateUIWizard {
/**
* We maintain a doubly-linked list of wizard steps, both to support
* previous/next, and to easily insert steps dynamically.
*
* The first step of the wizard, which has no predecessor. Will generally be
* an overview/introductory page.
*
* @var MigrateUIStep
*/
protected $firstStep;
/**
* The last step of the wizard, which has no successor. Will generally be a
* review page.
*
* @var MigrateUIStep
*/
protected $lastStep;
/**
* Get the list of steps currently defined.
*
* @return
* An array of MigrateUIStep objects, in the order defined, keyed by the step
* name.
*/
protected function getSteps() {
$steps = array();
$steps[$this->firstStep->getName()] = $this->firstStep;
$next_step = $this->firstStep->nextStep;
while (!is_null($next_step)) {
$steps[$next_step->getName()] = $next_step;
$next_step = $next_step->nextStep;
}
return $steps;
}
/**
* The current step of the wizard (the one being shown in the UI, and the one
* whose button is being clicked on).
*
* @var MigrateUIStep
*/
protected $currentStep;
/**
* The step number, used in the page title.
*
* @var int
*/
protected $stepNumber = 1;
/**
* The group name to assign to any Migration instances created.
*
* @var string
*/
protected $groupName = 'default';
public function getGroupName() {
return $this->groupName;
}
/**
* The user-visible title of the group.
*
* @var string
*/
protected $groupTitle = 'default';
/**
* Any arguments that apply to all migrations in the group.
*
* @var array
*/
protected $groupArguments = array();
/**
* Array of Migration argument arrays, keyed by machine name. On Finish, used
* to register Migrations.
*
* @var array
*/
protected $migrations = array();
/**
* Array of MigrateUIWizardExtender objects that extend this wizard.
*
* @var array
*/
protected $extenders = array();
public function getExtender($extender_class) {
if (isset($this->extenders[$extender_class])) {
return $this->extenders[$extender_class];
}
else {
return NULL;
}
}
/**
* Returns the translatable name representing the source of the data (e.g.,
* "Drupal", "WordPress", etc.).
*
* @return string
*/
abstract public function getSourceName();
public function __construct() {
}
/**
* Add a wizard extender.
*
* This initializes the new extender and adds it to our internal list.
*
* @param $extender_class
* The name of an extender class.
*/
public function addExtender($extender_class) {
$steps = $this->getSteps();
$extender = new $extender_class($this, $steps);
$this->extenders[$extender_class] = $extender;
}
/**
* Add a step to the wizard, using a step name and method.
*
* @param string $name
* Translatable name for the step, to be used in the page title.
* @param callable $form_method
* Callable returning the form array for the step. This can be either the
* name of a MigrateUIWizard method, or a callable array specifying a method
* on a wizard extender. The validation method is formed from the method's
* name with the suffix 'Validate' added.
* @param MigrateUIStep $after
* Optional step after which to insert the new step. If omitted, add it at
* the end.
* @param mixed $context
* Optional data to be used by this step's form.
*
* @return MigrateUIStep
*/
public function addStep($name, $form_method, MigrateUIStep $after = NULL, $context = NULL) {
if (!is_array($form_method)) {
$form_method = array($this, $form_method);
}
$new_step = new MigrateUIStep($name, $form_method, $context);
// There were no steps, so this is the only one.
if (is_null($this->firstStep)) {
$this->firstStep = $this->lastStep = $this->currentStep = $new_step;
}
else {
// If no insertion point is specified, append to the end.
if (is_null($after)) {
$after = $this->lastStep;
}
// Do the insert, rewriting the links appropriately.
$new_step->nextStep = $after->nextStep;
if (is_null($new_step->nextStep)) {
$this->lastStep = $new_step;
}
else {
$new_step->nextStep->previousStep = $new_step;
}
$new_step->previousStep = $after;
$after->nextStep = $new_step;
}
return $new_step;
}
/**
* Remove the named step from the wizard.
*
* @param $name
*/
protected function removeStep($name) {
for ($current_step = $this->firstStep; !is_null($current_step); $current_step = $current_step->nextStep) {
if ($current_step->getName() == $name) {
if (is_null($current_step->previousStep)) {
$this->firstStep = $current_step->nextStep;
}
else {
$current_step->previousStep->nextStep = $current_step->nextStep;
}
if (is_null($current_step->nextStep)) {
$this->lastStep = $current_step->previousStep;
}
else {
$current_step->nextStep->previousStep = $current_step->previousStep;
}
break;
}
}
}
/**
* Move the wizard to the next step in line (if any), first squirreling away
* the current step's form values.
*/
public function gotoNextStep(&$form_state) {
if ($this->currentStep && $this->currentStep->nextStep) {
$this->currentStep->setFormValues($form_state['values']);
$form_state['rebuild'] = TRUE;
$this->currentStep = $this->currentStep->nextStep;
$this->stepNumber++;
// Ensure a page reload remains on the current step.
$current_step_form_values = $this->currentStep->getFormValues();
if (!empty($current_step_form_values)) {
$form_state['values'] = $current_step_form_values;
}
else {
$form_state['values'] = array();
}
}
}
/**
* Move the wizard to the previous step in line (if any), restoring its
* form values.
*/
public function gotoPreviousStep(&$form_state) {
if ($this->currentStep && $this->currentStep->previousStep) {
$this->currentStep = $this->currentStep->previousStep;
$this->stepNumber--;
$form_state['values'] = $this->currentStep->getFormValues();
$form_state['rebuild'] = TRUE;
}
}
/**
* Build the form for the current step.
*
* @return array
*/
public function form(&$form_state) {
drupal_set_title(t('Import from @source_title',
array('@source_title' => $this->getSourceName())));
$form_method = $this->currentStep->getFormMethod();
$form['title'] = array(
'#prefix' => '<h2>',
'#markup' => t('Step @step: @step_name',
array(
'@step' => $this->stepNumber,
'@step_name' => $this->currentStep->getName(),
)),
'#suffix' => '</h2>',
);
$form += call_user_func_array($form_method, array(&$form_state));
$form['actions'] = array('#type' => 'actions');
// Show the 'previous' button if appropriate. Note that #submit is set to
// a special submit handler, and that we use #limit_validation_errors to
// skip all complaints about validation when using the back button. The
// values entered will be discarded, but they will not be validated, which
// would be annoying in a "back" button.
if ($this->currentStep != $this->firstStep) {
$form['actions']['prev'] = array(
'#type' => 'submit',
'#value' => t('Previous'),
'#name' => 'prev',
'#submit' => array('migrate_ui_wizard_previous_submit'),
'#limit_validation_errors' => array(),
);
}
// Show the Next button only if there are more steps defined.
if ($this->currentStep == $this->lastStep) {
$form['actions']['finish'] = array(
'#type' => 'submit',
'#value' => t('Save import settings'),
);
$form['actions']['migrate'] = array(
'#type' => 'submit',
'#value' => t('Save import settings and run import'),
'#submit' => array('migrate_ui_wizard_migrate_submit'),
);
}
else {
$form['actions']['next'] = array(
'#type' => 'submit',
'#value' => t('Next'),
'#name' => 'next',
'#submit' => array('migrate_ui_wizard_next_submit'),
'#validate' => array('migrate_ui_wizard_next_validate'),
);
}
return $form;
}
/**
* Call the validation function for the current form (which has the same
* name of the form function with 'Validate' appended).
*
* @param array $form_state
*/
public function formValidate(&$form_state) {
$validate_method = $this->currentStep->getFormMethod();
// This is an array for a method, or a function name.
if (is_array($validate_method)) {
$validate_method[1] .= 'Validate';
}
else {
$validate_method .= 'Validate';
}
if (is_callable($validate_method)) {
call_user_func_array($validate_method, array(&$form_state));
}
}
/**
* Take the information we've accumulated throughout the wizard, and create
* the Migrations to perform the import.
*/
public function formSaveSettings() {
MigrateGroup::register($this->groupName, $this->groupTitle, $this->groupArguments);
$info['arguments']['group_name'] = $this->groupName;
foreach ($this->migrations as $machine_name => $info) {
// Call the right registerMigration implementation. Note that this means
// that classes that override registerMigration() must handle registration
// themselves, they cannot leave it to us and expect their extension to be
// called.
if (is_subclass_of($info['class_name'], 'Migration')) {
Migration::registerMigration($info['class_name'], $machine_name,
$info['arguments']);
}
else {
MigrationBase::registerMigration($info['class_name'], $machine_name,
$info['arguments']);
}
};
menu_rebuild();
}
/**
* Run the import process for the migration group we've defined.
*/
public function formPerformImport() {
$migrations = migrate_migrations();
$operations = array();
/** @var Migration $migration */
foreach ($migrations as $migration) {
$group_name = $migration->getGroup()->getName();
if ($group_name == $this->groupName) {
$operations[] = array(
'migrate_ui_batch',
array('import', $migration->getMachineName(), NULL, 0),
);
}
}
if (count($operations) > 0) {
$batch = array(
'operations' => $operations,
'title' => t('Import processing'),
'file' => drupal_get_path('module', 'migrate_ui') . '/migrate_ui.pages.inc',
'init_message' => t('Starting import process'),
'progress_message' => t(''),
'error_message' => t('An error occurred. Some or all of the import processing has failed.'),
'finished' => 'migrate_ui_batch_finish',
);
batch_set($batch);
}
}
/**
* Record all the information necessary to register a migration when this is
* all over.
*
* @param string $machine_name
* Machine name for the migration class.
* @param string $class_name
* Name of the Migration class to instantiate.
* @param array $arguments
* Further information configuring the migration.
*/
public function addMigration($machine_name, $class_name, $arguments) {
// Give extenders an opportunity to modify or reject this migration.
foreach ($this->extenders as $extender) {
if (!$extender->addMigrationAlter($machine_name, $class_name, $arguments, $this)) {
return FALSE;
}
}
$machine_name = $this->groupName . $machine_name;
if (isset($arguments['dependencies'])) {
foreach ($arguments['dependencies'] as $index => $dependency) {
$arguments['dependencies'][$index] = $this->groupName . $dependency;
}
}
if (isset($arguments['soft_dependencies'])) {
foreach ($arguments['soft_dependencies'] as $index => $dependency) {
$arguments['soft_dependencies'][$index] = $this->groupName . $dependency;
}
}
$arguments += array(
'group_name' => $this->groupName,
'machine_name' => $machine_name,
);
$this->migrations[$machine_name] = array(
'class_name' => $class_name,
'arguments' => $arguments,
);
return TRUE;
}
}
/**
* Class representing one step of a wizard.
*/
class MigrateUIStep {
/**
* A translatable string briefly describing this step, to be used in the page
* title for the step form.
*
* @var string
*/
protected $name;
public function getName() {
return $this->name;
}
/**
* Callable that returns the form array for this step.
*
* @var string
*/
protected $formMethod;
public function getFormMethod() {
return $this->formMethod;
}
/**
* The form values ($form_state['values']) submitted for this step, saved in
* case we need to restore them on a Previous action.
*
* @var array
*/
protected $formValues;
public function getFormValues() {
return $this->formValues;
}
public function setFormValues($form_values) {
$this->formValues = $form_values;
}
/**
* Any contextual data needed by the form for this step. For example, a
* field mapping form would need to know the source and destination content
* types so it can determine what fields to expose.
*
* @var mixed
*/
protected $context;
public function getContext() {
return $this->context;
}
/**
* The step object is a node in a doubly-linked list - it links to its
* predecessor and successor steps.
*
* @var MigrateUIStep
*/
public $nextStep;
/**
* @var MigrateUIStep
*/
public $previousStep;
/**
* Class constructor.
*
* @param $name
* The machine name of the wizard step.
* @param $form_method
* A callable for the form array for this step. The validation method is
* formed from the method name with the suffix 'Validate' added, regardless
* of which object it is on.
* @param $context = NULL
* Contextual data needed by the form for this step.
*/
public function __construct($name, $form_method, $context = NULL) {
$this->name = $name;
$this->formMethod = $form_method;
$this->context = $context;
}
}
/**
*
*/
abstract class MigrateUIWizardExtender {
/**
* Reference to the wizard object that this extender applies to.
*/
protected $wizard;
/**
* Class constructor.
*
* Wizard extenders should override this to add their steps to the wizard.
*/
public function __construct(MigrateUIWizard $wizard, array $wizard_steps) {
$this->wizard = $wizard;
}
/**
* Alter the arguments to a migration before it is registered, or potentially
* reject it.
*
* @param string $machine_name
* Machine name for the migration class.
* @param string $class_name
* Name of the Migration class to instantiate.
* @param array $arguments
* Further information configuring the migration.
* @param MigrateUIWizard $wizard
* The wizard class performing the registration.
*
* @return bool
* Return FALSE to prevent registration of this migration.
*/
public function addMigrationAlter($machine_name, $class_name, &$arguments, $wizard) {
return TRUE;
}
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/block_custom.inc
+251
View File
@@ -0,0 +1,251 @@
<?php
/**
* @file
* Support for custom block destinations.
*/
/**
* Destination class implementing migration into {block_custom}.
*/
class MigrateDestinationCustomBlock extends MigrateDestination {
static public function getKeySchema() {
return array(
'bid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'ID of destination custom block',
),
);
}
public function __construct() {
parent::__construct();
}
public function __toString() {
$output = t('Custom blocks');
return $output;
}
/**
* Returns a list of fields available to be mapped for custom blocks.
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array(
'bid' => t('The custom block ID (bid).'),
'body' => t('Block contents.'),
'info' => t('Block description.'),
'format' => t('The {filter_format}.format of the block body.'),
);
return $fields;
}
/**
* Import a single row.
*
* @param $block
* Custom block object to build. Prefilled with any fields mapped in the
* Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $block, stdClass $row) {
// Updating previously-migrated content
if (isset($row->migrate_map_destid1)) {
$block->bid = $row->migrate_map_destid1;
}
// Load old values if necessary.
$migration = Migration::currentMigration();
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($block->bid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination bid provided'));
}
if (!$old_block = $this->loadCustomBlock($block->bid)) {
throw new MigrateException(t('System-of-record is DESTINATION, and the provided bid could not be found'));
}
$block_to_update = (object) $old_block;
foreach ($old_block as $key => $value) {
if (!isset($block->{$key})) {
$block->{$key} = $old_block[$key];
}
}
}
// Invoke migration prepare handlers
$this->prepare($block, $row);
// Custom blocks are handled as arrays, so clone the object to an array.
$item = clone $block;
$item = (array) $item;
migrate_instrument_start('block_custom_save');
// Check to see if this is a new custom block.
$update = FALSE;
if (isset($item['bid'])) {
$update = TRUE;
$bid = $this->saveCustomBlock($item);
}
else {
$bid = $this->saveCustomBlock($item);
}
migrate_instrument_stop('block_custom_save');
// Return the new id or FALSE on failure.
if (!empty($bid)) {
// Increment the count if the save succeeded.
if ($update) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
// Return the primary key to the mapping table.
$return = array($bid);
}
else {
$return = FALSE;
}
// Invoke migration complete handlers.
$block = (object) $this->loadCustomBlock($bid);
$this->complete($block, $row);
return $return;
}
/**
* Implementation of MigrateDestination::prepare().
*/
public function prepare($block, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$block->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('block_custom', 'prepare', $block, $row);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($block, $row);
}
}
/**
* Implementation of MigrateDestination::complete().
*/
public function complete($block, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$block->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('block_custom', 'complete', $block, $row);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'complete')) {
$migration->complete($block, $row);
}
}
/**
* Delete a batch of custom blocks at once.
*
* @param $bids
* Array of custom block IDs to be deleted.
*/
public function bulkRollback(array $bids) {
migrate_instrument_start('block_custom_delete_multiple');
$this->prepareRollback($bids);
$this->deleteMultipleCustomBlocks($bids);
$this->completeRollback($bids);
migrate_instrument_stop('block_custom_delete_multiple');
}
/**
* Give handlers a shot at cleaning up before a block has been rolled back.
*
* @param $bid
* ID of the custom block about to be deleted.
*/
public function prepareRollback($bid) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('block_custom', 'prepareRollback', $bid);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($bid);
}
}
/**
* Give handlers a shot at cleaning up after a block has been rolled back.
*
* @param $bid
* ID of the custom block which has been deleted.
*/
public function completeRollback($bid) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('block_custom', 'completeRollback', $bid);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($bid);
}
}
public function loadCustomBlock($bid) {
return block_custom_block_get($bid);
}
public function saveCustomBlock($block) {
drupal_alter('block_custom', $block);
if (!empty($block['bid'])) {
drupal_write_record('block_custom', $block, array('bid'));
module_invoke_all('block_custom_update', $block);
}
else {
drupal_write_record('block_custom', $block);
module_invoke_all('block_custom_insert', $block);
}
return $block['bid'];
}
public function deleteCustomBlock($bid) {
$this->deleteMultipleCustomBlocks(array($bid));
}
public function deleteMultipleCustomBlocks(array $bids) {
db_delete('block_custom')->condition('bid', $bids, 'IN')->execute();
db_delete('block')
->condition('module', 'block')
->condition('delta', $bids, 'IN')
->execute();
db_delete('block_role')
->condition('module', 'block')
->condition('delta', $bids, 'IN')
->execute();
db_delete('block_node_type')
->condition('module', 'block')
->condition('delta', $bids, 'IN')
->execute();
}
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/comment.inc
+99 -48
View File
@@ -12,6 +12,7 @@
* Destination class implementing migration into comments.
*/
class MigrateDestinationComment extends MigrateDestinationEntity {
static public function getKeySchema() {
return array(
'cid' => array(
@@ -24,6 +25,7 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
/**
* Save the original setting of comment_maintain_node_statistics
*
* @var boolean
*/
protected $maintainNodeStatistics;
@@ -58,6 +60,7 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -65,34 +68,34 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
public function fields($migration = NULL) {
$fields = array();
// First the core (comment table) properties
$fields['cid'] = t('Comment: <a href="@doc">Existing comment ID</a>',
array('@doc' => 'http://drupal.org/node/1349714#cid'));
$fields['nid'] = t('Comment: <a href="@doc">Node (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#nid'));
$fields['uid'] = t('Comment: <a href="@doc">User (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#uid'));
$fields['pid'] = t('Comment: <a href="@doc">Parent (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#pid'));
$fields['subject'] = t('Comment: <a href="@doc">Subject</a>',
array('@doc' => 'http://drupal.org/node/1349714#subject'));
$fields['created'] = t('Comment: <a href="@doc">Created timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349714#created'));
$fields['changed'] = t('Comment: <a href="@doc">Modified timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349714#changed'));
$fields['status'] = t('Comment: <a href="@doc">Status</a>',
array('@doc' => 'http://drupal.org/node/1349714#status'));
$fields['hostname'] = t('Comment: <a href="@doc">Hostname/IP address</a>',
array('@doc' => 'http://drupal.org/node/1349714#hostname'));
$fields['name'] = t('Comment: <a href="@doc">User name (not username)</a>',
array('@doc' => 'http://drupal.org/node/1349714#name'));
$fields['mail'] = t('Comment: <a href="@doc">Email address</a>',
array('@doc' => 'http://drupal.org/node/1349714#mail'));
$fields['homepage'] = t('Comment: <a href="@doc">Homepage</a>',
array('@doc' => 'http://drupal.org/node/1349714#homepage'));
$fields['language'] = t('Comment: <a href="@doc">Language</a>',
array('@doc' => 'http://drupal.org/node/1349714#language'));
$fields['thread'] = t('Comment: <a href="@doc">Thread</a>',
array('@doc' => 'http://drupal.org/node/1349714#thread'));
$fields['cid'] = t('<a href="@doc">Existing comment ID</a>',
array('@doc' => 'http://drupal.org/node/1349714#cid'));
$fields['nid'] = t('<a href="@doc">Node (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#nid'));
$fields['uid'] = t('<a href="@doc">User (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#uid'));
$fields['pid'] = t('<a href="@doc">Parent (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#pid'));
$fields['subject'] = t('<a href="@doc">Subject</a>',
array('@doc' => 'http://drupal.org/node/1349714#subject'));
$fields['created'] = t('<a href="@doc">Created timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349714#created'));
$fields['changed'] = t('<a href="@doc">Modified timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349714#changed'));
$fields['status'] = t('<a href="@doc">Status</a>',
array('@doc' => 'http://drupal.org/node/1349714#status'));
$fields['hostname'] = t('<a href="@doc">Hostname/IP address</a>',
array('@doc' => 'http://drupal.org/node/1349714#hostname'));
$fields['name'] = t('<a href="@doc">User name (not username)</a>',
array('@doc' => 'http://drupal.org/node/1349714#name'));
$fields['mail'] = t('<a href="@doc">Email address</a>',
array('@doc' => 'http://drupal.org/node/1349714#mail'));
$fields['homepage'] = t('<a href="@doc">Homepage</a>',
array('@doc' => 'http://drupal.org/node/1349714#homepage'));
$fields['language'] = t('<a href="@doc">Language</a>',
array('@doc' => 'http://drupal.org/node/1349714#language'));
$fields['thread'] = t('<a href="@doc">Thread</a>',
array('@doc' => 'http://drupal.org/node/1349714#thread'));
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
@@ -120,12 +123,14 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
* Import a single comment.
*
* @param $comment
* Comment object to build. Prefilled with any fields mapped in the Migration.
* Comment object to build. Prefilled with any fields mapped in the
* Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields (cid only in this case) of the comment that was saved if
* successful. FALSE on failure.
* Array of key fields (cid only in this case) of the comment that was saved
* if successful. FALSE on failure.
*/
public function import(stdClass $comment, stdClass $row) {
$migration = Migration::currentMigration();
@@ -134,7 +139,10 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
if (isset($comment->cid)) {
if ($comment->cid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming cid !cid and map destination nid !destid1 don't match",
array('!cid' => $comment->cid, '!destid1' => $row->migrate_map_destid1)));
array(
'!cid' => $comment->cid,
'!destid1' => $row->migrate_map_destid1,
)));
}
}
else {
@@ -149,6 +157,10 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
$comment->changed = MigrationBase::timestamp($comment->changed);
}
if (!isset($comment->node_type)) {
$comment->node_type = $this->bundle;
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($comment->cid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination cid provided'));
@@ -170,7 +182,7 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
}
$this->prepare($comment, $row);
foreach ($rawcomment as $field => $value) {
$old_comment->$field = $comment->$field;
$old_comment->{$field} = $comment->{$field};
}
$comment = $old_comment;
}
@@ -186,8 +198,8 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
'pid' => 0,
);
foreach ($defaults as $field => $value) {
if (!isset($comment->$field)) {
$comment->$field = $value;
if (!isset($comment->{$field})) {
$comment->{$field} = $value;
}
}
@@ -210,6 +222,10 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
else {
$updating = FALSE;
}
// Validate field data prior to saving.
MigrateDestinationEntity::fieldAttachValidate('comment', $comment);
migrate_instrument_start('comment_save');
comment_save($comment);
migrate_instrument_stop('comment_save');
@@ -265,8 +281,8 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
/**
* Updating node statistics on every comment imported or rolled back is
* expensive. We disable node statistics while performing imports and rollbacks,
* then re-enable and compute them in bulk when done.
* expensive. We disable node statistics while performing imports and
* rollbacks, then re-enable and compute them in bulk when done.
*/
protected function disableStatistics() {
// If maintaining node statistics is enabled, temporarily disable it
@@ -291,16 +307,47 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
// Empty table
db_truncate('node_comment_statistics')->execute();
// TODO: DBTNG. Ignore keyword is Mysql only? Is only used in the rare case when
// two comments on the same node share same timestamp.
$sql = "
INSERT IGNORE INTO {node_comment_statistics} (nid, cid, last_comment_timestamp, last_comment_name, last_comment_uid, comment_count) (
SELECT c.nid, c.cid, c.created, c.name, c.uid, c2.comment_count FROM {comment} c
JOIN (
SELECT c.nid, MAX(c.created) AS created, COUNT(*) AS comment_count FROM {comment} c WHERE status=:published GROUP BY c.nid
) AS c2 ON c.nid = c2.nid AND c.created=c2.created
)";
db_query($sql, array(':published' => COMMENT_PUBLISHED));
// DBTNG. IGNORE keyword is not compatible with Postgres. SQLite?
switch (db_driver()) {
case 'pgsql':
// We still want to run this under Postgres. On the very rare occasion
// when we have 2 comments on the same node with the same timestamp
// we will lose data.
$sql = "
INSERT INTO {node_comment_statistics} (nid, cid, last_comment_timestamp, last_comment_name, last_comment_uid, comment_count) (
SELECT c.nid, c.cid, c.created, c.name, c.uid, c2.comment_count
FROM {comment} c
JOIN (
SELECT c.nid, MAX(c.created) AS created, COUNT(*) AS comment_count
FROM {comment} c
WHERE status=:published
GROUP BY c.nid
) AS c2 ON c.nid = c2.nid AND c.created=c2.created
)";
break;
default:
$sql = "
INSERT IGNORE INTO {node_comment_statistics} (nid, cid, last_comment_timestamp, last_comment_name, last_comment_uid, comment_count) (
SELECT c.nid, c.cid, c.created, c.name, c.uid, c2.comment_count
FROM {comment} c
JOIN (
SELECT c.nid, MAX(c.created) AS created, COUNT(*) AS comment_count
FROM {comment} c
WHERE status=:published
GROUP BY c.nid
) AS c2 ON c.nid = c2.nid AND c.created=c2.created
)";
}
try {
db_query($sql, array(':published' => COMMENT_PUBLISHED));
} catch (Exception $e) {
// Our edge case has been hit. A Postgres migration has likely just
// lost data. Let the user know.
Migration::displayMessage(t('Failed to update node comment statistics: !message',
array('!message' => $e->getMessage())
));
}
// Insert records into the node_comment_statistics for nodes that are missing.
$query = db_select('node', 'n');
@@ -320,11 +367,15 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
}
class MigrateCommentNodeHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('node'));
}
public function fields($entity_type, $bundle) {
/**
* Implementation of MigrateDestinationHandler::fields().
*/
public function fields($entity_type, $bundle, $migration = NULL) {
$fields = array();
$fields['comment'] = t('Whether comments may be posted to the node');
return $fields;
sites/all/modules/contrib/migrate/migrate/plugins/destinations/entity.inc
+40 -4
View File
@@ -10,12 +10,14 @@
* Field API-related functions.
*/
abstract class MigrateDestinationEntity extends MigrateDestination {
/**
* The entity type (node, user, taxonomy_term, etc.) of the destination.
*
* @var string
*/
protected $entityType;
public function getEntityType() {
return $this->entityType;
}
@@ -26,6 +28,7 @@ abstract class MigrateDestinationEntity extends MigrateDestination {
* @var string
*/
protected $bundle;
public function getBundle() {
return $this->bundle;
}
@@ -36,6 +39,7 @@ abstract class MigrateDestinationEntity extends MigrateDestination {
* @var string
*/
protected $language;
public function getLanguage() {
return $this->language;
}
@@ -46,6 +50,7 @@ abstract class MigrateDestinationEntity extends MigrateDestination {
* @var int
*/
protected $textFormat;
public function getTextFormat() {
return $this->textFormat;
}
@@ -125,12 +130,12 @@ abstract class MigrateDestinationEntity extends MigrateDestination {
// Add source keys for debugging and identification of migrated data by hooks.
/* TODO: Restore
foreach ($migration->sourceKeyMap() as $field_name => $key_name) {
$keys[$key_name] = $source_row->$field_name;
$keys[$key_name] = $source_row->{$field_name};
}
*/
$migration = Migration::currentMigration();
$entity->migrate = array(
// 'source_keys' => $keys,
// 'source_keys' => $keys,
'machineName' => $migration->getMachineName(),
);
@@ -138,6 +143,11 @@ abstract class MigrateDestinationEntity extends MigrateDestination {
migrate_handler_invoke_all('Entity', 'prepare', $entity, $source_row);
// Then call any entity-specific handlers
migrate_handler_invoke_all($this->entityType, 'prepare', $entity, $source_row);
// Apply defaults, removing empty fields from the entity object.
$form = $form_state = array();
_field_invoke_default('submit', $this->entityType, $entity, $form, $form_state);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($entity, $source_row);
@@ -163,12 +173,38 @@ abstract class MigrateDestinationEntity extends MigrateDestination {
if (method_exists($migration, 'complete')) {
try {
$migration->complete($entity, $source_row);
}
catch (Exception $e) {
} catch (Exception $e) {
// If we catch any errors here, save the messages without letting
// the exception prevent the saving of the entity being recorded.
$migration->saveMessage($e->getMessage());
}
}
}
/**
* Perform field validation against the field data in an entity. Wraps
* field_attach_validate to handle exceptions cleanly and provide maximum
* information for identifying the cause of validation errors.
*
* @param $entity_type
* The type of $entity; e.g. 'node' or 'user'.
* @param $entity
* The entity with fields to validate.
*/
static public function fieldAttachValidate($entity_type, $entity) {
try {
field_attach_validate($entity_type, $entity);
} catch (FieldValidationException $e) {
$migration = Migration::currentMigration();
foreach ($e->errors as $field_name => $error_list) {
if (is_array($error_list)) {
foreach ($error_list as $index => $error) {
$message = $error['message'];
$migration->saveMessage(t('Field validation error for !field_name: !message',
array('!field_name' => $field_name, '!message' => $message)));
}
}
}
}
}
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/fields.inc
+226 -44
View File
@@ -6,21 +6,13 @@
*/
class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('entity'));
}
/**
* Implementation of MigrateDestinationHandler::fields().
*
* @param $entity_type
* The entity type (node, user, etc.) for which to list fields.
* @param $bundle
* The bundle (article, blog, etc.), if any, for which to list fields.
* @param Migration $migration
* Optionally, the migration providing the context.
* @return array
* An array keyed by field name, with field descriptions as values.
*/
public function fields($entity_type, $bundle, $migration = NULL) {
$fields = array();
@@ -29,22 +21,39 @@ class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
$field_info = field_info_field($machine_name);
$type = $field_info['type'];
$fields[$machine_name] = t('Field:') . ' ' . $instance['label'] .
' (' . $field_info['type'] . ')';
if (user_access(MIGRATE_ACCESS_ADVANCED)) {
$fields[$machine_name] = $instance['label'] . ' (' . $field_info['type'] . ')';
}
else {
$fields[$machine_name] = $instance['label'];
}
// Look for subfields
$class_list = _migrate_class_list('MigrateFieldHandler');
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
$fields_found = FALSE;
foreach ($class_list as $class_name => $handler) {
if (!in_array($class_name, $disabled) && $handler->handlesType($type)
&& method_exists($handler, 'fields')) {
&& method_exists($handler, 'fields')) {
migrate_instrument_start($class_name . '->fields');
$subfields = call_user_func(array($handler, 'fields'), $type,
$instance, $migration);
$instance, $migration);
migrate_instrument_stop($class_name . '->fields');
foreach ($subfields as $subfield_name => $subfield_label) {
$fields[$machine_name . ':' . $subfield_name] = $subfield_label;
}
$fields_found = TRUE;
}
}
if (!$fields_found) {
// Check the default field handler last.
migrate_instrument_start('MigrateDefaultFieldHandler->fields');
$subfields = call_user_func(
array(new MigrateDefaultFieldHandler, 'fields'), $type, $instance,
$migration);
migrate_instrument_stop('MigrateDefaultFieldHandler->fields');
foreach ($subfields as $subfield_name => $subfield_label) {
$fields[$machine_name . ':' . $subfield_name] = $subfield_label;
}
}
}
@@ -63,12 +72,12 @@ class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
foreach ($instances as $machine_name => $instance) {
if (property_exists($entity, $machine_name)) {
// Normalize to an array
if (!is_array($entity->$machine_name)) {
$entity->$machine_name = array($entity->$machine_name);
if (!is_array($entity->{$machine_name})) {
$entity->{$machine_name} = array($entity->{$machine_name});
}
$field_info = field_info_field($machine_name);
$entity->$machine_name = migrate_field_handler_invoke_all($entity, $field_info,
$instance, $entity->$machine_name);
$entity->{$machine_name} = migrate_field_handler_invoke_all($entity, $field_info,
$instance, $entity->{$machine_name});
}
}
migrate_instrument_stop('MigrateDestinationEntity->prepareFields');
@@ -85,12 +94,12 @@ class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
foreach ($instances as $machine_name => $instance) {
if (property_exists($entity, $machine_name)) {
// Normalize to an array
if (!is_array($entity->$machine_name)) {
$entity->$machine_name = array($entity->$machine_name);
if (!is_array($entity->{$machine_name})) {
$entity->{$machine_name} = array($entity->{$machine_name});
}
$field_info = field_info_field($machine_name);
migrate_field_handler_invoke_all($entity, $field_info,
$instance, $entity->$machine_name, 'complete');
$instance, $entity->{$machine_name}, 'complete');
}
}
migrate_instrument_stop('MigrateDestinationEntity->completeFields');
@@ -98,6 +107,7 @@ class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
}
abstract class MigrateFieldHandler extends MigrateHandler {
// Derived classes are expected to implement one or both of the prepare/complete
// handlers.
@@ -110,12 +120,14 @@ abstract class MigrateFieldHandler extends MigrateHandler {
* @param $entity
* @param $field_info
* @param $arguments
*
* @return string language code
*/
function getFieldLanguage($entity, $field_info, array $arguments) {
$migration = Migration::currentMigration();
switch (TRUE) {
case !field_is_translatable($migration->getDestination()->getEntityType(), $field_info):
case !field_is_translatable($migration->getDestination()
->getEntityType(), $field_info):
return LANGUAGE_NONE;
case isset($arguments['language']):
return $arguments['language'];
@@ -128,11 +140,103 @@ abstract class MigrateFieldHandler extends MigrateHandler {
}
}
/**
* A fallback field handler to do basic copying of field data.
*/
class MigrateDefaultFieldHandler extends MigrateFieldHandler {
public function __construct() {
}
/**
* Implements MigrateFieldHandler::fields().
*
* @param $field_type
* @param $field_instance
*
* @return array
*/
public function fields($field_type, $field_instance) {
$field_info = field_info_field($field_instance['field_name']);
$fields = array();
$first = TRUE;
foreach ($field_info['columns'] as $column_name => $column_info) {
// The first column is the primary value, which is mapped directly to
// the field name - so, don't include it here among the subfields.
if ($first) {
$first = FALSE;
}
else {
$fields[$column_name] = empty($column_info['description']) ?
$column_name : $column_info['description'];
}
}
return $fields;
}
/**
* Implements MigrateFieldHandler::prepare().
*
* @param $entity
* @param array $field_info
* @param array $instance
* @param array $values
*
* @return null
*/
public function prepare($entity, array $field_info, array $instance,
array $values) {
$arguments = array();
if (isset($values['arguments'])) {
$arguments = array_filter($values['arguments']);
unset($values['arguments']);
}
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
// Get the name of the primary (first) column, which is mapped separately.
reset($field_info['columns']);
$primary_column = key($field_info['columns']);
// Setup the standard Field API array for saving.
$delta = 0;
foreach ($values as $value) {
// Handle multivalue arguments (especially for subfields).
$delta_arguments = array();
foreach ($arguments as $name => $argument) {
if (is_array($argument) && array_key_exists($delta, $argument)) {
$delta_arguments[$name] = $argument[$delta];
}
else {
$delta_arguments[$name] = $argument;
}
}
$return[$language][$delta] = array($primary_column => $value) +
array_intersect_key($delta_arguments, $field_info['columns']);
$delta++;
}
return isset($return) ? $return : NULL;
}
/**
* Overrides MigrateHandler::handlesType().
*
* @param string $type
*
* @return bool
*/
public function handlesType($type) {
// We claim to handle any type.
return TRUE;
}
}
/**
* Base class for creating field handlers for fields with a single value.
*
* To use this class just extend it and pass key where the field's value should
* be stored to the constructor, then register the type(s):
*
* @code
* class MigrateLinkFieldHandler extends MigrateSimpleFieldHandler {
* public function __construct() {
@@ -143,6 +247,7 @@ abstract class MigrateFieldHandler extends MigrateHandler {
* @endcode
*/
abstract class MigrateSimpleFieldHandler extends MigrateFieldHandler {
protected $fieldValueKey = 'value';
protected $skipEmpty = FALSE;
@@ -197,6 +302,7 @@ abstract class MigrateSimpleFieldHandler extends MigrateFieldHandler {
* Returns TRUE only for values which are not NULL.
*
* @param $value
*
* @return bool
*/
protected function notNull($value) {
@@ -205,6 +311,7 @@ abstract class MigrateSimpleFieldHandler extends MigrateFieldHandler {
}
class MigrateTextFieldHandler extends MigrateFieldHandler {
public function __construct() {
$this->registerTypes(array('text', 'text_long', 'text_with_summary'));
}
@@ -233,20 +340,28 @@ class MigrateTextFieldHandler extends MigrateFieldHandler {
* @param Migration $migration
* The migration context for the parent field. We can look at the mappings
* and determine which subfields are relevant.
*
* @return array
*/
public function fields($type, $instance, $migration = NULL) {
$fields = array();
if ($type == 'text_with_summary') {
$fields['summary'] = t('Subfield: Summary of field contents');
$fields['summary'] = t('Subfield: <a href="@doc">Summary of field contents</a>',
array('@doc' => 'http://drupal.org/node/1224042#summary'));
}
if ($instance['settings']['text_processing']) {
$fields['format'] = t('Subfield: Text format for the field');
$fields['format'] = t('Subfield: <a href="@doc">Text format for the field</a>',
array('@doc' => 'http://drupal.org/node/1224042#format'));
}
$field = field_info_field($instance['field_name']);
if (field_is_translatable($instance['entity_type'], $field)) {
$fields['language'] = t('Subfield: <a href="@doc">Language for the field</a>',
array('@doc' => 'http://drupal.org/node/1224042#language'));
}
$fields['language'] = t('Subfield: Language for the field');
return $fields;
}
public function prepare($entity, array $field_info, array $instance, array $values) {
if (isset($values['arguments'])) {
$arguments = $values['arguments'];
@@ -278,7 +393,7 @@ class MigrateTextFieldHandler extends MigrateFieldHandler {
if (is_array($arguments['format'])) {
$format = $arguments['format'][$delta];
}
else{
else {
$format = $arguments['format'];
}
}
@@ -289,6 +404,11 @@ class MigrateTextFieldHandler extends MigrateFieldHandler {
$format = NULL;
}
$item['format'] = $item['value_format'] = $format;
// If the value is an array, which it might be if the destination entity
// was loaded via entity_load(), ensure we're handling a string.
if (is_array($value) && !empty($value[0]) && array_key_exists('value', $value[0])) {
$value = $value[0]['value'];
}
// Make sure the value will fit
if ($max_length) {
$item['value'] = drupal_substr($value, 0, $max_length);
@@ -297,8 +417,11 @@ class MigrateTextFieldHandler extends MigrateFieldHandler {
if ($value_length > $max_length) {
$migration->saveMessage(
t('Value for field !field exceeds max length of !max_length, actual length is !length',
array('!field' => $instance['field_name'], '!max_length' => $max_length,
'!length' => $value_length)),
array(
'!field' => $instance['field_name'],
'!max_length' => $max_length,
'!length' => $value_length,
)),
Migration::MESSAGE_INFORMATIONAL);
}
}
@@ -322,17 +445,28 @@ class MigrateTextFieldHandler extends MigrateFieldHandler {
}
class MigrateValueFieldHandler extends MigrateSimpleFieldHandler {
public function __construct() {
parent::__construct(array(
'value_key' => 'value',
'skip_empty' => FALSE,
));
$this->registerTypes(array('value', 'list', 'list_boolean', 'list_integer',
'list_float', 'list_text', 'number_integer', 'number_decimal', 'number_float'));
$this->registerTypes(array(
'value',
'list',
'list_boolean',
'list_integer',
'list_float',
'list_text',
'number_integer',
'number_decimal',
'number_float',
));
}
}
class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
public function __construct() {
$this->registerTypes(array('taxonomy_term_reference'));
}
@@ -347,13 +481,17 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
* @param Migration $migration
* The migration context for the parent field. We can look at the mappings
* and determine which subfields are relevant.
*
* @return array
*/
public function fields($type, $instance, $migration = NULL) {
return array(
'source_type' => t('Option: Set to \'tid\' when the value is a source ID'),
'create_term' => t('Option: Set to TRUE to create referenced terms when necessary'),
'ignore_case' => t('Option: Set to TRUE to ignore case differences between source data and existing term names'),
'source_type' => t('Option: <a href="@doc">Set to \'tid\' when the value is a source ID</a>',
array('@doc' => 'http://drupal.org/node/1224042#source_type')),
'create_term' => t('Option: <a href="@doc">Set to TRUE to create referenced terms when necessary</a>',
array('@doc' => 'http://drupal.org/node/1224042#create_term')),
'ignore_case' => t('Option: <a href="@doc">Set to TRUE to ignore case differences between source data and existing term names</a>',
array('@doc' => 'http://drupal.org/node/1224042#ignore_case')),
);
}
@@ -365,7 +503,7 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
else {
$arguments = array();
}
if (empty($values[0])) {
if (count($values) == 1 && empty($values[0])) {
$values = array();
}
@@ -375,6 +513,7 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
$tids = $values;
}
elseif ($values) {
$vocab_name = $field_info['settings']['allowed_values'][0]['vocabulary'];
$names = taxonomy_vocabulary_get_names();
// Get the vocabulary for this term
@@ -382,10 +521,12 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
$vid = $field_info['settings']['allowed_values'][0]['vid'];
}
else {
$vocab_name = $field_info['settings']['allowed_values'][0]['vocabulary'];
$vid = $names[$vocab_name]->vid;
}
// Remove leading and trailing spaces in term names
$values = array_map('trim', $values);
// Cannot use taxonomy_term_load_multiple() since we have an array of names.
// It wants a singular value. This query may return case-insensitive
// matches.
@@ -398,17 +539,33 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
// If we're ignoring case, change both the matched term name keys and the
// source values to lowercase.
if (isset($arguments['ignore_case']) && $arguments['ignore_case']) {
$ignore_case = TRUE;
$existing_terms = array_change_key_case($existing_terms);
$values = array_map('strtolower', $values);
foreach ($values as $value) {
$lower_values[$value] = strtolower($value);
}
}
else {
$ignore_case = FALSE;
}
foreach ($values as $value) {
if (isset($existing_terms[$value])) {
$tids[] = $existing_terms[$value];
}
elseif ($ignore_case && isset($existing_terms[$lower_values[$value]])) {
$tids[] = $existing_terms[$lower_values[$value]];
}
elseif (!empty($arguments['create_term'])) {
$new_term = new stdClass();
$new_term->vid = $vid;
$new_term->name = $value;
$new_term->vocabulary_machine_name = $vocab_name;
// This term is being created with no fields, but we should still call
// field_attach_validate() before saving, as that invokes
// hook_field_attach_validate().
MigrateDestinationEntity::fieldAttachValidate('taxonomy_term', $new_term);
taxonomy_term_save($new_term);
$tids[] = $new_term->tid;
// Add newly created term to existing array.
@@ -449,6 +606,7 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
* dealing with the file entity to an embedded MigrateFileInterface instance.
*/
abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
/**
* Implementation of MigrateFieldHandler::fields().
*
@@ -459,15 +617,20 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
* @param Migration $migration
* The migration context for the parent field. We can look at the mappings
* and determine which subfields are relevant.
*
* @return array
*/
public function fields($type, $instance, $migration = NULL) {
$fields = array(
'file_class' => t('Option: <a href="@doc">Implementation of MigrateFile to use</a>',
array('@doc' => 'http://drupal.org/node/1540106#file_class')),
'language' => t('Subfield: Language for the field'),
);
$field = field_info_field($instance['field_name']);
if (field_is_translatable($instance['entity_type'], $field)) {
$fields['language'] = t('Subfield: Language for the field');
}
// If we can identify the file class mapped to this field, pick up the
// subfields specific to that class.
if ($migration) {
@@ -478,7 +641,7 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
$file_class = $mapping->getDefaultValue();
}
}
if (!isset($file_class)) {
if (empty($file_class)) {
$file_class = 'MigrateFileUri';
}
$fields += call_user_func(array($file_class, 'fields'));
@@ -502,7 +665,7 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
$arguments = array();
}
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
$default_language = $this->getFieldLanguage($entity, $field_info, $arguments);
$migration = Migration::currentMigration();
// One can override the source class via CLI or drushrc.php (the
@@ -543,8 +706,12 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
}
else {
$migration->saveMessage(
t('No data for subfield %key at row %delta for field %field',
array('%key' => $key, '%delta' => $delta, '%field' => $field_info['field_name'])),
t('No data for subfield %key at row %delta for field %field',
array(
'%key' => $key,
'%delta' => $delta,
'%field' => $field_info['field_name'],
)),
Migration::MESSAGE_WARNING);
}
}
@@ -561,6 +728,10 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
// MigrateFile class has saved a message indicating why.
if ($file) {
$field_array = array('fid' => $file->fid);
$language = isset($instance_arguments['language']) ? $instance_arguments['language'] : $default_language;
if (is_array($language)) {
$language = $language[$delta];
}
$return[$language][] = $this->buildFieldArray($field_array, $instance_arguments, $delta);
}
}
@@ -576,6 +747,7 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
* Field API info on the general field.
* @param $instance
* Field API info on the field instance for this entity type.
*
* @return string
* Directory relative to the Drupal public files directory.
*/
@@ -609,6 +781,7 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
* Handle for file fields.
*/
class MigrateFileFieldHandler extends MigrateFileFieldBaseHandler {
public function __construct() {
$this->registerTypes(array('file'));
}
@@ -624,13 +797,16 @@ class MigrateFileFieldHandler extends MigrateFileFieldBaseHandler {
* @param Migration $migration
* The migration context for the parent field. We can look at the mappings
* and determine which subfields are relevant.
*
* @return array
*/
public function fields($type, $instance, $migration = NULL) {
$fields = parent::fields($type, $instance, $migration);
$fields += array(
'description' => t('Subfield: String to be used as the description value'),
'display' => t('Subfield: String to be used as the display value'),
'description' => t('Subfield: <a href="@doc">String to be used as the description value</a>',
array('@doc' => 'http://drupal.org/node/1224042#description')),
'display' => t('Subfield: <a href="@doc">String to be used as the display value</a>',
array('@doc' => 'http://drupal.org/node/1224042#display')),
);
return $fields;
}
@@ -670,6 +846,7 @@ class MigrateFileFieldHandler extends MigrateFileFieldBaseHandler {
* Handle for image fields;
*/
class MigrateImageFieldHandler extends MigrateFileFieldBaseHandler {
public function __construct() {
$this->registerTypes(array('image'));
}
@@ -685,13 +862,16 @@ class MigrateImageFieldHandler extends MigrateFileFieldBaseHandler {
* @param Migration $migration
* The migration context for the parent field. We can look at the mappings
* and determine which subfields are relevant.
*
* @return array
*/
public function fields($type, $instance, $migration = NULL) {
$fields = parent::fields($type, $instance, $migration);
$fields += array(
'alt' => t('Subfield: String to be used as the alt value'),
'title' => t('Subfield: String to be used as the title value'),
'alt' => t('Subfield: <a href="@doc">String to be used as the alt value</a>',
array('@doc' => 'http://drupal.org/node/1224042#alt')),
'title' => t('Subfield: <a href="@doc">String to be used as the title value</a>',
array('@doc' => 'http://drupal.org/node/1224042#title')),
);
return $fields;
}
@@ -721,6 +901,7 @@ class MigrateImageFieldHandler extends MigrateFileFieldBaseHandler {
}
class MigrateNodeReferenceFieldHandler extends MigrateSimpleFieldHandler {
public function __construct() {
parent::__construct(array(
'value_key' => 'nid',
@@ -735,6 +916,7 @@ class MigrateNodeReferenceFieldHandler extends MigrateSimpleFieldHandler {
}
class MigrateUserReferenceFieldHandler extends MigrateSimpleFieldHandler {
public function __construct() {
parent::__construct(array(
'value_key' => 'uid',
sites/all/modules/contrib/migrate/migrate/plugins/destinations/file.inc
+206 -117
View File
@@ -12,6 +12,7 @@
* a Drupal file entity (creating the entity if necessary).
*/
interface MigrateFileInterface {
/**
* Return a list of subfields and options specific to this implementation,
* keyed by name.
@@ -35,13 +36,112 @@ interface MigrateFileInterface {
}
abstract class MigrateFileBase implements MigrateFileInterface {
/**
* Extension of the core FILE_EXISTS_* constants, offering an alternative to
* reuse the existing file if present as-is (core only offers the options of
* replacing it or renaming to avoid collision).
*/
const FILE_EXISTS_REUSE = -1;
/**
* An optional file object to use as a default starting point for building the
* file entity.
*
* @var stdClass
*/
protected $defaultFile;
/**
* How to handle destination filename collisions.
*
* @var int
*/
protected $fileReplace = FILE_EXISTS_RENAME;
/**
* Set to TRUE to prevent file deletion on rollback.
*
* @var bool
*/
protected $preserveFiles = FALSE;
public function __construct($arguments = array(), $default_file = NULL) {
if (isset($arguments['preserve_files'])) {
$this->preserveFiles = $arguments['preserve_files'];
}
if (isset($arguments['file_replace'])) {
$this->fileReplace = $arguments['file_replace'];
}
if ($default_file) {
$this->defaultFile = $default_file;
}
else {
$this->defaultFile = new stdClass;
}
}
/**
* Default implementation of MigrateFileInterface::fields().
*
* @return array
*/
static public function fields() {
return array();
return array(
'preserve_files' => t('Option: <a href="@doc">Boolean indicating whether files should be preserved or deleted on rollback</a>',
array('@doc' => 'http://drupal.org/node/1540106#preserve_files')),
);
}
/**
* Setup a file entity object suitable for saving.
*
* @param $destination
* Path to the Drupal copy of the file.
* @param $owner
* Uid of the file owner.
*
* @return stdClass
* A file object ready to be saved.
*/
protected function createFileEntity($destination, $owner) {
$file = clone $this->defaultFile;
$file->uri = $destination;
$file->uid = $owner;
if (!isset($file->filename)) {
$file->filename = drupal_basename($destination);
}
if (!isset($file->filemime)) {
$file->filemime = file_get_mimetype(urldecode($destination));
}
if (!isset($file->status)) {
$file->status = FILE_STATUS_PERMANENT;
}
if (empty($file->type) || $file->type == 'file') {
// Try to determine the file type.
if (module_exists('file_entity')) {
$type = file_get_type($file);
}
elseif ($slash_pos = strpos($file->filemime, '/')) {
$type = substr($file->filemime, 0, $slash_pos);
}
$file->type = isset($type) ? $type : 'file';
}
// If we are replacing or reusing an existing filesystem entry,
// also re-use its database record.
if ($this->fileReplace == FILE_EXISTS_REPLACE ||
$this->fileReplace == self::FILE_EXISTS_REUSE) {
$existing_files = file_load_multiple(array(), array('uri' => $destination));
if (count($existing_files)) {
$existing = reset($existing_files);
$file->fid = $existing->fid;
$file->filename = $existing->filename;
}
}
return $file;
}
/**
@@ -68,10 +168,24 @@ abstract class MigrateFileBase implements MigrateFileInterface {
}
}
/**
* The simplest possible file class - where the value is a remote URI which
* simply needs to be saved as the URI on the destination side, with no attempt
* to copy or otherwise use it.
*/
class MigrateFileUriAsIs extends MigrateFileBase {
public function processFile($value, $owner) {
$file = file_save($this->createFileEntity($value, $owner));
return $file;
}
}
/**
* Handle the degenerate case where we already have a file ID.
*/
class MigrateFileFid extends MigrateFileBase {
/**
* Implementation of MigrateFileInterface::processFile().
*
@@ -79,6 +193,7 @@ class MigrateFileFid extends MigrateFileBase {
* An existing file entity ID (fid).
* @param $owner
* User ID (uid) to be the owner of the file. Ignored in this case.
*
* @return int
* The file entity corresponding to the fid that was passed in.
*/
@@ -92,12 +207,6 @@ class MigrateFileFid extends MigrateFileBase {
* Base class for creating core file entities.
*/
abstract class MigrateFile extends MigrateFileBase {
/**
* Extension of the core FILE_EXISTS_* constants, offering an alternative to
* reuse the existing file if present as-is (core only offers the options of
* replacing it or renaming to avoid collision).
*/
const FILE_EXISTS_REUSE = -1;
/**
* The destination directory within Drupal.
@@ -113,47 +222,14 @@ abstract class MigrateFile extends MigrateFileBase {
*/
protected $destinationFile = '';
/**
* How to handle destination filename collisions.
*
* @var int
*/
protected $fileReplace = FILE_EXISTS_RENAME;
/**
* Set to TRUE to prevent file deletion on rollback.
*
* @var bool
*/
protected $preserveFiles = FALSE;
/**
* An optional file object to use as a default starting point for building the
* file entity.
*
* @var stdClass
*/
protected $defaultFile;
public function __construct($arguments = array(), $default_file = NULL) {
parent::__construct($arguments, $default_file);
if (isset($arguments['destination_dir'])) {
$this->destinationDir = $arguments['destination_dir'];
}
if (isset($arguments['destination_file'])) {
$this->destinationFile = $arguments['destination_file'];
}
if (isset($arguments['file_replace'])) {
$this->fileReplace = $arguments['file_replace'];
}
if (isset($arguments['preserve_files'])) {
$this->preserveFiles = $arguments['preserve_files'];
}
if ($default_file) {
$this->defaultFile = $default_file;
}
else {
$this->defaultFile = new stdClass;
}
}
/**
@@ -162,53 +238,14 @@ abstract class MigrateFile extends MigrateFileBase {
* @return array
*/
static public function fields() {
return array(
'destination_dir' => t('Subfield: <a href="@doc">Path within Drupal files directory to store file</a>',
array('@doc' => 'http://drupal.org/node/1540106#destination_dir')),
'destination_file' => t('Subfield: <a href="@doc">Path within destination_dir to store the file.</a>',
array('@doc' => 'http://drupal.org/node/1540106#destination_file')),
'file_replace' => t('Option: <a href="@doc">Value of $replace in that file function. Does not apply to file_fast(). Defaults to FILE_EXISTS_RENAME.</a>',
array('@doc' => 'http://drupal.org/node/1540106#file_replace')),
'preserve_files' => t('Option: <a href="@doc">Boolean indicating whether files should be preserved or deleted on rollback</a>',
array('@doc' => 'http://drupal.org/node/1540106#preserve_files')),
);
}
/**
* Setup a file entity object suitable for saving.
*
* @param $destination
* Path to the Drupal copy of the file.
* @param $owner
* Uid of the file owner.
* @return stdClass
* A file object ready to be saved.
*/
protected function createFileEntity($destination, $owner) {
$file = clone $this->defaultFile;
$file->uri = $destination;
$file->uid = $owner;
if (!isset($file->filename)) {
$file->filename = drupal_basename($destination);
}
if (!isset($file->filemime)) {
$file->filemime = file_get_mimetype($destination);
}
if (!isset($file->status)) {
$file->status = FILE_STATUS_PERMANENT;
}
// If we are replacing or reusing an existing filesystem entry,
// also re-use its database record.
if ($this->fileReplace == FILE_EXISTS_REPLACE ||
$this->fileReplace == self::FILE_EXISTS_REUSE) {
$existing_files = file_load_multiple(array(), array('uri' => $destination));
if (count($existing_files)) {
$existing = reset($existing_files);
$file->fid = $existing->fid;
$file->filename = $existing->filename;
}
}
return $file;
return parent::fields() + array(
'destination_dir' => t('Subfield: <a href="@doc">Path within Drupal files directory to store file</a>',
array('@doc' => 'http://drupal.org/node/1540106#destination_dir')),
'destination_file' => t('Subfield: <a href="@doc">Path within destination_dir to store the file.</a>',
array('@doc' => 'http://drupal.org/node/1540106#destination_file')),
'file_replace' => t('Option: <a href="@doc">Value of $replace in that file function. Defaults to FILE_EXISTS_RENAME.</a>',
array('@doc' => 'http://drupal.org/node/1540106#file_replace')),
);
}
/**
@@ -216,6 +253,7 @@ abstract class MigrateFile extends MigrateFileBase {
*
* @param $destination
* Destination path within Drupal.
*
* @return bool
* TRUE if the file is successfully saved, FALSE otherwise.
*/
@@ -228,6 +266,7 @@ abstract class MigrateFile extends MigrateFileBase {
* The URI or local filespec of a file to be imported.
* @param $owner
* User ID (uid) to be the owner of the file.
*
* @return object
* The file entity being created or referenced.
*/
@@ -242,13 +281,10 @@ abstract class MigrateFile extends MigrateFileBase {
// Our own file_replace behavior - if the file exists, use it without
// replacing it
if ($this->fileReplace == self::FILE_EXISTS_REUSE) {
// See if we this file already (we'll reuse a file entity if it exists).
// See if we this file already (we'll reuse and resave a file entity if it exists).
if (file_exists($destination)) {
$file = $this->createFileEntity($destination, $owner);
// File entity didn't already exist, create it
if (empty($file->fid)) {
$file = file_save($file);
}
$file = file_save($file);
$this->markForPreservation($file->fid);
return $file;
}
@@ -257,18 +293,18 @@ abstract class MigrateFile extends MigrateFileBase {
}
// Prepare the destination directory.
if (!file_prepare_directory(drupal_dirname($destination),
FILE_CREATE_DIRECTORY | FILE_MODIFY_PERMISSIONS)) {
$destdir = drupal_dirname($destination);
if (!file_prepare_directory($destdir,
FILE_CREATE_DIRECTORY | FILE_MODIFY_PERMISSIONS)) {
$migration->saveMessage(t('Could not create destination directory for !dest',
array('!dest' => $destination)));
array('!dest' => $destination)));
return FALSE;
}
// Determine whether we can perform this operation based on overwrite rules.
$destination = file_destination($destination, $this->fileReplace);
if ($destination === FALSE) {
$migration->saveMessage(t('The file could not be copied because ' .
'file %dest already exists in the destination directory.',
$migration->saveMessage(t('The file could not be copied because file %dest already exists in the destination directory.',
array('%dest' => $destination)));
return FALSE;
}
@@ -303,6 +339,7 @@ abstract class MigrateFile extends MigrateFileBase {
* to be imported to Drupal.
*/
class MigrateFileUri extends MigrateFile {
/**
* The source directory for the file, relative to which the value (source
* file) will be taken.
@@ -318,11 +355,19 @@ class MigrateFileUri extends MigrateFile {
*/
protected $sourcePath = '';
/**
* Whether to apply rawurlencode to the components of an incoming file path.
*/
protected $urlEncode = TRUE;
public function __construct($arguments = array(), $default_file = NULL) {
parent::__construct($arguments, $default_file);
if (isset($arguments['source_dir'])) {
$this->sourceDir = rtrim($arguments['source_dir'], "/\\");
}
if (isset($arguments['urlencode'])) {
$this->urlEncode = $arguments['urlencode'];
}
}
/**
@@ -335,6 +380,8 @@ class MigrateFileUri extends MigrateFile {
array(
'source_dir' => t('Subfield: <a href="@doc">Path to source file.</a>',
array('@doc' => 'http://drupal.org/node/1540106#source_dir')),
'urlencode' => t('Option: <a href="@doc">Encode all segments of the incoming path (defaults to TRUE).</a>',
array('@doc' => 'http://drupal.org/node/1540106#urlencode')),
);
}
@@ -348,17 +395,27 @@ class MigrateFileUri extends MigrateFile {
* TRUE if the copy succeeded, FALSE otherwise.
*/
protected function copyFile($destination) {
// Perform the copy operation, with a cleaned-up path.
$this->sourcePath = self::urlencode($this->sourcePath);
if (!@copy($this->sourcePath, $destination)) {
$migration = Migration::currentMigration();
$migration->saveMessage(t('The specified file %file could not be copied to ' .
'%destination.',
array('%file' => $this->sourcePath, '%destination' => $destination)));
return FALSE;
if ($this->urlEncode) {
// Perform the copy operation, with a cleaned-up path.
$this->sourcePath = self::urlencode($this->sourcePath);
}
else {
return TRUE;
try {
$copied = copy($this->sourcePath, $destination);
if ($copied == FALSE) {
$migration = Migration::currentMigration();
$migration->saveMessage(t('The specified file %file could not be copied to %destination',
array('%file' => $this->sourcePath, '%destination' => $destination)));
}
return $copied;
} catch (Exception $e) {
$migration = Migration::currentMigration();
$migration->saveMessage(t('The specified file %file could not be copied to %destination: "%exception_msg"',
array(
'%file' => $this->sourcePath,
'%destination' => $destination,
'%exception_msg' => $e->getMessage(),
)));
return FALSE;
}
}
@@ -377,8 +434,10 @@ class MigrateFileUri extends MigrateFile {
$components[$key] = rawurlencode($component);
}
$filename = implode('/', $components);
// Actually, we don't want colons encoded
// Actually, we don't want certain characters encoded
$filename = str_replace('%3A', ':', $filename);
$filename = str_replace('%3F', '?', $filename);
$filename = str_replace('%26', '&', $filename);
}
return $filename;
}
@@ -390,6 +449,7 @@ class MigrateFileUri extends MigrateFile {
* The URI or local filespec of a file to be imported.
* @param $owner
* User ID (uid) to be the owner of the file.
*
* @return object
* The file entity being created or referenced.
*/
@@ -419,6 +479,7 @@ class MigrateFileUri extends MigrateFile {
* such as image data) to be stored as a real file in Drupal.
*/
class MigrateFileBlob extends MigrateFile {
/**
* The file contents we will be writing to a real file.
*
@@ -431,6 +492,7 @@ class MigrateFileBlob extends MigrateFile {
*
* @param $destination
* Drupal destination path.
*
* @return bool
* TRUE if the file contents were successfully written, FALSE otherwise.
*/
@@ -441,7 +503,7 @@ class MigrateFileBlob extends MigrateFile {
else {
$migration = Migration::currentMigration();
$migration->saveMessage(t('Failed to write blob data to %destination',
array('%destination' => $destination)));
array('%destination' => $destination)));
return FALSE;
}
}
@@ -453,6 +515,7 @@ class MigrateFileBlob extends MigrateFile {
* The file contents to be saved as a file.
* @param $owner
* User ID (uid) to be the owner of the file.
*
* @return object
* File entity being created or referenced.
*/
@@ -467,6 +530,7 @@ class MigrateFileBlob extends MigrateFile {
* Destination class implementing migration into the files table.
*/
class MigrateDestinationFile extends MigrateDestinationEntity {
/**
* File class (MigrateFileUri etc.) doing the dirty wrk.
*
@@ -474,6 +538,10 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
*/
protected $fileClass;
public function setFileClass($file_class) {
$this->fileClass = $file_class;
}
/**
* Boolean indicating whether we should avoid deleting the actual file on
* rollback.
@@ -510,10 +578,12 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
}
/**
* Returns a list of fields available to be mapped for the entity type (bundle)
* Returns a list of fields available to be mapped for the entity type
* (bundle)
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -521,10 +591,10 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
public function fields($migration = NULL) {
$fields = array();
// First the core properties
$fields['fid'] = t('File: Existing file ID');
$fields['uid'] = t('File: Uid of user associated with file');
$fields['value'] = t('File: Representation of the source file (usually a URI)');
$fields['timestamp'] = t('File: UNIX timestamp for the date the file was added');
$fields['fid'] = t('Existing file ID');
$fields['uid'] = t('Uid of user associated with file');
$fields['value'] = t('Representation of the source file (usually a URI)');
$fields['timestamp'] = t('UNIX timestamp for the date the file was added');
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
@@ -559,12 +629,14 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
else {
$preserve_files = FALSE;
}
$this->prepareRollback($fid);
if ($preserve_files) {
$this->fileDelete($file);
}
else {
file_delete($file, TRUE);
}
$this->completeRollback($fid);
migrate_instrument_stop('file_delete');
}
}
@@ -589,6 +661,7 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
* File object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields (fid only in this case) of the file that was saved if
* successful. FALSE on failure.
@@ -600,7 +673,10 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
if (isset($file->fid)) {
if ($file->fid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming fid !fid and map destination fid !destid1 don't match",
array('!fid' => $file->fid, '!destid1' => $row->migrate_map_destid1)));
array(
'!fid' => $file->fid,
'!destid1' => $row->migrate_map_destid1,
)));
}
}
else {
@@ -616,6 +692,19 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
$old_file = file_load($file->fid);
}
// 'type' is the bundle property on file entities. It must be set here for
// the sake of the prepare handlers, although it may be overridden later
// based on the detected mime type.
if (empty($file->type)) {
// If a bundle was specified in the constructor we use it for filetype.
if ($this->bundle != 'file') {
$file->type = $this->bundle;
}
else {
$file->type = 'file';
}
}
// Invoke migration prepare handlers
$this->prepare($file, $row);
@@ -641,7 +730,7 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
$file->preserve_files = FALSE;
$file_class = $this->fileClass;
$source = new $file_class((array)$file, $file);
$source = new $file_class((array) $file, $file);
$file = $source->processFile($file->value, $file->uid);
if (is_object($file) && isset($file->fid)) {
sites/all/modules/contrib/migrate/migrate/plugins/destinations/menu.inc
+3
View File
@@ -9,6 +9,7 @@
* Destination class implementing migration into {menu_custom}.
*/
class MigrateDestinationMenu extends MigrateDestination {
static public function getKeySchema() {
return array(
'menu_name' => array(
@@ -35,6 +36,7 @@ class MigrateDestinationMenu extends MigrateDestination {
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -55,6 +57,7 @@ class MigrateDestinationMenu extends MigrateDestination {
* Menu object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
sites/all/modules/contrib/migrate/migrate/plugins/destinations/menu_links.inc
+9 -4
View File
@@ -9,6 +9,7 @@
* Destination class implementing migration into {menu_links}.
*/
class MigrateDestinationMenuLinks extends MigrateDestination {
static public function getKeySchema() {
return array(
'mlid' => array(
@@ -19,6 +20,7 @@ class MigrateDestinationMenuLinks extends MigrateDestination {
),
);
}
public function __construct() {
parent::__construct();
}
@@ -33,6 +35,7 @@ class MigrateDestinationMenuLinks extends MigrateDestination {
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -72,9 +75,11 @@ class MigrateDestinationMenuLinks extends MigrateDestination {
* Import a single row.
*
* @param $menu_link
* Menu link object to build. Prefilled with any fields mapped in the Migration.
* Menu link object to build. Prefilled with any fields mapped in the
* Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
@@ -95,9 +100,9 @@ class MigrateDestinationMenuLinks extends MigrateDestination {
throw new MigrateException(t('System-of-record is DESTINATION, and the provided mlid could not be found'));
}
$menu_link_to_update = (object) $old_menu_link;
foreach($old_menu_link as $key => $value) {
if (!isset($menu_link->$key)) {
$menu_link->$key = $old_menu_link[$key];
foreach ($old_menu_link as $key => $value) {
if (!isset($menu_link->{$key})) {
$menu_link->{$key} = $old_menu_link[$key];
}
}
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/node.inc
+208 -41
View File
@@ -12,6 +12,9 @@
* Destination class implementing migration into nodes.
*/
class MigrateDestinationNode extends MigrateDestinationEntity {
protected $bypassDestIdCheck = FALSE;
static public function getKeySchema() {
return array(
'nid' => array(
@@ -51,6 +54,7 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -59,39 +63,39 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
$fields = array();
// First the core (node table) properties
$fields['nid'] = t('Node: <a href="@doc">Existing node ID</a>',
array('@doc' => 'http://drupal.org/node/1349696#nid'));
array('@doc' => 'http://drupal.org/node/1349696#nid'));
$node_type = node_type_load($this->bundle);
if ($node_type->has_title) {
$fields['title'] = t('Node: <a href="@doc">',
array('@doc' => 'http://drupal.org/node/1349696#title'))
. $node_type->title_label . '</a>';
array('@doc' => 'http://drupal.org/node/1349696#title'))
. $node_type->title_label . '</a>';
}
$fields['uid'] = t('Node: <a href="@doc">Authored by (uid)</a>',
array('@doc' => 'http://drupal.org/node/1349696#uid'));
$fields['created'] = t('Node: <a href="@doc">Created timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349696#created'));
$fields['changed'] = t('Node: <a href="@doc">Modified timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349696#changed'));
$fields['status'] = t('Node: <a href="@doc">Published</a>',
array('@doc' => 'http://drupal.org/node/1349696#status'));
$fields['promote'] = t('Node: <a href="@doc">Promoted to front page</a>',
array('@doc' => 'http://drupal.org/node/1349696#promote'));
$fields['sticky'] = t('Node: <a href="@doc">Sticky at top of lists</a>',
array('@doc' => 'http://drupal.org/node/1349696#sticky'));
$fields['revision'] = t('Node: <a href="@doc">Create new revision</a>',
array('@doc' => 'http://drupal.org/node/1349696#revision'));
$fields['log'] = t('Node: <a href="@doc">Revision Log message</a>',
array('@doc' => 'http://drupal.org/node/1349696#log'));
$fields['language'] = t('Node: <a href="@doc">Language (fr, en, ...)</a>',
array('@doc' => 'http://drupal.org/node/1349696#language'));
$fields['tnid'] = t('Node: <a href="@doc">The translation set id for this node</a>',
array('@doc' => 'http://drupal.org/node/1349696#tnid'));
$fields['translate'] = t('Node: <a href="@doc">A boolean indicating whether this translation page needs to be updated</a>',
array('@doc' => 'http://drupal.org/node/1349696#translate'));
$fields['revision_uid'] = t('Node: <a href="@doc">Modified (uid)</a>',
array('@doc' => 'http://drupal.org/node/1349696#revision_uid'));
$fields['uid'] = t('<a href="@doc">Authored by (uid)</a>',
array('@doc' => 'http://drupal.org/node/1349696#uid'));
$fields['created'] = t('<a href="@doc">Created timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349696#created'));
$fields['changed'] = t('<a href="@doc">Modified timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349696#changed'));
$fields['status'] = t('<a href="@doc">Published</a>',
array('@doc' => 'http://drupal.org/node/1349696#status'));
$fields['promote'] = t('<a href="@doc">Promoted to front page</a>',
array('@doc' => 'http://drupal.org/node/1349696#promote'));
$fields['sticky'] = t('<a href="@doc">Sticky at top of lists</a>',
array('@doc' => 'http://drupal.org/node/1349696#sticky'));
$fields['revision'] = t('<a href="@doc">Create new revision</a>',
array('@doc' => 'http://drupal.org/node/1349696#revision'));
$fields['log'] = t('<a href="@doc">Revision Log message</a>',
array('@doc' => 'http://drupal.org/node/1349696#log'));
$fields['language'] = t('<a href="@doc">Language (fr, en, ...)</a>',
array('@doc' => 'http://drupal.org/node/1349696#language'));
$fields['tnid'] = t('<a href="@doc">The translation set id for this node</a>',
array('@doc' => 'http://drupal.org/node/1349696#tnid'));
$fields['translate'] = t('<a href="@doc">A boolean indicating whether this translation page needs to be updated</a>',
array('@doc' => 'http://drupal.org/node/1349696#translate'));
$fields['revision_uid'] = t('<a href="@doc">Modified (uid)</a>',
array('@doc' => 'http://drupal.org/node/1349696#revision_uid'));
$fields['is_new'] = t('Option: <a href="@doc">Indicates a new node with the specified nid should be created</a>',
array('@doc' => 'http://drupal.org/node/1349696#is_new'));
array('@doc' => 'http://drupal.org/node/1349696#is_new'));
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
@@ -121,6 +125,7 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
* Node object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields (nid only in this case) of the node that was saved if
* successful. FALSE on failure.
@@ -128,13 +133,16 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
public function import(stdClass $node, stdClass $row) {
// Updating previously-migrated content?
$migration = Migration::currentMigration();
if (isset($row->migrate_map_destid1)) {
if (isset($row->migrate_map_destid1) && !$this->bypassDestIdCheck) {
// Make sure is_new is off
$node->is_new = FALSE;
if (isset($node->nid)) {
if ($node->nid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming nid !nid and map destination nid !destid1 don't match",
array('!nid' => $node->nid, '!destid1' => $row->migrate_map_destid1)));
array(
'!nid' => $node->nid,
'!destid1' => $row->migrate_map_destid1,
)));
}
}
else {
@@ -142,13 +150,13 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
}
// Get the existing vid, tnid so updates don't generate notices
$values = db_select('node', 'n')
->fields('n', array('vid', 'tnid'))
->condition('nid', $node->nid)
->execute()
->fetchAssoc();
->fields('n', array('vid', 'tnid'))
->condition('nid', $node->nid)
->execute()
->fetchAssoc();
if (empty($values)) {
throw new MigrateException(t("Incoming node ID !nid no longer exists",
array('!nid' => $node->nid)));
array('!nid' => $node->nid)));
}
$node->vid = $values['vid'];
if (empty($node->tnid)) {
@@ -162,7 +170,7 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
$old_node = node_load($node->nid);
if (empty($old_node)) {
throw new MigrateException(t('System-of-record is DESTINATION, but node !nid does not exist',
array('!nid' => $node->nid)));
array('!nid' => $node->nid)));
}
if (!isset($node->created)) {
$node->created = $old_node->created;
@@ -177,7 +185,8 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
$node->uid = $old_node->uid;
}
}
elseif (!isset($node->type)) {
if (!isset($node->type)) {
// Default the type to our designated destination bundle (by doing this
// conditionally, we permit some flexibility in terms of implementing
// migrations which can affect more than one type).
@@ -207,7 +216,12 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
if (isset($node->uid)) {
$uid = $node->uid;
}
if (isset($node->revision)) {
$revision = $node->revision;
}
node_object_prepare($node);
if (isset($created)) {
$node->created = $created;
}
@@ -215,6 +229,9 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
if (isset($uid)) {
$node->uid = $uid;
}
if (isset($revision)) {
$node->revision = $revision;
}
}
// Invoke migration prepare handlers
@@ -230,11 +247,11 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
foreach ($old_node as $field => $value) {
// An explicit NULL in the source data means to wipe to old value (i.e.,
// don't copy it over from $old_node)
if (property_exists($node, $field) && $node->$field === NULL) {
if (property_exists($node, $field) && $node->{$field} === NULL) {
// Ignore this field
}
elseif (!isset($node->$field)) {
$node->$field = $old_node->$field;
elseif (!isset($node->{$field})) {
$node->{$field} = $old_node->{$field};
}
}
}
@@ -246,6 +263,14 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
$updating = FALSE;
}
// Make sure that if is_new is not TRUE, it is not present.
if (isset($node->is_new) && empty($node->is_new)) {
unset($node->is_new);
}
// Validate field data prior to saving.
MigrateDestinationEntity::fieldAttachValidate('node', $node);
migrate_instrument_start('node_save');
node_save($node);
migrate_instrument_stop('node_save');
@@ -270,7 +295,7 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
// Potentially fix uid and timestamp in node_revisions.
$query = db_update('node_revision')
->condition('vid', $node->vid);
->condition('vid', $node->vid);
if (isset($changed)) {
$fields['timestamp'] = $changed;
}
@@ -296,3 +321,145 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
return $return;
}
}
/**
* Allows you to import revisions.
*
* Adapted from
* http://www.darrenmothersele.com/blog/2012/07/16/migrating-node-revisions-drupal-7/
*
* Class MigrateDestinationNodeRevision
*
* @author darrenmothersele
* @author cthos
*/
class MigrateDestinationNodeRevision extends MigrateDestinationNode {
/**
* Basic initialization.
*
* @see parent::__construct
*
* @param string $bundle
* A.k.a. the content type (page, article, etc.) of the node.
* @param array $options
* Options applied to nodes.
*/
public function __construct($bundle, array $options = array()) {
parent::__construct($bundle, $options);
$this->bypassDestIdCheck = TRUE;
}
/**
* Get key schema for the node revision destination.
*
* @see MigrateDestination::getKeySchema
*
* @return array
* Returns the key schema.
*/
static public function getKeySchema() {
return array(
'vid' => array(
'type' => 'int',
'unsigned' => TRUE,
'description' => 'ID of destination node revision',
),
);
}
/**
* Returns additional fields on top of node destinations.
*
* @param string $migration
* Active migration
*
* @return array
* Fields.
*/
public function fields($migration = NULL) {
$fields = parent::fields($migration);
$fields['vid'] = t('Node: <a href="@doc">Revision (vid)</a>', array('@doc' => 'http://drupal.org/node/1298724'));
return $fields;
}
/**
* Rolls back any versions that have been created.
*
* @param array $vids
* Version ids to roll back.
*/
public function bulkRollback(array $vids) {
migrate_instrument_start('revision_delete_multiple');
$this->prepareRollback($vids);
$nids = array();
foreach ($vids as $vid) {
if ($revision = node_load(NULL, $vid)) {
db_delete('node_revision')
->condition('vid', $revision->vid)
->execute();
module_invoke_all('node_revision_delete', $revision);
field_attach_delete_revision('node', $revision);
$nids[$revision->nid] = $revision->nid;
}
}
$this->completeRollback($vids);
foreach ($nids as $nid) {
$vid = db_select('node_revision', 'nr')
->fields('nr', array('vid'))
->condition('nid', $nid, '=')
->execute()
->fetchField();
if (!empty($vid)) {
db_update('node')
->fields(array('vid' => $vid))
->condition('nid', $nid, '=')
->execute();
}
}
migrate_instrument_stop('revision_delete_multiple');
}
/**
* Overridden import method.
*
* This is done because parent::import will return the nid of the newly
* created nodes. This is bad since the migrate_map_* table will have
* nids instead of vids, which could cause a nightmare explosion on
* rollback.
*
* @param stdClass $node
* Populated entity.
*
* @param stdClass $row
* Source information in object format.
*
* @return array|bool
* Array with newly created vid, or FALSE on error.
*
* @throws MigrateException
*/
public function import(stdClass $node, stdClass $row) {
// We're importing revisions, this should be set.
$node->revision = 1;
if (empty($node->nid)) {
throw new MigrateException(t('Missing incoming nid.'));
}
$original_updated = $this->numUpdated;
parent::import($node, $row);
// Reset num updated and increment created since new revision is always an update.
$this->numUpdated = $original_updated;
$this->numCreated++;
if (empty($node->vid)) {
return FALSE;
}
return array($node->vid);
}
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/path.inc
+20 -12
View File
@@ -6,29 +6,37 @@
*/
class MigratePathEntityHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('entity'));
}
public function fields($entity_type, $bundle) {
/**
* Implementation of MigrateDestinationHandler::fields().
*/
public function fields($entity_type, $bundle, $migration = NULL) {
if (module_exists('path')) {
switch ($entity_type) {
case 'node':
return array('path' => t('Node: Path alias'));
case 'user':
return array('path' => t('User: Path alias'));
case 'taxonomy_term':
return array('path' => t('Term: Path alias'));
}
return array('path' => t('Path alias'));
}
return array();
}
public function prepare($entity, stdClass $row) {
if (module_exists('path') && isset($entity->path)) {
$path = $entity->path;
$entity->path = array();
$entity->path['alias'] = $path;
// Make sure the alias doesn't already exist
$query = db_select('url_alias')
->condition('alias', $entity->path)
->condition('language', $entity->language);
$query->addExpression('1');
$query->range(0, 1);
if (!$query->execute()->fetchField()) {
$path = $entity->path;
$entity->path = array();
$entity->path['alias'] = $path;
}
else {
unset($entity->path);
}
}
}
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/poll.inc
+25 -14
View File
@@ -21,7 +21,8 @@
* public function prepareRow($row);
* $choices = Database::getConnection('default', 'legacy')
* ->select('src_poll_choice', 'c')
* ->fields('c', array('choice_label', 'choice_order', 'choice_total'))
* ->fields('c', array('choice_label', 'choice_order',
* 'choice_total'))
* ->condition('c.choiceid', $row->src_contentid);
* ->execute();
* $row->src_choices = array();
@@ -38,7 +39,8 @@
* ->select('src_poll_vote', 'v')
* ->fields('v', array('choice_uid', 'hostname', 'timestamp'))
* ->condition('v.choiceid', $row->src_contentid);
* $votes = $query->innerJoin('src_poll_choice', 'c', 'v.choice_id=c.choice_id')
* $votes = $query->innerJoin('src_poll_choice', 'c',
* 'v.choice_id=c.choice_id')
* ->fields('c', array('choice_label'))
* ->execute();
* $row->src_votes = array();
@@ -56,17 +58,21 @@
*/
class MigratePollEntityHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('node'));
}
public function fields($entity_type, $bundle) {
/**
* Implementation of MigrateDestinationHandler::fields().
*/
public function fields($entity_type, $bundle, $migration = NULL) {
if ($bundle == 'poll') {
$fields = array(
'active' => t('Poll: Active status'),
'runtime' => t('Poll: How long the poll runs for in seconds'),
'choice' => t('Poll: Choices. Each choice is an array with chtext, chvotes, and weight keys.'),
'votes' => t('Poll: Votes. Each vote is an array with chid (or chtext), uid, hostname, and timestamp keys'),
'active' => t('Active status'),
'runtime' => t('How long the poll runs for in seconds'),
'choice' => t('Choices. Each choice is an array with chtext, chvotes, and weight keys.'),
'votes' => t('Votes. Each vote is an array with chid (or chtext), uid, hostname, and timestamp keys'),
);
}
else {
@@ -89,7 +95,10 @@ class MigratePollEntityHandler extends MigrateDestinationHandler {
foreach ($row->choice as $choice) {
// Have no mapping tracking for chid, so assume choice text is unique.
db_update('poll_choice')
->fields(array('chvotes' => $choice['chvotes'], 'weight' => $choice['weight']))
->fields(array(
'chvotes' => $choice['chvotes'],
'weight' => $choice['weight'],
))
->condition('nid', $entity->nid)
->condition('chtext', $choice['chtext'])
->execute();
@@ -97,15 +106,17 @@ class MigratePollEntityHandler extends MigrateDestinationHandler {
// Insert actual votes.
foreach ($row->votes as $vote) {
$chid = $vote['chid'];
if (!isset($chid)) {
if (!isset($vote['chid'])) {
$result = db_select('poll_choice', 'pc')
->fields('pc', array('chid'))
->condition('pc.nid', $entity->nid)
->condition('pc.chtext', $vote['chtext'])
->execute();
->fields('pc', array('chid'))
->condition('pc.nid', $entity->nid)
->condition('pc.chtext', $vote['chtext'])
->execute();
$chid = $result->fetchField();
}
else {
$chid = $vote['chid'];
}
db_insert('poll_vote')
->fields(array(
'chid' => $chid,
sites/all/modules/contrib/migrate/migrate/plugins/destinations/statistics.inc
+14 -7
View File
@@ -6,16 +6,20 @@
*/
class MigrateStatisticsEntityHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('node'));
}
public function fields() {
/**
* Implementation of MigrateDestinationHandler::fields().
*/
public function fields($entity_type, $bundle, $migration = NULL) {
if (module_exists('statistics')) {
$fields = array(
'totalcount' => t('Node: The total number of times the node has been viewed.'),
'daycount' => t('Node: The total number of times the node has been viewed today.'),
'timestamp' => t('Node: The most recent time the node has been viewed.'),
'totalcount' => t('The total number of times the node has been viewed.'),
'daycount' => t('The total number of times the node has been viewed today.'),
'timestamp' => t('The most recent time the node has been viewed.'),
);
}
else {
@@ -26,9 +30,12 @@ class MigrateStatisticsEntityHandler extends MigrateDestinationHandler {
public function complete($node, stdClass $row) {
if (module_exists('statistics') && isset($node->nid)) {
$totalcount = isset($node->totalcount) ? $node->totalcount : 0;
$daycount = isset($node->daycount) ? $node->daycount : 0;
$timestamp = isset($node->timestamp) ? $node->timestamp : 0;
$totalcount = isset($node->totalcount) && is_numeric($node->totalcount) ?
$node->totalcount : 0;
$daycount = isset($node->daycount) && is_numeric($node->daycount) ?
$node->daycount : 0;
$timestamp = isset($node->timestamp) && is_numeric($node->timestamp) ?
$node->timestamp : 0;
db_merge('node_counter')
->key(array('nid' => $node->nid))
->fields(array(
sites/all/modules/contrib/migrate/migrate/plugins/destinations/table.inc
+54 -10
View File
@@ -10,6 +10,7 @@
* the Schema API.
*/
class MigrateDestinationTable extends MigrateDestination {
/**
* The schema of the current table.
*
@@ -57,19 +58,20 @@ class MigrateDestinationTable extends MigrateDestination {
/**
* Delete a single row.
*
* @param $id
* Primary key values.
* @param array $ids
* The primary key values of the row to be deleted.
*/
public function rollback(array $id) {
public function rollback(array $ids) {
migrate_instrument_start('table rollback');
$delete = db_delete($this->tableName);
$keys = array_keys(self::getKeySchema($this->tableName));
$i = 0;
foreach ($id as $value) {
$key = $keys[$i++];
$values = array_combine($keys, $ids);
$this->prepareRollback($values);
$delete = db_delete($this->tableName);
foreach ($values as $key => $value) {
$delete->condition($key, $value);
}
$delete->execute();
$this->completeRollback($values);
migrate_instrument_stop('table rollback');
}
@@ -80,6 +82,7 @@ class MigrateDestinationTable extends MigrateDestination {
* Object object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
@@ -121,7 +124,7 @@ class MigrateDestinationTable extends MigrateDestination {
}
$select = db_select($this->tableName)
->fields($this->tableName);
->fields($this->tableName);
foreach ($this->schema['primary key'] as $key) {
$select->condition($key, $entity->{$key});
}
@@ -130,7 +133,7 @@ class MigrateDestinationTable extends MigrateDestination {
throw new MigrateException(t('System-of-record is DESTINATION, but the destination entity does not exist'));
}
foreach ($entity as $field => $value) {
$old_entity->$field = $entity->$field;
$old_entity->{$field} = $entity->{$field};
}
$entity = $old_entity;
}
@@ -158,6 +161,7 @@ class MigrateDestinationTable extends MigrateDestination {
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -165,7 +169,7 @@ class MigrateDestinationTable extends MigrateDestination {
public function fields($migration = NULL) {
$fields = array();
foreach ($this->schema['fields'] as $column => $schema) {
$fields[$column] = t('Type: !type', array('!type' => $schema['type']));
$fields[$column] = t('!type', array('!type' => $schema['type']));
}
return $fields;
}
@@ -207,4 +211,44 @@ class MigrateDestinationTable extends MigrateDestination {
$migration->complete($entity, $source_row);
}
}
/**
* Give handlers a shot at cleaning up before the row has been rolled back.
*
* @param array $ids
* The primary key values of the row about to be deleted, keyed by field
* name.
*/
public function prepareRollback(array $ids) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('table', 'prepareRollback', $ids);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($ids);
}
}
/**
* Give handlers a shot at cleaning up after a row has been rolled back.
*
* @param array $ids
* The primary key values of the row which has been deleted, keyed by field
* name.
*/
public function completeRollback(array $ids) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('table', 'completeRollback', $ids);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($ids);
}
}
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/table_copy.inc
+19 -12
View File
@@ -9,6 +9,7 @@
* Destination class implementing migration into a single table.
*/
class MigrateDestinationTableCopy extends MigrateDestination {
public function __construct($tableName, $keySchema) {
parent::__construct();
$this->tableName = $tableName;
@@ -41,6 +42,7 @@ class MigrateDestinationTableCopy extends MigrateDestination {
* Object object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
@@ -49,14 +51,21 @@ class MigrateDestinationTableCopy extends MigrateDestination {
$migration = MigrationBase::currentMigration();
$fields = clone $row;
$keys = array_keys($this->keySchema);
$values = array();
foreach ($keys as $key) {
$values[] = $row->$key;
// Remove all map data, otherwise we'll try to write it to the destination
// table.
foreach ($fields as $field => $data) {
if (strpos($field, 'migrate_map_') === 0) {
unset($fields->{$field});
}
}
unset($fields->migrate_map_destid1);
unset($fields->needs_update);
$query = db_merge($this->tableName)->key($keys, $values)->fields((array)$fields);
$keys = array_keys($this->keySchema);
$values = array();
foreach ($keys as $key) {
$values[] = $row->{$key};
}
$query = db_merge($this->tableName)
->key($keys, $values)
->fields((array) $fields);
try {
$status = $query->execute();
if ($status == MergeQuery::STATUS_INSERT) {
@@ -66,13 +75,11 @@ class MigrateDestinationTableCopy extends MigrateDestination {
$this->numUpdated++;
}
return $values;
}
catch (MigrateException $e) {
} catch (MigrateException $e) {
$migration->saveMessage($e->getMessage(), $e->getLevel());
Migration::displayMessage($e->getMessage());
}
catch (Exception $e) {
$this->handleException($e);
} catch (Exception $e) {
$migration->handleException($e);
}
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/term.inc
+51 -26
View File
@@ -13,6 +13,7 @@
* Destination class implementing migration into terms.
*/
class MigrateDestinationTerm extends MigrateDestinationEntity {
/**
* Boolean indicating whether to permit duplicate terms to be created.
*
@@ -59,10 +60,12 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
}
/**
* Returns a list of fields available to be mapped for this vocabulary (bundle)
* Returns a list of fields available to be mapped for this vocabulary
* (bundle)
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -70,21 +73,21 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
public function fields($migration = NULL) {
$fields = array();
// First the core (taxonomy_term_data table) properties
$fields['tid'] = t('Term: <a href="@doc">Existing term ID</a>',
array('@doc' => 'http://drupal.org/node/1349702#tid'));
$fields['name'] = t('Term: <a href="@doc">Name</a>',
array('@doc' => 'http://drupal.org/node/1349702#name'));
$fields['description'] = t('Term: <a href="@doc">Description</a>',
array('@doc' => 'http://drupal.org/node/1349702#description'));
$fields['parent'] = t('Term: <a href="@doc">Parent (by Drupal term ID)</a>',
array('@doc' => 'http://drupal.org/node/1349702#parent'));
$fields['tid'] = t('<a href="@doc">Existing term ID</a>',
array('@doc' => 'http://drupal.org/node/1349702#tid'));
$fields['name'] = t('<a href="@doc">Name</a>',
array('@doc' => 'http://drupal.org/node/1349702#name'));
$fields['description'] = t('<a href="@doc">Description</a>',
array('@doc' => 'http://drupal.org/node/1349702#description'));
$fields['parent'] = t('<a href="@doc">Parent (by Drupal term ID)</a>',
array('@doc' => 'http://drupal.org/node/1349702#parent'));
// TODO: Remove parent_name, implement via arguments
$fields['parent_name'] = t('Term: <a href="@doc">Parent (by name)</a>',
array('@doc' => 'http://drupal.org/node/1349702#parent_name'));
$fields['format'] = t('Term: <a href="@doc">Format</a>',
array('@doc' => 'http://drupal.org/node/1349702#format'));
$fields['weight'] = t('Term: <a href="@doc">Weight</a>',
array('@doc' => 'http://drupal.org/node/1349702#weight'));
$fields['parent_name'] = t('<a href="@doc">Parent (by name)</a>',
array('@doc' => 'http://drupal.org/node/1349702#parent_name'));
$fields['format'] = t('<a href="@doc">Format</a>',
array('@doc' => 'http://drupal.org/node/1349702#format'));
$fields['weight'] = t('<a href="@doc">Weight</a>',
array('@doc' => 'http://drupal.org/node/1349702#weight'));
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('entity', 'fields', $this->entityType, $this->bundle, $migration);
@@ -131,6 +134,7 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
* Term object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields (tid only in this case) of the term that was saved if
* successful. FALSE on failure.
@@ -142,13 +146,22 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
if (isset($term->tid)) {
if ($term->tid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming tid !tid and map destination nid !destid1 don't match",
array('!tid' => $term->tid, '!destid1' => $row->migrate_map_destid1)));
array(
'!tid' => $term->tid,
'!destid1' => $row->migrate_map_destid1,
)));
}
}
else {
$term->tid = $row->migrate_map_destid1;
}
}
// Default to bundle if no vocabulary machine name provided
if (!isset($term->vocabulary_machine_name)) {
$term->vocabulary_machine_name = $this->bundle;
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($term->tid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination tid provided'));
@@ -161,15 +174,11 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
array('!tid' => $term->tid)));
}
foreach ($rawterm as $field => $value) {
$old_term->$field = $term->$field;
$old_term->{$field} = $term->{$field};
}
$term = $old_term;
}
else {
// Default to bundle if no vocabulary machine name provided
if (!isset($term->vocabulary_machine_name)) {
$term->vocabulary_machine_name = $this->bundle;
}
// vid is required
if (empty($term->vid)) {
static $vocab_map = array();
@@ -203,7 +212,13 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
if (empty($term->parent)) {
$term->parent = array(0);
}
if (is_array($term->parent) && isset($term->parent['arguments'])) {
elseif (!is_array($term->parent)) {
// Convert to an array for comparison in findMatchingTerm().
// Note: taxonomy_term_save() also normalizes to an array.
$term->parent = array($term->parent);
}
if (isset($term->parent['arguments'])) {
// Unset arguments here to avoid duplicate entries in the
// term_hierarchy table.
unset($term->parent['arguments']);
@@ -211,12 +226,19 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
if (!isset($term->format)) {
$term->format = $this->textFormat;
}
if (!isset($term->language)) {
$term->language = $this->language;
}
$this->prepare($term, $row);
if (empty($term->name)) {
throw new MigrateException(t('Taxonomy term name is required.'));
}
if (!$this->allowDuplicateTerms && $existing_term = $this->findMatchingTerm($term)) {
foreach ($existing_term as $field => $value) {
if (!isset($term->$field)) {
$term->$field = $existing_term->$field;
if (!isset($term->{$field})) {
$term->{$field} = $existing_term->{$field};
}
}
}
@@ -228,8 +250,8 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
if ($existing_term) {
// Incoming data overrides existing data, so only copy non-existent fields
foreach ($existing_term as $field => $value) {
if (!isset($term->$field)) {
$term->$field = $existing_term->$field;
if (!isset($term->{$field})) {
$term->{$field} = $existing_term->{$field};
}
}
}
@@ -242,6 +264,9 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
$updating = FALSE;
}
// Validate field data prior to saving.
MigrateDestinationEntity::fieldAttachValidate('taxonomy_term', $term);
migrate_instrument_start('taxonomy_term_save');
$status = taxonomy_term_save($term);
migrate_instrument_stop('taxonomy_term_save');
sites/all/modules/contrib/migrate/migrate/plugins/destinations/user.inc
+70 -47
View File
@@ -13,6 +13,7 @@
* Destination class implementing migration into users.
*/
class MigrateDestinationUser extends MigrateDestinationEntity {
/**
* Indicates whether incoming passwords are md5-encrypted - if so, we will
* rehash them similarly to the D6->D7 upgrade path.
@@ -68,6 +69,7 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
@@ -75,44 +77,44 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
public function fields($migration = NULL) {
$fields = array();
// First the core (users table) properties
$fields['uid'] = t('User: <a href="@doc">Existing user ID</a>',
array('@doc' => 'http://drupal.org/node/1349632#uid'));
$fields['mail'] = t('User: <a href="@doc">Email address</a>',
array('@doc' => 'http://drupal.org/node/1349632#mail'));
$fields['name'] = t('User: <a href="@doc">Username</a>',
array('@doc' => 'http://drupal.org/node/1349632#name'));
$fields['pass'] = t('User: <a href="@doc">Password (plain text)</a>',
array('@doc' => 'http://drupal.org/node/1349632#pass'));
$fields['status'] = t('User: <a href="@doc">Status</a>',
array('@doc' => 'http://drupal.org/node/1349632#status'));
$fields['created'] = t('User: <a href="@doc">Registered timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#created'));
$fields['access'] = t('User: <a href="@doc">Last access timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#access'));
$fields['login'] = t('User: <a href="@doc">Last login timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#login'));
$fields['roles'] = t('User: <a href="@doc">Role IDs</a>',
array('@doc' => 'http://drupal.org/node/1349632#roles'));
$fields['role_names'] = t('User: <a href="@doc">Role Names</a>',
array('@doc' => 'http://drupal.org/node/1349632#role_names'));
$fields['picture'] = t('User: <a href="@doc">Picture</a>',
array('@doc' => 'http://drupal.org/node/1349632#picture'));
$fields['signature'] = t('User: <a href="@doc">Signature</a>',
array('@doc' => 'http://drupal.org/node/1349632#signature'));
$fields['signature_format'] = t('User: <a href="@doc">Signature format</a>',
array('@doc' => 'http://drupal.org/node/1349632#signature_format'));
$fields['timezone'] = t('User: <a href="@doc">Timezone</a>',
array('@doc' => 'http://drupal.org/node/1349632#timezone'));
$fields['language'] = t('User: <a href="@doc">Language</a>',
array('@doc' => 'http://drupal.org/node/1349632#language'));
$fields['theme'] = t('User: <a href="@doc">Default theme</a>',
array('@doc' => 'http://drupal.org/node/1349632#theme'));
$fields['init'] = t('User: <a href="@doc">Init</a>',
array('@doc' => 'http://drupal.org/node/1349632#init'));
$fields['data'] = t('User: <a href="@doc">Data</a>',
array('@doc' => 'http://drupal.org/node/1349632#init'));
$fields['uid'] = t('<a href="@doc">Existing user ID</a>',
array('@doc' => 'http://drupal.org/node/1349632#uid'));
$fields['mail'] = t('<a href="@doc">Email address</a>',
array('@doc' => 'http://drupal.org/node/1349632#mail'));
$fields['name'] = t('<a href="@doc">Username</a>',
array('@doc' => 'http://drupal.org/node/1349632#name'));
$fields['pass'] = t('<a href="@doc">Password</a>',
array('@doc' => 'http://drupal.org/node/1349632#pass'));
$fields['status'] = t('<a href="@doc">Status</a>',
array('@doc' => 'http://drupal.org/node/1349632#status'));
$fields['created'] = t('<a href="@doc">Registered timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#created'));
$fields['access'] = t('<a href="@doc">Last access timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#access'));
$fields['login'] = t('<a href="@doc">Last login timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#login'));
$fields['roles'] = t('<a href="@doc">Role IDs</a>',
array('@doc' => 'http://drupal.org/node/1349632#roles'));
$fields['role_names'] = t('<a href="@doc">Role Names</a>',
array('@doc' => 'http://drupal.org/node/1349632#role_names'));
$fields['picture'] = t('<a href="@doc">Picture</a>',
array('@doc' => 'http://drupal.org/node/1349632#picture'));
$fields['signature'] = t('<a href="@doc">Signature</a>',
array('@doc' => 'http://drupal.org/node/1349632#signature'));
$fields['signature_format'] = t('<a href="@doc">Signature format</a>',
array('@doc' => 'http://drupal.org/node/1349632#signature_format'));
$fields['timezone'] = t('<a href="@doc">Timezone</a>',
array('@doc' => 'http://drupal.org/node/1349632#timezone'));
$fields['language'] = t('<a href="@doc">Language</a>',
array('@doc' => 'http://drupal.org/node/1349632#language'));
$fields['theme'] = t('<a href="@doc">Default theme</a>',
array('@doc' => 'http://drupal.org/node/1349632#theme'));
$fields['init'] = t('<a href="@doc">Init</a>',
array('@doc' => 'http://drupal.org/node/1349632#init'));
$fields['data'] = t('<a href="@doc">Data</a>',
array('@doc' => 'http://drupal.org/node/1349632#init'));
$fields['is_new'] = t('Option: <a href="@doc">Indicates a new user with the specified uid should be created</a>',
array('@doc' => 'http://drupal.org/node/1349632#is_new'));
array('@doc' => 'http://drupal.org/node/1349632#is_new'));
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
@@ -139,9 +141,11 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
* Import a single user.
*
* @param $account
* Account object to build. Prefilled with any fields mapped in the Migration.
* Account object to build. Prefilled with any fields mapped in the
* Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields (uid only in this case) of the user that was saved if
* successful. FALSE on failure.
@@ -155,7 +159,10 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
if (isset($account->uid)) {
if ($account->uid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming uid !uid and map destination uid !destid1 don't match",
array('!uid' => $account->uid, '!destid1' => $row->migrate_map_destid1)));
array(
'!uid' => $account->uid,
'!destid1' => $row->migrate_map_destid1,
)));
}
}
else {
@@ -195,7 +202,7 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
}
$account->roles = drupal_map_assoc($account->roles);
}
if (empty($account->roles) && empty($old_account->roles)) {
if (empty($account->roles) && empty($old_account->roles)) {
$account->roles = array();
}
@@ -230,8 +237,11 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
$account->login = MigrationBase::timestamp($account->login);
}
// Validate field data prior to saving.
MigrateDestinationEntity::fieldAttachValidate('user', $account);
migrate_instrument_start('user_save');
$newaccount = user_save($old_account, (array)$account);
$newaccount = user_save($old_account, (array) $account);
migrate_instrument_stop('user_save');
if ($newaccount) {
if ($this->md5Passwords && !empty($account->pass)) {
@@ -262,6 +272,14 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
}
else {
$this->numCreated++;
// user_save() doesn't update file_usage on account creation, we have
// to do it ourselves.
if (!empty($newaccount->picture)) {
$file = file_load($newaccount->picture);
if (is_object($file)) {
file_usage_add($file, 'user', 'user', $newaccount->uid);
}
}
}
$this->complete($newaccount, $row);
$return = array($newaccount->uid);
@@ -274,6 +292,7 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
}
class MigrateDestinationRole extends MigrateDestinationTable {
public function __construct() {
parent::__construct('role');
}
@@ -298,7 +317,7 @@ class MigrateDestinationRole extends MigrateDestinationTable {
public function rollback(array $id) {
migrate_instrument_start('role rollback');
$rid = reset($id);
user_role_delete((int)$rid);
user_role_delete((int) $rid);
migrate_instrument_stop('role rollback');
}
@@ -309,6 +328,7 @@ class MigrateDestinationRole extends MigrateDestinationTable {
* Object object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
@@ -322,12 +342,15 @@ class MigrateDestinationRole extends MigrateDestinationTable {
if (isset($entity->rid)) {
if ($entity->rid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming id !id and map destination id !destid don't match",
array('!id' => $entity->rid, '!destid' => $row->migrate_map_destid1)));
}
else {
$entity->rid = $row->migrate_map_destid1;
array(
'!id' => $entity->rid,
'!destid' => $row->migrate_map_destid1,
)));
}
}
else {
$entity->rid = $row->migrate_map_destid1;
}
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
@@ -339,7 +362,7 @@ class MigrateDestinationRole extends MigrateDestinationTable {
$old_entity = user_role_load($entity->rid);
foreach ($entity as $field => $value) {
$old_entity->$field = $entity->$field;
$old_entity->{$field} = $entity->{$field};
}
$entity = $old_entity;
}
sites/all/modules/contrib/migrate/migrate/plugins/destinations/variable.inc
+190
View File
@@ -0,0 +1,190 @@
<?php
/**
* @file
* Support for variable destinations.
*/
/**
* Destination class implementing migration into {variable}.
*/
class MigrateDestinationVariable extends MigrateDestination {
static public function getKeySchema() {
return array(
'name' => array(
'description' => 'The name of the variable.',
'type' => 'varchar',
'length' => 128,
'not null' => TRUE,
'default' => '',
),
);
}
public function __construct() {
parent::__construct();
}
public function __toString() {
$output = t('Variable');
return $output;
}
/**
* Returns a list of fields available to be mapped for variables.
*
* @param Migration $migration
* Optionally, the migration containing this destination.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array(
'name' => t('The name of the variable.'),
'value' => t('The value of the variable.'),
);
return $fields;
}
/**
* Import a single row.
*
* @param $variable
* Variable object to build. Prefilled with any fields mapped in the
* Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
*
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $variable, stdClass $row) {
// Invoke migration prepare handlers
$this->prepare($variable, $row);
// Check to see if this is a new variable.
$update = FALSE;
// We cannot just check against NULL because a variable might actually be
// set to NULL. Attempt to use a unique variable default value that nothing
// else would use.
$default = 'migrate:' . REQUEST_TIME . ':' . drupal_random_key();
if (variable_get($variable->name, $default) !== $default) {
$update = TRUE;
}
// variable_set() provides no return callback, so we can't really test this
// without running a variable_get() check.
migrate_instrument_start('variable_set');
variable_set($variable->name, $variable->value);
migrate_instrument_stop('variable_set');
// Return the new id or FALSE on failure.
if (variable_get($variable->name, $default) === $variable->value) {
// Increment the count if the save succeeded.
if ($update) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
// Return the primary key to the mapping table.
$return = array($variable->name);
}
else {
$return = FALSE;
}
// Invoke migration complete handlers.
$this->complete($variable, $row);
return $return;
}
/**
* Implementation of MigrateDestination::prepare().
*/
public function prepare($variable, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$variable->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('variable', 'prepare', $variable, $row);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($variable, $row);
}
}
/**
* Implementation of MigrateDestination::complete().
*/
public function complete($variable, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$variable->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('variable', 'complete', $variable, $row);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'complete')) {
$migration->complete($variable, $row);
}
}
/**
* Delete a single variable.
*
* @param $id
* Array of fields representing the key (in this case, just variable name).
*/
public function rollback(array $id) {
$name = reset($id);
migrate_instrument_start('variable_delete');
$this->prepareRollback($name);
variable_del($name);
$this->completeRollback($name);
migrate_instrument_stop('variable_delete');
}
/**
* Give handlers a shot at cleaning up before a variable has been rolled back.
*
* @param $name
* The name of the variable about to be deleted.
*/
public function prepareRollback($name) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('variable', 'prepareRollback', $name);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($name);
}
}
/**
* Give handlers a shot at cleaning up after a variable has been rolled back.
*
* @param $name
* The name of the variable which has been deleted.
*/
public function completeRollback($name) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('variable', 'completeRollback', $name);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($name);
}
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/csv.inc
+248 -3
View File
@@ -5,6 +5,213 @@
* Define a MigrateSource for importing from comma separated values files.
*/
/**
* Implementation of MigrateList, for retrieving a list of IDs to be migrated
* from a CSV file.
*/
class MigrateListCSV extends MigrateList {
/**
* The path to the CSV file containing a list of IDs to be processed.
*
* @var string
*/
protected $file;
/**
* File handle for the CSV file being iterated.
*
* @var resource
*/
protected $csvHandle = NULL;
/**
* The column which contains the ID.
*
* If numeric, this is the index of the column from 0; if non-numeric, this
* is the column name as given by the header row.
*/
protected $idColumn;
/**
* The number of rows in the CSV file before the data starts.
*
* @var integer
*/
protected $headerRows = 0;
/**
* The array keys taken from the first row.
*
* @var array
*/
protected $headers = array();
/**
* Parameters for the fgetcsv() call.
*
* @var array
*/
protected $fgetcsv = array();
/**
* Constructor for MigrateListCSV.
*
* @param $list_path
* The path to the CSV file that holds the list of IDs.
* @param $id_column
* The column in the file that holds the IDs. A numeric index for the column,
* from 0.
* @param $options = array()
* An array of further options for the CSV file.
*/
public function __construct($list_path, $id_column, $options = array()) {
parent::__construct();
$this->file = $list_path;
$this->idColumn = $id_column;
$this->options = $options;
// fgetcsv specific options
foreach (array(
'length' => NULL,
'delimiter' => ',',
'enclosure' => '"',
'escape' => '\\',
) as $key => $default) {
$this->fgetcsv[$key] = isset($options[$key]) ? $options[$key] : $default;
}
if (!empty($options['header_rows'])) {
$this->headerRows = $options['header_rows'];
}
else {
$this->headerRows = 0;
}
if (!is_numeric($id_column)) {
//we need to store the headers.
$this->csvHandle = fopen($this->file, 'r');
if (!$this->validResource()) {
return;
}
$this->headers = $this->getNextLine();
fclose($this->csvHandle);
$this->csvHandle = NULL;
}
}
/**
* Return a string representing the source file.
*
* @return string
*/
public function __toString() {
return $this->file;
}
/**
* Return an array of IDs from the CSV file.
*
* @return array
*/
public function getIdList() {
$ids = array();
$this->csvHandle = fopen($this->file, 'r');
if (!$this->validResource()) {
return $ids;
}
// Skip the headers if any,
for ($i = 0; $i < $this->headerRows; $i++) {
$this->getNextLine();
}
while ($row = $this->getNextLine()) {
$ids[] = $row[$this->idColumn];
}
fclose($this->csvHandle);
$this->csvHandle = NULL;
return $ids;
}
/**
* Return a count of all available IDs from the source listing.
*
* @return integer
*/
public function computeCount() {
// If the data may have embedded newlines, the file line count won't reflect
// the number of CSV records (one record will span multiple lines). We need
// to scan with fgetcsv to get the true count.
if (!empty($this->options['embedded_newlines'])) {
$this->csvHandle = fopen($this->file, 'r');
$count = 0;
if (!$this->validResource()) {
return $count;
}
// Skip all but the last header
for ($i = 0; $i < $this->headerRows; $i++) {
fgets($this->csvHandle);
}
while ($this->getNextLine()) {
$count++;
}
fclose($this->csvHandle);
$this->csvHandle = NULL;
}
else {
$file = new SplFileObject($this->file, 'r');
$file->seek(PHP_INT_MAX);
$count = $file->key() + 1;
$file->rewind();
$count -= $this->headerRows;
}
return $count;
}
/**
* Get the next line of the CSV file.
* Returns an array of the next line, keyed by headers if required
*
* @return array
*/
protected function getNextLine() {
// escape parameter was added in PHP 5.3.
if (version_compare(phpversion(), '5.3', '<')) {
$row = fgetcsv($this->csvHandle, $this->fgetcsv['length'],
$this->fgetcsv['delimiter'], $this->fgetcsv['enclosure']);
}
else {
$row = fgetcsv($this->csvHandle, $this->fgetcsv['length'],
$this->fgetcsv['delimiter'], $this->fgetcsv['enclosure'],
$this->fgetcsv['escape']);
}
// Key the array with the headers
if (!empty($this->headers) && $row) {
$row = array_combine($this->headers, $row);
}
return $row;
}
/**
* Check if resource loaded correctly.
*
* @return bool
*/
public function validResource() {
if (!$this->csvHandle) {
Migration::displayMessage(t('Could not open CSV file !url',
array('!url' => $this->file)));
}
return (bool) $this->csvHandle;
}
}
/**
* Implementation of MigrateSource, to handle imports from CSV files.
*
@@ -12,6 +219,7 @@
* UTF BOM (Byte Order Marker) so they are interpreted correctly.
*/
class MigrateSourceCSV extends MigrateSource {
/**
* List of available source fields.
*
@@ -47,6 +255,13 @@ class MigrateSourceCSV extends MigrateSource {
*/
protected $rowNumber;
/**
* The path to the source file.
*
* @var string
*/
protected $file;
/**
* Simple initialization.
*
@@ -73,7 +288,12 @@ class MigrateSourceCSV extends MigrateSource {
$this->options = $options;
$this->fields = $fields;
// fgetcsv specific options
foreach (array('length' => NULL, 'delimiter' => ',', 'enclosure' => '"', 'escape' => '\\') as $key => $default) {
foreach (array(
'length' => NULL,
'delimiter' => ',',
'enclosure' => '"',
'escape' => '\\',
) as $key => $default) {
$this->fgetcsv[$key] = isset($options[$key]) ? $options[$key] : $default;
}
// One can either pass in an explicit list of column names to use, or if we have
@@ -81,6 +301,9 @@ class MigrateSourceCSV extends MigrateSource {
if ($this->headerRows && empty($csvcolumns)) {
$this->csvcolumns = array();
$this->csvHandle = fopen($this->file, 'r');
if (!$this->validResource()) {
return;
}
// Skip all but the last header
for ($i = 0; $i < $this->headerRows - 1; $i++) {
$this->getNextLine();
@@ -139,11 +362,14 @@ class MigrateSourceCSV extends MigrateSource {
// to scan with fgetcsv to get the true count.
if (!empty($this->options['embedded_newlines'])) {
$this->csvHandle = fopen($this->file, 'r');
$count = 0;
if (!$this->validResource()) {
return $count;
}
// Skip all but the last header
for ($i = 0; $i < $this->headerRows; $i++) {
fgets($this->csvHandle);
}
$count = 0;
while ($this->getNextLine()) {
$count++;
}
@@ -171,6 +397,9 @@ class MigrateSourceCSV extends MigrateSource {
}
// Load up the first row, skipping the header(s) if necessary
$this->csvHandle = fopen($this->file, 'r');
if (!$this->validResource()) {
return;
}
for ($i = 0; $i < $this->headerRows; $i++) {
$this->getNextLine();
}
@@ -186,6 +415,8 @@ class MigrateSourceCSV extends MigrateSource {
public function getNextRow() {
$row = $this->getNextLine();
if ($row) {
// only use rows specified in $this->csvcolumns().
$row = array_intersect_key($row, $this->csvcolumns);
// Set meaningful keys for the columns mentioned in $this->csvcolumns().
foreach ($this->csvcolumns as $int => $values) {
list($key, $description) = $values;
@@ -194,7 +425,7 @@ class MigrateSourceCSV extends MigrateSource {
unset($row[$int]);
}
$row['csvrownum'] = $this->rowNumber++;
return (object)$row;
return (object) $row;
}
else {
fclose($this->csvHandle);
@@ -216,4 +447,18 @@ class MigrateSourceCSV extends MigrateSource {
}
return $row;
}
/**
* Check if resource loaded correctly.
*
* @return bool
*/
public function validResource() {
if (!$this->csvHandle) {
Migration::displayMessage(t('Could not open CSV file !url',
array('!url' => $this->file)));
}
return (bool) $this->csvHandle;
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/db2.inc
+165
View File
@@ -0,0 +1,165 @@
<?php
/**
* @file
* Define a MigrateSource class for importing from IBM DB2 databases.
*/
/**
* Implementation of MigrateSource, to handle imports from remote DB2 servers.
*/
class MigrateSourceDB2 extends MigrateSource {
/**
* Array containing information for connecting to DB2:
* database - Database to connect to
* username - Username to connect as
* password - Password for authentication
*
* @var array
*/
protected $configuration;
/**
* The active DB2 connection for this source.
*
* @var resource
*/
protected $connection;
public function getConnection() {
return $this->connection;
}
/**
* The SQL query from which to obtain data. Is a string.
*/
protected $query;
/**
* The statement resource from executing the query - traversed to process the
* incoming data.
*/
protected $stmt;
/**
* Return an options array for DB2 sources.
*
* @param boolean $cache_counts
* Indicates whether to cache counts of source records.
*/
static public function options($cache_counts = FALSE) {
return compact('cache_counts');
}
/**
* Simple initialization.
*/
public function __construct(array $configuration, $query, $count_query,
array $fields, array $options = array()) {
parent::__construct($options);
$this->query = $query;
$this->countQuery = $count_query;
$this->configuration = $configuration;
$this->fields = $fields;
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
return $this->query;
}
/**
* Connect lazily to the DB server.
*/
protected function connect() {
if (!isset($this->connection)) {
// Check for the ibm_db2 extension before attempting to connect with it.
if (!extension_loaded('ibm_db2')) {
throw new Exception(t('You must configure the ibm_db2 extension in PHP.'));
}
// Connect to db2.
$this->connection = db2_connect($this->configuration['database'],
$this->configuration['username'], $this->configuration['password']);
}
if ($this->connection) {
return TRUE;
}
// If we failed to connect, throw an exception with the connection error
// message.
else {
$e = db2_conn_errormsg();
throw new Exception($e);
return FALSE;
}
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
// The fields are passed to the constructor for this plugin.
return $this->fields;
}
/**
* Return a count of all available source records.
*/
public function computeCount() {
migrate_instrument_start('MigrateSourceDB2 count');
// Make sure we're connected.
if ($this->connect()) {
// Execute the count query.
$stmt = db2_exec($this->connection, $this->countQuery);
// If something went wrong, throw an exception with the error message.
if (!$stmt) {
$e = db2_stmt_errormsg($stmt);
throw new Exception($e);
}
// Grab the first row as an array.
$count_array = db2_fetch_array($stmt);
// The first item in this array will be our count.
$count = reset($count_array);
}
else {
// Connection failed.
$count = FALSE;
}
migrate_instrument_stop('MigrateSourceDB2 count');
return $count;
}
/**
* Implementation of MigrateSource::performRewind().
*/
public function performRewind() {
migrate_instrument_start('db2_query');
// Ensure we're connected to the database.
$this->connect();
// Execute the query.
$this->stmt = db2_exec($this->connection, $this->query);
// Throw an exception with the error message if something went wrong.
if (!$this->stmt) {
$e = db2_stmt_errormsg($this->stmt);
throw new Exception($e);
}
migrate_instrument_stop('db2_query');
}
/**
* Implementation of MigrateSource::getNextRow().
*
* @return object
*/
public function getNextRow() {
return db2_fetch_object($this->stmt);
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/files.inc
+113 -27
View File
@@ -23,15 +23,71 @@ define('MIGRATE_CHUNK_SEPARATOR', '-?MIGRATECHUNK?-');
* count, and then retrieve each chunk by it's ID.
*/
abstract class MigrateContentParser {
/**
* Tells the MigrateListFiles implementation to always use chunk IDs when
* constructing item IDs.
*/
public $alwaysUseChunkIDs = FALSE;
/**
* The content of the current file to parse.
*/
protected $content;
public function setContent($content) {
/**
* The ID of the current file being parsed.
*/
protected $item_id;
/**
* Set the content and file ID for the current file to parse.
*
* @param $content
* The content of the current file to parse.
* @param $item_id = NULL
* The ID of the file. The chunk IDs are appended to this to form the IDs
* that are returned by the MigrateList implementation.
*/
public function setContent($content, $item_id = NULL) {
$this->content = $content;
$this->item_id = $item_id;
}
/**
* Return the list of IDs for all the chunks found in the current file.
*
* Unless $alwaysUseChunkIDs is set to TRUE, this is only called if
* getChunkCount() returned a count greater than 1.
*
*
* @return
* An array of IDs.
*/
abstract public function getChunkIDs();
/**
* Get a single chunk from the current file.
*
* @param $id
* The ID for the chunk. This is one of the IDs returned by getChunkIDs().
*
* @return
* The content of the chunk.
*/
abstract public function getChunk($id);
/**
* Return the count of chunks in the current file.
*
* If this returns 1, then getChunkIDs() is not called and the file is treated
* as a single chunk. To override this, set $alwaysUseChunkIDs to TRUE.
*
* @return int
* The number of chunks in the current file.
*/
abstract public function getChunkCount();
}
/**
@@ -40,12 +96,15 @@ abstract class MigrateContentParser {
* primarily for when your HTML files map one-to-one to nodes.
*/
class MigrateSimpleContentParser extends MigrateContentParser {
public function getChunkIDs() {
return array('1');
}
public function getChunk($id) {
return $this->content;
}
public function getChunkCount() {
return 1;
}
@@ -54,15 +113,24 @@ class MigrateSimpleContentParser extends MigrateContentParser {
/**
* Implementation of MigrateList, for retrieving a list of IDs to be migrated
* from a directory listing. Each item is a file, it's ID is the path.
*
* If individual files contain more than one source, use a MigrateContentParser
* to parse the file and split it into chunks.
*/
class MigrateListFiles extends MigrateList {
protected $listDirs;
protected $baseDir;
protected $fileMask;
protected $directoryOptions;
protected $parser;
protected $getContents;
/**
* Constructor.
*
@@ -76,18 +144,25 @@ class MigrateListFiles extends MigrateList {
* );
* @param $base_dir
* The base dir is the part of the path that will be excluded when making
* an ID for each file. To continue the example from above, you want base_dir
* to be = '/var/html_source', so that the files will have IDs in the format
* an ID for each file. To continue the example from above, you want
* base_dir to be = '/var/html_source', so that the files will have IDs in
* the format
* '/en/news/news_2011_03_4.html'.
* @param $file_mask
* Passed on and used to filter for certain types of files. Use a regular
* expression, for example '/(.*\.htm$|.*\.html$)/i' to match all .htm and
* .html files, case insensitive.
* (optional) Passed on and used to filter for certain types of files. Use
* a
* regular expression, for example '/(.*\.htm$|.*\.html$)/i' to match all
* .htm and .html files, case insensitive. Defaults to matching all files.
* @param $options
* Options that will be passed on to file_scan_directory(). See docs of that
* Options that will be passed on to file_scan_directory(). See docs of
* that
* core Drupal function for more information.
* @param MigrateContentParser $parser
* Content parser class to use.
* @param $get_contents
* Whether to load the contents of files.
*/
public function __construct($list_dirs, $base_dir, $file_mask = NULL, $options = array(), MigrateContentParser $parser = NULL) {
public function __construct($list_dirs, $base_dir, $file_mask = '//', $options = array(), MigrateContentParser $parser = NULL, $get_contents = TRUE) {
if (!$parser) {
$parser = new MigrateSimpleContentParser();
}
@@ -97,6 +172,7 @@ class MigrateListFiles extends MigrateList {
$this->fileMask = $file_mask;
$this->directoryOptions = $options;
$this->parser = $parser;
$this->getContents = $get_contents;
}
/**
@@ -136,15 +212,21 @@ class MigrateListFiles extends MigrateList {
protected function getIDsFromFiles(array $files) {
$ids = array();
foreach ($files as $file) {
$contents = file_get_contents($file->uri);
$this->parser->setContent($contents);
if ($this->parser->getChunkCount() > 1) {
foreach ($this->parser->getChunkIDs() as $chunk_id) {
$ids[] = str_replace($this->baseDir, '', (string) $file->uri) . MIGRATE_CHUNK_SEPARATOR . $chunk_id;
$file_base_id = str_replace($this->baseDir, '', (string) $file->uri);
if ($this->getContents) {
$contents = file_get_contents($file->uri);
$this->parser->setContent($contents, $file_base_id);
if ($this->parser->alwaysUseChunkIDs || $this->parser->getChunkCount() > 1) {
foreach ($this->parser->getChunkIDs() as $chunk_id) {
$ids[] = $file_base_id . MIGRATE_CHUNK_SEPARATOR . $chunk_id;
}
}
else {
$ids[] = $file_base_id;
}
}
else {
$ids[] = str_replace($this->baseDir, '', (string) $file->uri);
$ids[] = $file_base_id;
}
}
@@ -172,7 +254,9 @@ class MigrateListFiles extends MigrateList {
class MigrateItemFile extends MigrateItem {
protected $baseDir;
protected $getContents;
protected $parser;
/**
@@ -204,28 +288,30 @@ class MigrateItemFile extends MigrateItem {
* The item object for migration.
*/
public function getItem($id) {
$pieces = explode(MIGRATE_CHUNK_SEPARATOR ,$id);
$pieces = explode(MIGRATE_CHUNK_SEPARATOR, $id);
$item_uri = $this->baseDir . $pieces[0];
$chunk = $pieces[1];
$chunk = !empty($pieces[1]) ? $pieces[1] : '';
// Get the file data at the specified URI
$data = $this->loadFile($item_uri);
if (is_string($data)) {
$this->parser->setContent($data);
$this->parser->setContent($data, $pieces[0]);
$return = new stdClass;
$return->filedata = $this->parser->getChunk($chunk);
return $return;
}
else if ($data === TRUE) {
$return = new stdClass;
return $return;
}
else {
$migration = Migration::currentMigration();
$message = t('Loading of !objecturi failed:', array('!objecturi' => $item_uri));
$migration->getMap()->saveMessage(
array($id), $message, MigrationBase::MESSAGE_ERROR);
return NULL;
if ($data === TRUE) {
$return = new stdClass;
return $return;
}
else {
$migration = Migration::currentMigration();
$message = t('Loading of !objecturi failed:', array('!objecturi' => $item_uri));
$migration->getMap()->saveMessage(
array($id), $message, MigrationBase::MESSAGE_ERROR);
return NULL;
}
}
}
@@ -242,4 +328,4 @@ class MigrateItemFile extends MigrateItem {
}
return $data;
}
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/json.inc
+20 -12
View File
@@ -10,6 +10,7 @@
* from a JSON object.
*/
class MigrateListJSON extends MigrateList {
/**
* A URL pointing to an JSON object containing a list of IDs to be processed.
*
@@ -35,7 +36,8 @@ class MigrateListJSON extends MigrateList {
}
/**
* Load the JSON at the given URL, and return an array of the IDs found within it.
* Load the JSON at the given URL, and return an array of the IDs found
* within it.
*
* @return array
*/
@@ -51,12 +53,12 @@ class MigrateListJSON extends MigrateList {
migrate_instrument_stop("Retrieve $this->listUrl");
if ($json) {
$data = drupal_json_decode($json);
if ($data) {
if ($data !== NULL) {
return $this->getIDsFromJSON($data);
}
}
Migration::displayMessage(t('Loading of !listurl failed:',
array('!listurl' => $this->listUrl)));
array('!listurl' => $this->listUrl)));
return NULL;
}
@@ -92,7 +94,7 @@ class MigrateListJSON extends MigrateList {
if ($json) {
$data = drupal_json_decode($json);
if ($data) {
$count = count($data);
$count = count($this->getIDsFromJSON($data));
}
}
return $count;
@@ -104,6 +106,7 @@ class MigrateListJSON extends MigrateList {
* an ID provided by a MigrateList class.
*/
class MigrateItemJSON extends MigrateItem {
/**
* A URL pointing to a JSON object containing the data for one item to be
* migrated.
@@ -136,7 +139,7 @@ class MigrateItemJSON extends MigrateItem {
}
else {
$migration = Migration::currentMigration();
$message = t('Loading of !objecturl failed:', array('!objecturl' => $item_url));
$message = t('Loading of !objecturl failed:', array('!objecturl' => $item_url));
$migration->getMap()->saveMessage(
array($id), $message, MigrationBase::MESSAGE_ERROR);
return NULL;
@@ -178,6 +181,7 @@ class MigrateItemJSON extends MigrateItem {
* To deal with different structures, extend this class and override next().
*/
class MigrateJSONReader implements Iterator {
/**
* URL of the source JSON file.
*
@@ -242,7 +246,8 @@ class MigrateJSONReader implements Iterator {
* The next non-whitespace character, or FALSE on end-of-file.
*/
protected function getNonBlank() {
while (($c = fgetc($this->fileHandle)) !== FALSE && trim($c) == '') {}
while (($c = fgetc($this->fileHandle)) !== FALSE && trim($c) == '') {
}
return $c;
}
@@ -266,7 +271,7 @@ class MigrateJSONReader implements Iterator {
$this->fileHandle = fopen($this->url, 'r');
if (!$this->fileHandle) {
Migration::displayMessage(t('Could not open JSON file !url',
array('!url' => $this->url)));
array('!url' => $this->url)));
return;
}
@@ -278,7 +283,7 @@ class MigrateJSONReader implements Iterator {
}
if ($char != '[') {
Migration::displayMessage(t('!url is not a JSON file containing an array of objects',
array('!url' => $this->url)));
array('!url' => $this->url)));
return;
}
}
@@ -390,9 +395,11 @@ class MigrateJSONReader implements Iterator {
}
/**
* Implementation of MigrateSource, to handle imports from stand-alone JSON files.
* Implementation of MigrateSource, to handle imports from stand-alone JSON
* files.
*/
class MigrateSourceJSON extends MigrateSource {
/**
* The MigrateJSONReader object serving as a cursor over the JSON source file.
*
@@ -402,6 +409,7 @@ class MigrateSourceJSON extends MigrateSource {
/**
* Name of the field within the JSON object holding the ID value.
*
* @var string
*/
protected $idField;
@@ -454,7 +462,7 @@ class MigrateSourceJSON extends MigrateSource {
* MigrateJSONReader).
*/
public function __construct($urls, $id_field, array $fields = array(),
array $options = array()) {
array $options = array()) {
parent::__construct($options);
$this->idField = $id_field;
if (empty($options['reader_class'])) {
@@ -503,7 +511,7 @@ class MigrateSourceJSON extends MigrateSource {
* @return string
*/
public function activeUrl() {
if ($this->activeUrl) {
if (isset($this->activeUrl)) {
return $this->sourceUrls[$this->activeUrl];
}
}
@@ -581,7 +589,7 @@ class MigrateSourceJSON extends MigrateSource {
// Return value
$status = FALSE;
while ($this->activeUrl === NULL || (count($this->sourceUrls)-1) > $this->activeUrl) {
while ($this->activeUrl === NULL || (count($this->sourceUrls) - 1) > $this->activeUrl) {
if (is_null($this->activeUrl)) {
$this->activeUrl = 0;
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/list.inc
+48 -8
View File
@@ -15,7 +15,9 @@
* obtain a list of IDs from an XML document).
*/
abstract class MigrateList {
public function __construct() {}
public function __construct() {
}
/**
* Implementors are expected to return a string representing where the listing
@@ -34,7 +36,8 @@ abstract class MigrateList {
abstract public function getIdList();
/**
* Implementors are expected to return a count of IDs available to be migrated.
* Implementors are expected to return a count of IDs available to be
* migrated.
*
* @return int
*/
@@ -46,7 +49,9 @@ abstract class MigrateList {
* given migratable item given its ID as provided by the MigrateList class.
*/
abstract class MigrateItem {
public function __construct() {}
public function __construct() {
}
/**
* Implementors are expected to return an object representing a source item.
@@ -56,6 +61,17 @@ abstract class MigrateItem {
* @return stdClass
*/
abstract public function getItem($id);
/**
* Implementors may optionally implement a hash function, applied to the
* entire source row, if this particular item type makes it difficult to
* do on the raw row.
*
* @param $row
*
* @return mixed
*/
//abstract public function hash($row);
}
/**
@@ -63,6 +79,7 @@ abstract class MigrateItem {
* IDs provided by a MigrateList and retrieving data from a MigrateItem.
*/
class MigrateSourceList extends MigrateSource {
/**
* MigrateList object used to obtain ID lists.
*
@@ -94,8 +111,7 @@ class MigrateSourceList extends MigrateSource {
/**
* Simple initialization.
*/
public function __construct(MigrateList $list_class, MigrateItem $item_class, $fields = array(),
$options = array()) {
public function __construct(MigrateList $list_class, MigrateItem $item_class, $fields = array(), $options = array()) {
parent::__construct($options);
$this->listClass = $list_class;
$this->itemClass = $item_class;
@@ -111,6 +127,15 @@ class MigrateSourceList extends MigrateSource {
return (string) $this->listClass;
}
/**
* Return MigrateList object used to obtain ID lists.
*
* @return MigrateList
*/
public function listClass() {
return $this->listClass;
}
/**
* Returns a list of fields available to be mapped from the source query.
* Since we can't reliably figure out what "fields" are in the source,
@@ -184,12 +209,27 @@ class MigrateSourceList extends MigrateSource {
foreach (array_keys($this->activeMap->getSourceKey()) as $key_name) {
// Grab the first id and advance the array cursor. Then save the ID
// using the map source key - it will be used for mapping.
list(, $id) = each($ids);
$row->$key_name = $id;
$id = current($ids);
next($ids);
$row->{$key_name} = $id;
}
break;
}
break;
}
return $row;
}
/**
* Overrides MigrateSource::hash().
*/
protected function hash($row) {
// Let the item class override the default hash function.
if (method_exists($this->itemClass, 'hash')) {
$hash = $this->itemClass->hash($row);
}
else {
$hash = parent::hash($row);
}
return $hash;
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/mongodb.inc
+191
View File
@@ -0,0 +1,191 @@
<?php
/**
* @file
* Define a MigrateSource for importing from MongoDB connections
*/
/**
* Implementation of MigrateSource, to handle imports from MongoDB connections.
*/
class MigrateSourceMongoDB extends MigrateSource {
/**
* The mongodb collection object.
*
* @var MongoCollection
*/
protected $collection;
/**
* The mongodb cursor object.
*
* @var MongoCursor
*/
protected $cursor;
/**
* The mongodb query.
*
* @var array
*/
protected $query;
/**
* List of available source fields.
*
* @var array
*/
protected $fields = array();
/**
* Simple initialization.
*/
public function __construct(MongoCollection $collection, array $query,
array $fields = array(), array $sort = array('_id' => 1),
array $options = array()) {
parent::__construct($options);
$this->collection = $collection;
$this->query = $query;
$this->sort = $sort;
$this->fields = $fields;
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
// The fields are passed to the constructor for this plugin.
return $this->fields;
}
/**
* Return a count of all available source records.
*/
public function computeCount() {
return $this->cursor->count(TRUE);
}
/**
* Implementation of MigrateSource::getNextRow().
*
* @return object
*/
public function getNextRow() {
$row = $this->cursor->getNext();
if ($row) {
return (object) $row;
}
return NULL;
}
/**
* Implementation of MigrateSource::performRewind().
*
* @return void
*/
public function performRewind() {
$keys = $this->getSourceKeyNameAndType();
// If we have an existing idlist we use it.
if ($this->idList) {
foreach ($this->idList as $key => $id) {
// Try make new ObjectID.
$this->idList[$key] = $this->getMongoId($id, $keys);
}
$this->query[$keys[0]['name']]['$in'] = $this->idList;
}
migrate_instrument_start('MigrateSourceMongoDB execute');
try {
$this->cursor = $this->collection
->find($this->query)
->sort($this->sort);
$this->cursor->timeout(-1);
} catch (MongoCursorException $e) {
Migration::displayMessage($e->getMessage(), 'error');
}
migrate_instrument_stop('MigrateSourceMongoDB execute');
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
if (is_null($this->cursor)) {
$this->cursor = $this->collection
->find($this->query)
->sort($this->sort);
$this->cursor->timeout(-1);
}
$query_info = $this->cursor->info();
$query = 'query: ' . drupal_json_encode($query_info['query']['$query']);
$sort = 'order by: ' . drupal_json_encode($query_info['query']['$orderby']);
$fields = 'fields: ' . drupal_json_encode($query_info['fields']);
return $query . PHP_EOL .
$sort . PHP_EOL .
$fields . PHP_EOL;
}
/**
* Check if given document id is a mongo ObjectId and return mongo ObjectId
* or simple value.
*
* @param mixed $document_id
* Document key value.
* @param array $keys
* List of keys.
*
* @return type
*/
public function getMongoId($document_id, $keys) {
if ($keys[0]['name'] != '_id') {
switch ($keys[0]['type']) {
case 'int':
return (int) $document_id;
break;
default:
return $document_id;
}
}
// Trying create Mongo ObjectId
$mongoid = new MongoId($document_id);
// If (string) $mongoid == $document_id we return $mongoid object
if ((string) $mongoid == $document_id) {
return $mongoid;
}
return $document_id;
}
/**
* Get source keys array.
*/
public function getSourceKeyNameAndType() {
// Get the key name, and type.
$keys = array();
foreach ($this->activeMap->getSourceKey() as $field_name => $field_schema) {
$keys[] = array(
'name' => $field_name,
'type' => $field_schema['type'],
);
}
return $keys;
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/mssql.inc
+6 -3
View File
@@ -6,9 +6,11 @@
*/
/**
* Implementation of MigrateSource, to handle imports from remote MS SQL Server db servers.
* Implementation of MigrateSource, to handle imports from remote MS SQL Server
* db servers.
*/
class MigrateSourceMSSQL extends MigrateSource {
/**
* Array containing information for connecting to SQL Server:
* servername - Hostname of the SQL Server
@@ -62,7 +64,7 @@ class MigrateSourceMSSQL extends MigrateSource {
* Simple initialization.
*/
public function __construct(array $configuration, $query, $count_query,
array $fields, array $options = array()) {
array $fields, array $options = array()) {
parent::__construct($options);
$this->query = $query;
$this->countQuery = $count_query;
@@ -125,7 +127,8 @@ class MigrateSourceMSSQL extends MigrateSource {
migrate_instrument_start('MigrateSourceMSSQL count');
if ($this->connect()) {
$result = mssql_query($this->countQuery);
$count = reset(mssql_fetch_object($result));
$result_array = mssql_fetch_array($result);
$count = reset($result_array);
}
else {
// Do something else?
sites/all/modules/contrib/migrate/migrate/plugins/sources/multiitems.inc
+36 -9
View File
@@ -14,7 +14,9 @@
* to obtain the data for a given migratable item given its ID.
*/
abstract class MigrateItems {
public function __construct() {}
public function __construct() {
}
/**
* Implementors are expected to return a string representing where the listing
@@ -33,7 +35,8 @@ abstract class MigrateItems {
abstract public function getIdList();
/**
* Implementors are expected to return a count of IDs available to be migrated.
* Implementors are expected to return a count of IDs available to be
* migrated.
*
* @return int
*/
@@ -47,8 +50,18 @@ abstract class MigrateItems {
* @return stdClass
*/
abstract public function getItem($id);
}
/**
* Implementors may optionally implement a hash function, applied to the
* entire source row, if this particular item type makes it difficult to
* do on the raw row.
*
* @param $row
*
* @return mixed
*/
//abstract public function hash($row);
}
/**
* Implementation of MigrateItems, for providing a list of IDs and for
@@ -60,6 +73,7 @@ abstract class MigrateItems {
* IDs provided by a MigrateItems and retrieving data from a MigrateItems.
*/
class MigrateSourceMultiItems extends MigrateSource {
/**
* MigrateItems object used to obtain the list of IDs and source for
* all objects.
@@ -127,7 +141,7 @@ class MigrateSourceMultiItems extends MigrateSource {
* @return int
*/
public function computeCount() {
// @API: Support old count method for now.
// @API: Support old count method for now.
if (method_exists($this->itemsClass, 'computeCount')) {
return $this->itemsClass->computeCount();
}
@@ -146,8 +160,8 @@ class MigrateSourceMultiItems extends MigrateSource {
if ($this->idList) {
$this->idsToProcess = $this->idList;
}
else {
$this->idsToProcess = $this->itemsClass->getIdList();
elseif (!$this->idsToProcess = $this->itemsClass->getIdList()) {
$this->idsToProcess = array();
}
$this->idIterator = ($this->idsToProcess instanceof Iterator) ?
$this->idsToProcess : new ArrayIterator($this->idsToProcess);
@@ -176,11 +190,24 @@ class MigrateSourceMultiItems extends MigrateSource {
// Save the ID using the map source key - it will be used for mapping
$sourceKey = $this->activeMap->getSourceKey();
$key_name = key($sourceKey);
$row->$key_name = $id;
$row->{$key_name} = $id;
break;
}
break;
}
return $row;
}
}
/**
* Overrides MigrateSource::hash().
*/
protected function hash($row) {
// Let the item class override the default hash function.
if (method_exists($this->itemsClass, 'hash')) {
$hash = $this->itemsClass->hash($row);
}
else {
$hash = parent::hash($row);
}
return $hash;
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/oracle.inc
+10 -6
View File
@@ -6,14 +6,17 @@
*/
/**
* Implementation of MigrateSource, to handle imports from remote Oracle servers.
* Implementation of MigrateSource, to handle imports from remote Oracle
* servers.
*/
class MigrateSourceOracle extends MigrateSource {
/**
* Array containing information for connecting to Oracle:
* username - Username to connect as
* password - Password for logging in
* connection_string - See http://us.php.net/manual/en/function.oci-connect.php.
* connection_string - See
* http://us.php.net/manual/en/function.oci-connect.php.
*
* @var array
*/
@@ -25,6 +28,7 @@ class MigrateSourceOracle extends MigrateSource {
* @var resource
*/
protected $connection;
public function getConnection() {
return $this->connection;
}
@@ -63,7 +67,7 @@ class MigrateSourceOracle extends MigrateSource {
* Simple initialization.
*/
public function __construct(array $configuration, $query, $count_query,
array $fields, array $options = array()) {
array $fields, array $options = array()) {
parent::__construct($options);
$this->query = $query;
$this->countQuery = $count_query;
@@ -206,13 +210,13 @@ class MigrateSourceOracle extends MigrateSource {
* Implementation of MigrateSource::getNextRow().
*
* Returns the next row of the result set as an object, making sure NULLs are
* represented as PHP NULLs and that LOBs are returned directly without special
* handling.
* represented as PHP NULLs and that LOBs are returned directly without
* special handling.
*/
public function getNextRow() {
$row = oci_fetch_array($this->result, OCI_ASSOC | OCI_RETURN_NULLS | OCI_RETURN_LOBS);
if (!empty($row)) {
return (object)$row;
return (object) $row;
}
else {
return FALSE;
sites/all/modules/contrib/migrate/migrate/plugins/sources/spreadsheet.inc
+253
View File
@@ -0,0 +1,253 @@
<?php
/**
* @file
* Define a MigrateSource for importing from spreadsheet files.
*
* Requires the PHPExcel library to be installed.
* - Download PHPExcel at http://phpexcel.codeplex.com/.
* - Extract the archive to a temporary folder.
* - Ensure the hosting environment fulfills the requirements found in
* install.txt.
* - Copy the contents of the Classes folder to an appropriate location
* (sites/all/libraries/PHPExcel).
*/
/**
* Implements MigrateSource, to handle imports from XLS files.
*/
class MigrateSourceSpreadsheet extends MigrateSource {
/**
* PHPExcel object for storing the workbook data.
*
* @var PHPExcel
*/
protected $workbook;
/**
* PHPExcel object for storing the worksheet data.
*
* @var PHPExcel_Worksheet
*/
protected $worksheet;
/**
* The name of the worksheet that will be processed.
*
* @var string
*/
protected $sheetName;
/**
* Number of rows in the worksheet that is being processed.
*
* @var integer
*/
protected $rows = 0;
/**
* Number of columns in the worksheet that is being processed.
*
* @var integer
*/
protected $cols = 0;
/**
* List of available source fields.
*
* @var array
*/
protected $fields = array();
/**
* The current row number in the XLS file.
*
* @var integer+
*/
protected $rowNumber;
/**
* The columns to be read from Excel
*/
protected $columns;
/**
* The first row from where the table starts. It's a "zero based" value.
*
* @var int
*/
protected $headerRows = 0;
/**
* Simple initialization.
*
* @param string $path
* The path to the source file.
* @param string $sheet_name
* The name of the sheet to be processed.
* @param array $options
* Options applied to this source. If the source data begins on different
* row than the first row, pass this "zero based" value as 'header_rows' in
* options, along with other options inherited from MigrateSource.
*/
public function __construct($path, $sheet_name, $columns = array(), array $options = array()) {
parent::__construct($options);
$this->file = $path;
$this->sheetName = $sheet_name;
$this->columns = $columns;
if (!empty($options['header_rows'])) {
$this->headerRows = $options['header_rows'];
}
// Load the workbook.
if ($this->load()) {
// Get the dimensions of the worksheet.
$this->rows = $this->worksheet->getHighestDataRow();
$this->cols = PHPExcel_Cell::columnIndexFromString($this->worksheet->getHighestDataColumn());
// Map field names to their column index.
for ($col = 0; $col < $this->cols; ++$col) {
$this->fields[$col] = trim($this->worksheet->getCellByColumnAndRow($col, $this->headerRows + 1)
->getValue());
}
$this->unload();
}
}
/**
* Loads the workbook.
*
* @return bool
* Returns true if the workbook was successfully loaded, otherwise false.
*/
public function load() {
// Check that the file exists.
if (!file_exists($this->file)) {
Migration::displayMessage(t('The file !filename does not exist.', array('!filename' => $this->file)));
return FALSE;
}
// Check that required modules are enabled.
if (!module_exists('libraries')) {
Migration::displayMessage(t('The Libraries API module is not enabled.'));
return FALSE;
}
if (!module_exists('phpexcel')) {
Migration::displayMessage(t('The PHPExcel module is not enabled.'));
return FALSE;
}
$library = libraries_load('PHPExcel');
if (empty($library['loaded'])) {
Migration::displayMessage(t('The PHPExcel library could not be found.'));
return FALSE;
}
// Load the workbook.
try {
// Identify the type of the input file.
$type = PHPExcel_IOFactory::identify($this->file);
// Create a new Reader of the file type.
$reader = PHPExcel_IOFactory::createReader($type);
// Advise the Reader that we only want to load cell data.
$reader->setReadDataOnly(TRUE);
// Advise the Reader of which worksheet we want to load.
$reader->setLoadSheetsOnly($this->sheetName);
// Load the source file.
$this->workbook = $reader->load($this->file);
$this->worksheet = $this->workbook->getSheet();
} catch (Exception $e) {
Migration::displayMessage(t('Error loading file: %message', array('%message' => $e->getMessage())));
return FALSE;
}
return TRUE;
}
/**
* Unloads the workbook.
*/
public function unload() {
$this->workbook->disconnectWorksheets();
unset($this->workbook);
}
/**
* Returns a string representing the source query.
*
* @return string
*/
public function __toString() {
return $this->file;
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping).
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
$fields = array();
foreach ($this->fields as $name) {
$fields[$name] = $name;
}
return $fields;
}
/**
* Returns a count of all available source records.
*/
public function computeCount() {
// Subtract the non-data rows (rows before header and the header).
return $this->rows - $this->headerRows - 1;
}
/**
* Implements MigrateSource::performRewind().
*
* @return void
*/
public function performRewind() {
// Initialize the workbook if it isn't already.
if (!isset($this->workbook)) {
$this->load();
}
$this->rowNumber = $this->headerRows + 1;
}
/**
* Implements MigrateSource::getNextRow().
*
* @return null|object
*/
public function getNextRow() {
migrate_instrument_start('MigrateSourceSpreadsheet::next');
++$this->rowNumber;
if ($this->rowNumber <= $this->rows) {
$row_values = array();
for ($col = 0; $col < $this->cols; ++$col) {
if (in_array($this->fields[$col], $this->columns) || empty($this->columns)) {
$row_values[$this->fields[$col]] = trim($this->worksheet->getCellByColumnAndRow($col, $this->rowNumber)
->getValue());
}
}
return (object) $row_values;
}
else {
// EOF, close the workbook.
$this->unload();
migrate_instrument_stop('MigrateSourceSpreadsheet::next');
return NULL;
}
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/sql.inc
+211 -57
View File
@@ -9,12 +9,13 @@
* Implementation of MigrateSource, to handle imports from Drupal connections.
*/
class MigrateSourceSQL extends MigrateSource {
/**
* The SQL query objects from which to obtain data, and counts of data
*
* @var SelectQueryInterface
*/
protected $originalQuery, $query, $countQuery;
protected $originalQuery, $query, $countQuery, $alteredQuery;
/**
* Return a reference to the base query, in particular so Migration classes
@@ -24,7 +25,7 @@ class MigrateSourceSQL extends MigrateSource {
* @return SelectQueryInterface
*/
public function &query() {
return $this->originalQuery();
return $this->originalQuery;
}
/**
@@ -42,6 +43,21 @@ class MigrateSourceSQL extends MigrateSource {
*/
protected $numProcessed = 0;
/**
* Current data batch.
*
* @var int
*/
protected $batch = 0;
/**
* Number of records to fetch from the database during each batch. A value
* of zero indicates no batching is to be done.
*
* @var int
*/
protected $batchSize = 0;
/**
* List of available source fields.
*
@@ -57,6 +73,7 @@ class MigrateSourceSQL extends MigrateSource {
* @var boolean
*/
protected $mapJoinable = FALSE;
// Dynamically set whether the map is joinable - not really for production use,
// this is primarily to support simpletests
public function setMapJoinable($map_joinable) {
@@ -82,7 +99,8 @@ class MigrateSourceSQL extends MigrateSource {
* Return an options array for PDO sources.
*
* @param boolean $map_joinable
* Indicates whether the map table can be joined directly to the source query.
* Indicates whether the map table can be joined directly to the source
* query.
* @param boolean $cache_counts
* Indicates whether to cache counts of source records.
*/
@@ -106,7 +124,7 @@ class MigrateSourceSQL extends MigrateSource {
* Options applied to this source.
*/
public function __construct(SelectQueryInterface $query, array $fields = array(),
SelectQueryInterface $count_query = NULL, array $options = array()) {
SelectQueryInterface $count_query = NULL, array $options = array()) {
parent::__construct($options);
$this->originalQuery = $query;
$this->query = clone $query;
@@ -118,6 +136,18 @@ class MigrateSourceSQL extends MigrateSource {
$this->countQuery = $count_query;
}
if (isset($options['batch_size'])) {
$this->batchSize = $options['batch_size'];
// Joining to the map table is incompatible with batching, disable it.
$options['map_joinable'] = FALSE;
}
// If we're tracking changes, then we need to fetch all rows to see if
// they've changed, we can't make that determination through a direct join.
if (!empty($options['track_changes'])) {
$options['map_joinable'] = FALSE;
}
if (isset($options['map_joinable'])) {
$this->mapJoinable = $options['map_joinable'];
}
@@ -126,41 +156,42 @@ class MigrateSourceSQL extends MigrateSource {
// directly to the query, but this won't work unless/until
// http://drupal.org/node/802514 is committed, assume joinable for now
$this->mapJoinable = TRUE;
/* // To be able to join the map directly, it must be a PDO map on the same
// connection, or a compatible connection
$map = $migration->getMap();
if (is_a($map, 'MigrateSQLMap')) {
$map_options = $map->getConnection()->getConnectionOptions();
$query_options = $this->query->connection()->getConnectionOptions();
/* // To be able to join the map directly, it must be a PDO map on the same
// connection, or a compatible connection
$map = $migration->getMap();
if (is_a($map, 'MigrateSQLMap')) {
$map_options = $map->getConnection()->getConnectionOptions();
$query_options = $this->query->connection()->getConnectionOptions();
// Identical options means it will work
if ($map_options == $query_options) {
$this->mapJoinable = TRUE;
}
else {
// Otherwise, the one scenario we know will work is if it's MySQL and
// the credentials match (SQLite too?)
if ($map_options['driver'] == 'mysql' && $query_options['driver'] == 'mysql') {
if ($map_options['host'] == $query_options['host'] &&
$map_options['port'] == $query_options['port'] &&
$map_options['username'] == $query_options['username'] &&
$map_options['password'] == $query_options['password']) {
$this->mapJoinable = TRUE;
}
}
}
}*/
// Identical options means it will work
if ($map_options == $query_options) {
$this->mapJoinable = TRUE;
}
else {
// Otherwise, the one scenario we know will work is if it's MySQL and
// the credentials match (SQLite too?)
if ($map_options['driver'] == 'mysql' && $query_options['driver'] == 'mysql') {
if ($map_options['host'] == $query_options['host'] &&
$map_options['port'] == $query_options['port'] &&
$map_options['username'] == $query_options['username'] &&
$map_options['password'] == $query_options['password']) {
$this->mapJoinable = TRUE;
}
}
}
}*/
}
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
return (string) $this->query;
$query = clone $this->query;
$query = $query->extend('MigrateConnectionQuery');
return $query->getString();
}
/**
@@ -192,26 +223,26 @@ class MigrateSourceSQL extends MigrateSource {
}
}
/*
* Handle queries without explicit field lists
* TODO: Waiting on http://drupal.org/node/814312
$info = Database::getConnectionInfo($query->getConnection());
$database = $info['default']['database'];
foreach ($this->query->getTables() as $table) {
if (isset($table['all_fields']) && $table['all_fields']) {
/*
* Handle queries without explicit field lists
* TODO: Waiting on http://drupal.org/node/814312
$info = Database::getConnectionInfo($query->getConnection());
$database = $info['default']['database'];
foreach ($this->query->getTables() as $table) {
if (isset($table['all_fields']) && $table['all_fields']) {
$database = 'plants';
$table = $table['table'];
$sql = 'SELECT column_name
FROM information_schema.columns
WHERE table_schema=:database AND table_name = :table
ORDER BY ordinal_position';
$result = dbtng_query($sql, array(':database' => $database, ':table' => $table));
foreach ($result as $row) {
$fields[$row->column_name] = $table . '.' . $row->column_name;
}
}
}*/
$database = 'plants';
$table = $table['table'];
$sql = 'SELECT column_name
FROM information_schema.columns
WHERE table_schema=:database AND table_name = :table
ORDER BY ordinal_position';
$result = dbtng_query($sql, array(':database' => $database, ':table' => $table));
foreach ($result as $row) {
$fields[$row->column_name] = $table . '.' . $row->column_name;
}
}
}*/
$expressionFields = $this->query->getExpressions();
foreach ($expressionFields as $field_name => $field_info) {
$fields[$field_name] = $field_info['alias'];
@@ -243,6 +274,7 @@ class MigrateSourceSQL extends MigrateSource {
public function performRewind() {
$this->result = NULL;
$this->query = clone $this->originalQuery;
$this->batch = 0;
// Get the key values, for potential use in joining to the map table, or
// enforcing idlist.
@@ -259,7 +291,41 @@ class MigrateSourceSQL extends MigrateSource {
// 1. If idlist is provided, then only process items in that list (AND key
// IN (idlist)). Only applicable with single-value keys.
if ($this->idList) {
$this->query->condition($keys[0], $this->idList, 'IN');
$simple_ids = array();
$compound_ids = array();
$key_count = count($keys);
foreach ($this->idList as $id) {
// Look for multi-key separator. If there is only 1 key, ignore.
if (strpos($id, $this->multikeySeparator) === FALSE || $key_count == 1) {
$simple_ids[] = $id;
continue;
}
$compound_ids[] = explode($this->multikeySeparator, $id);
}
// Check for compunded ids. If present add them with subsequent OR statements.
if (!empty($compound_ids)) {
$condition = db_or();
if (!empty($simple_ids)) {
$condition->condition($keys[0], $simple_ids, 'IN');
}
foreach ($compound_ids as $values) {
$temp_and = db_and();
foreach ($values as $pos => $value) {
$temp_and->condition($keys[$pos], $value);
}
$condition->condition($temp_and);
}
$this->query->condition($condition);
}
else {
$this->query->condition($keys[0], $simple_ids, 'IN');
}
}
else {
// 2. If the map is joinable, join it. We will want to accept all rows
@@ -292,7 +358,7 @@ class MigrateSourceSQL extends MigrateSource {
}
$alias = $this->query->leftJoin($this->activeMap->getQualifiedMapTable(),
'map', $map_join);
'map', $map_join);
$conditions->isNull($alias . '.sourceid1');
$conditions->condition($alias . '.needs_update', MigrateMap::STATUS_NEEDS_UPDATE);
$condition_added = TRUE;
@@ -313,20 +379,43 @@ class MigrateSourceSQL extends MigrateSource {
// 3. If we are using highwater marks, also include rows above the mark.
// But, include all rows if the highwater mark is not set.
if (isset($this->highwaterField['name']) && $this->activeMigration->getHighwater() !== '') {
if (isset($this->highwaterField['alias'])) {
$highwater = $this->highwaterField['alias'] . '.' . $this->highwaterField['name'];
// But, if there are any existing items marked as needing update which
// fall below the highwater mark, and map_joinable is FALSE, those
// items will be skipped. Thus, in that case do not add the highwater
// optimization to the query.
$add_highwater_condition = TRUE;
if (!$this->mapJoinable) {
$count_needs_update = db_query('SELECT COUNT(*) FROM {' .
$this->activeMap->getQualifiedMapTable() . '} WHERE needs_update = 1')
->fetchField();
if ($count_needs_update > 0) {
$add_highwater_condition = FALSE;
}
}
else {
$highwater = $this->highwaterField['name'];
if ($add_highwater_condition) {
if (isset($this->highwaterField['alias'])) {
$highwater = $this->highwaterField['alias'] . '.' . $this->highwaterField['name'];
}
else {
$highwater = $this->highwaterField['name'];
}
$conditions->condition($highwater, $this->activeMigration->getHighwater(), '>');
$condition_added = TRUE;
}
$conditions->condition($highwater, $this->activeMigration->getHighwater(), '>');
$condition_added = TRUE;
}
if ($condition_added) {
$this->query->condition($conditions);
}
// 4. Download data in batches for performance.
if ($this->batchSize > 0) {
$this->query->range($this->batch * $this->batchSize, $this->batchSize);
}
}
// Save our fixed-up query so getNextBatch() matches it.
$this->alteredQuery = clone $this->query;
migrate_instrument_start('MigrateSourceSQL execute');
$this->result = $this->query->execute();
migrate_instrument_stop('MigrateSourceSQL execute');
@@ -338,6 +427,71 @@ class MigrateSourceSQL extends MigrateSource {
* @return object
*/
public function getNextRow() {
return $this->result->fetchObject();
$row = $this->result->fetchObject();
// We might be out of data entirely, or just out of data in the current batch.
// Attempt to fetch the next batch and see.
if (!is_object($row) && $this->batchSize > 0) {
$this->getNextBatch();
$row = $this->result->fetchObject();
}
if (is_object($row)) {
return $row;
}
else {
return NULL;
}
}
/**
* Downloads the next set of data from the source database.
*/
protected function getNextBatch() {
$this->batch++;
$query = clone $this->alteredQuery;
$query->range($this->batch * $this->batchSize, $this->batchSize);
$this->result = $query->execute();
}
}
/**
* Query extender for retrieving the connection used on the query.
*/
class MigrateConnectionQuery extends SelectQueryExtender {
public function __construct(SelectQueryInterface $query, DatabaseConnection $connection) {
parent::__construct($query, $connection);
// Add the connection as metadata if anything else wants to access it.
$query->addMetaData('connection', $connection);
}
/**
* Return a string representing the source query.
*
* This is copied from devel module's dpq() function.
*
* @param bool $prefix
* If the tables should be prefixed. If FALSE will return tables names in
* the query like {tablename}.
*
* @return string
* The SQL query.
*/
public function getString($prefix = TRUE) {
$query = $this;
if (method_exists($this, 'preExecute')) {
$query->preExecute();
}
$sql = (string) $this;
$quoted = array();
foreach ((array) $this->arguments() as $key => $val) {
$quoted[$key] = $this->connection->quote($val);
}
$sql = strtr($sql, $quoted);
if ($prefix) {
$sql = $this->connection->prefixTables($sql);
}
return $sql;
}
}
sites/all/modules/contrib/migrate/migrate/plugins/sources/sqlmap.inc
+167 -42
View File
@@ -6,15 +6,18 @@
*/
class MigrateSQLMap extends MigrateMap {
/**
* Names of tables created for tracking the migration.
*
* @var string
*/
protected $mapTable, $messageTable;
public function getMapTable() {
return $this->mapTable;
}
public function getMessageTable() {
return $this->messageTable;
}
@@ -36,7 +39,13 @@ class MigrateSQLMap extends MigrateMap {
return $this->mapTable;
}
else {
return $options['database'] . '.' . $this->mapTable;
$dbname = $options['database'];
if (!empty($options['driver']) && $options['driver'] == 'mysql') {
// Backtick the db name on mysql to ensure dbs with hyphens work.
$dbname = "`{$dbname}`";
}
return $dbname . '.' . $this->mapTable;
}
}
@@ -49,15 +58,18 @@ class MigrateSQLMap extends MigrateMap {
public function getSourceKey() {
return $this->sourceKey;
}
public function getDestinationKey() {
return $this->destinationKey;
}
/**
* Drupal connection object on which to create the map/message tables
*
* @var DatabaseConnection
*/
protected $connection;
public function getConnection() {
return $this->connection;
}
@@ -69,19 +81,52 @@ class MigrateSQLMap extends MigrateMap {
*/
protected $ensured;
/**
* Provide caching for Source or Desination Map Lookups.
*/
protected $cacheMapLookups;
/**
* Constructor.
*
* @param string $machine_name
* The unique reference to the migration that we are mapping.
* @param array $source_key
* The database schema for the source key.
* @param array $destination_key
* The database schema for the destination key.
* @param string $connection_key
* Optional - The connection used to create the mapping tables. By default
* this is the destination (Drupal). If it's not possible to make joins
* between the destination database and your source database you can specify
* a different connection to create the mapping tables on.
* @param array $options
* Optional - Options applied to this source.
*/
public function __construct($machine_name, array $source_key,
array $destination_key, $connection_key = 'default', $options = array()) {
array $destination_key, $connection_key = 'default', $options = array()) {
if (isset($options['track_last_imported'])) {
$this->trackLastImported = TRUE;
}
if (isset($options['cache_map_lookups'])) {
$this->cacheMapLookups = $options['cache_map_lookups'];
}
else {
$this->cacheMapLookups = FALSE;
}
$this->connection = Database::getConnection('default', $connection_key);
// Default generated table names, limited to 63 characters
$prefixLength = strlen($this->connection->tablePrefix());
$this->mapTable = 'migrate_map_' . drupal_strtolower($machine_name);
$this->mapTable = drupal_substr($this->mapTable, 0, 63);
$this->mapTable = drupal_substr($this->mapTable, 0, 63 - $prefixLength);
$this->messageTable = 'migrate_message_' . drupal_strtolower($machine_name);
$this->messageTable = drupal_substr($this->messageTable, 0, 63);
$this->messageTable = drupal_substr($this->messageTable, 0, 63 - $prefixLength);
$this->sourceKey = $source_key;
$this->destinationKey = $destination_key;
$this->connection = Database::getConnection('default', $connection_key);
// Build the source and destination key maps
$this->sourceKeyMap = array();
$count = 1;
@@ -110,6 +155,7 @@ class MigrateSQLMap extends MigrateMap {
foreach ($this->sourceKey as $field_schema) {
$mapkey = 'sourceid' . $count++;
$source_key_schema[$mapkey] = $field_schema;
$source_key_schema[$mapkey]['not null'] = TRUE;
$pks[] = $mapkey;
}
@@ -121,6 +167,7 @@ class MigrateSQLMap extends MigrateMap {
foreach ($this->destinationKey as $field_schema) {
// Allow dest key fields to be NULL (for IGNORED/FAILED cases)
$field_schema['not null'] = FALSE;
$field_schema['default'] = NULL;
$mapkey = 'destid' . $count++;
$fields[$mapkey] = $field_schema;
}
@@ -147,6 +194,12 @@ class MigrateSQLMap extends MigrateMap {
'default' => 0,
'description' => 'UNIX timestamp of the last time this row was imported',
);
$fields['hash'] = array(
'type' => 'varchar',
'length' => '32',
'not null' => FALSE,
'description' => 'Hash of source row data, for detecting changes',
);
$schema = array(
'description' => t('Mappings from source key to destination key'),
'fields' => $fields,
@@ -182,6 +235,30 @@ class MigrateSQLMap extends MigrateMap {
);
$this->connection->schema()->createTable($this->messageTable, $schema);
}
else {
// Add any missing columns to the map table
if (!$this->connection->schema()->fieldExists($this->mapTable,
'rollback_action')) {
$this->connection->schema()->addField($this->mapTable,
'rollback_action', array(
'type' => 'int',
'size' => 'tiny',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => 0,
'description' => 'Flag indicating what to do for this item on rollback',
));
}
if (!$this->connection->schema()
->fieldExists($this->mapTable, 'hash')) {
$this->connection->schema()->addField($this->mapTable, 'hash', array(
'type' => 'varchar',
'length' => '32',
'not null' => FALSE,
'description' => 'Hash of source row data, for detecting changes',
));
}
}
$this->ensured = TRUE;
}
}
@@ -194,7 +271,7 @@ class MigrateSQLMap extends MigrateMap {
public function getRowBySource(array $source_id) {
migrate_instrument_start('mapRowBySource');
$query = $this->connection->select($this->mapTable, 'map')
->fields('map');
->fields('map');
foreach ($this->sourceKeyMap as $key_name) {
$query = $query->condition("map.$key_name", array_shift($source_id), '=');
}
@@ -211,7 +288,7 @@ class MigrateSQLMap extends MigrateMap {
public function getRowByDestination(array $destination_id) {
migrate_instrument_start('getRowByDestination');
$query = $this->connection->select($this->mapTable, 'map')
->fields('map');
->fields('map');
foreach ($this->destinationKeyMap as $key_name) {
$query = $query->condition("map.$key_name", array_shift($destination_id), '=');
}
@@ -225,16 +302,17 @@ class MigrateSQLMap extends MigrateMap {
*
* @param int $count
* Maximum rows to return; defaults to 10,000
*
* @return array
* Array of map row objects with needs_update==1.
*/
public function getRowsNeedingUpdate($count) {
$rows = array();
$result = $this->connection->select($this->mapTable, 'map')
->fields('map')
->condition('needs_update', MigrateMap::STATUS_NEEDS_UPDATE)
->range(0, $count)
->execute();
->fields('map')
->condition('needs_update', MigrateMap::STATUS_NEEDS_UPDATE)
->range(0, $count)
->execute();
foreach ($result as $row) {
$rows[] = $row;
}
@@ -242,45 +320,79 @@ class MigrateSQLMap extends MigrateMap {
}
/**
* Given a (possibly multi-field) destination key, return the (possibly multi-field)
* source key mapped to it.
* Given a (possibly multi-field) destination key, return the (possibly
* multi-field) source key mapped to it.
*
* @param array $destination_id
* Array of destination key values.
*
* @return array
* Array of source key values, or NULL on failure.
*/
public function lookupSourceID(array $destination_id) {
migrate_instrument_start('lookupSourceID');
// Try a cache lookup if enabled.
if ($this->cacheMapLookups) {
$cache = &drupal_static($this->mapTable . '_sourceIDCache');
$serialized = json_encode($destination_id);
if (isset($cache[$serialized])) {
migrate_instrument_stop('lookupSourceID');
return $cache[$serialized];
}
}
$query = $this->connection->select($this->mapTable, 'map')
->fields('map', $this->sourceKeyMap);
->fields('map', $this->sourceKeyMap);
foreach ($this->destinationKeyMap as $key_name) {
$query = $query->condition("map.$key_name", array_shift($destination_id), '=');
}
$result = $query->execute();
$source_id = $result->fetchAssoc();
// Store the id in a cache if enabled.
if ($this->cacheMapLookups) {
$cache[$serialized] = $destination_id;
}
migrate_instrument_stop('lookupSourceID');
return $source_id;
}
/**
* Given a (possibly multi-field) source key, return the (possibly multi-field)
* destination key it is mapped to.
* Given a (possibly multi-field) source key, return the (possibly
* multi-field) destination key it is mapped to.
*
* @param array $source_id
* Array of source key values.
*
* @return array
* Array of destination key values, or NULL on failure.
*/
public function lookupDestinationID(array $source_id) {
migrate_instrument_start('lookupDestinationID');
// Try a cache lookup if enabled.
if ($this->cacheMapLookups) {
$cache = &drupal_static($this->mapTable . '_destinationIDCache');
$serialized = json_encode($source_id);
if (isset($cache[$serialized])) {
migrate_instrument_stop('lookupDestinationID');
return $cache[$serialized];
}
}
$query = $this->connection->select($this->mapTable, 'map')
->fields('map', $this->destinationKeyMap);
->fields('map', $this->destinationKeyMap);
foreach ($this->sourceKeyMap as $key_name) {
$query = $query->condition("map.$key_name", array_shift($source_id), '=');
}
$result = $query->execute();
$destination_id = $result->fetchAssoc();
// Store the id in a cache if enabled.
if ($this->cacheMapLookups) {
$cache[$serialized] = $destination_id;
}
migrate_instrument_stop('lookupDestinationID');
return $destination_id;
}
@@ -300,27 +412,31 @@ class MigrateSQLMap extends MigrateMap {
* @param int $rollback_action
* How to handle the destination object on rollback. Defaults to
* ROLLBACK_DELETE.
* $param string $hash
* If hashing is enabled, the hash of the raw source row.
*/
public function saveIDMapping(stdClass $source_row, array $dest_ids,
$needs_update = MigrateMap::STATUS_IMPORTED, $rollback_action = MigrateMap::ROLLBACK_DELETE) {
$needs_update = MigrateMap::STATUS_IMPORTED,
$rollback_action = MigrateMap::ROLLBACK_DELETE, $hash = NULL) {
migrate_instrument_start('saveIDMapping');
// Construct the source key
$keys = array();
foreach ($this->sourceKeyMap as $field_name => $key_name) {
// A NULL key value will fail.
if (is_null($source_row->$field_name)) {
if (is_null($source_row->{$field_name})) {
Migration::displayMessage(t(
'Could not save to map table due to NULL value for key field !field',
array('!field' => $field_name)));
migrate_instrument_stop('saveIDMapping');
return;
}
$keys[$key_name] = $source_row->$field_name;
$keys[$key_name] = $source_row->{$field_name};
}
$fields = array(
'needs_update' => (int)$needs_update,
'rollback_action' => (int)$rollback_action,
'needs_update' => (int) $needs_update,
'rollback_action' => (int) $rollback_action,
'hash' => $hash,
);
$count = 1;
if (!empty($dest_ids)) {
@@ -354,8 +470,8 @@ class MigrateSQLMap extends MigrateMap {
if (is_array($source_key)) {
foreach ($source_key as $key_value) {
$fields['sourceid' . $count++] = $key_value;
// If any key value is empty, we can't save - print out and abort
if (empty($key_value)) {
// If any key value is not set, we can't save - print out and abort
if (!isset($key_value)) {
print($message);
return;
}
@@ -379,8 +495,8 @@ class MigrateSQLMap extends MigrateMap {
*/
public function prepareUpdate() {
$this->connection->update($this->mapTable)
->fields(array('needs_update' => MigrateMap::STATUS_NEEDS_UPDATE))
->execute();
->fields(array('needs_update' => MigrateMap::STATUS_NEEDS_UPDATE))
->execute();
}
/**
@@ -404,7 +520,10 @@ class MigrateSQLMap extends MigrateMap {
public function importedCount() {
$query = $this->connection->select($this->mapTable);
$query->addExpression('COUNT(*)', 'count');
$query->condition('needs_update', array(MigrateMap::STATUS_IMPORTED, MigrateMap::STATUS_NEEDS_UPDATE), 'IN');
$query->condition('needs_update', array(
MigrateMap::STATUS_IMPORTED,
MigrateMap::STATUS_NEEDS_UPDATE,
), 'IN');
$count = $query->execute()->fetchField();
return $count;
}
@@ -450,7 +569,8 @@ class MigrateSQLMap extends MigrateMap {
}
/**
* Delete the map entry and any message table entries for the specified source row.
* Delete the map entry and any message table entries for the specified
* source row.
*
* @param array $source_key
*/
@@ -475,7 +595,8 @@ class MigrateSQLMap extends MigrateMap {
}
/**
* Delete the map entry and any message table entries for the specified destination row.
* Delete the map entry and any message table entries for the specified
* destination row.
*
* @param array $destination_key
*/
@@ -504,7 +625,7 @@ class MigrateSQLMap extends MigrateMap {
*/
public function setUpdate(array $source_key) {
$query = $this->connection->update($this->mapTable)
->fields(array('needs_update' => MigrateMap::STATUS_NEEDS_UPDATE));
->fields(array('needs_update' => MigrateMap::STATUS_NEEDS_UPDATE));
$count = 1;
foreach ($source_key as $key_value) {
$query->condition('sourceid' . $count++, $key_value);
@@ -552,7 +673,7 @@ class MigrateSQLMap extends MigrateMap {
*/
public function clearMessages() {
$this->connection->truncate($this->messageTable)
->execute();
->execute();
}
/**
@@ -564,15 +685,18 @@ class MigrateSQLMap extends MigrateMap {
}
protected $result = NULL;
protected $currentRow = NULL;
protected $currentKey = array();
public function getCurrentKey() {
return $this->currentKey;
}
/**
* Implementation of Iterator::rewind() - called before beginning a foreach loop.
* TODO: Support idlist, itemlimit
* Implementation of Iterator::rewind() - called before beginning a foreach
* loop. TODO: Support idlist, itemlimit
*/
public function rewind() {
$this->currentRow = NULL;
@@ -604,17 +728,18 @@ class MigrateSQLMap extends MigrateMap {
}
/**
* Implementation of Iterator::key - called when entering a loop iteration, returning
* the key of the current row. It must be a scalar - we will serialize
* to fulfill the requirement, but using getCurrentKey() is preferable.
* Implementation of Iterator::key - called when entering a loop iteration,
* returning the key of the current row. It must be a scalar - we will
* serialize to fulfill the requirement, but using getCurrentKey() is
* preferable.
*/
public function key() {
return serialize($this->currentKey);
}
/**
* Implementation of Iterator::next() - called at the bottom of the loop implicitly,
* as well as explicitly from rewind().
* Implementation of Iterator::next() - called at the bottom of the loop
* implicitly, as well as explicitly from rewind().
*/
public function next() {
$this->currentRow = $this->result->fetchObject();
@@ -624,16 +749,16 @@ class MigrateSQLMap extends MigrateMap {
}
else {
foreach ($this->sourceKeyMap as $map_field) {
$this->currentKey[$map_field] = $this->currentRow->$map_field;
$this->currentKey[$map_field] = $this->currentRow->{$map_field};
// Leave only destination fields
unset($this->currentRow->$map_field);
unset($this->currentRow->{$map_field});
}
}
}
/**
* Implementation of Iterator::valid() - called at the top of the loop, returning
* TRUE to process the loop and FALSE to terminate it
* Implementation of Iterator::valid() - called at the top of the loop,
* returning TRUE to process the loop and FALSE to terminate it
*/
public function valid() {
// TODO: Check numProcessed against itemlimit
sites/all/modules/contrib/migrate/migrate/plugins/sources/xml.inc
+538 -210
View File
File diff suppressed because it is too large Load Diff
sites/all/modules/contrib/migrate/migrate/tests/import/options.test
+2 -5
View File
@@ -9,6 +9,7 @@
* Test node migration.
*/
class MigrateImportOptionsTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Import options',
@@ -21,7 +22,7 @@ class MigrateImportOptionsTest extends DrupalWebTestCase {
parent::setUp('migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testItemLimitOption() {
@@ -40,17 +41,13 @@ class MigrateImportOptionsTest extends DrupalWebTestCase {
$result = $migration->processImport($options);
$this->verbose(print_r($timers, 1));
$successes = $migration->importedCount();
$this->verbose("Total successes: {$successes}");
$assertion = format_plural($limit, 'The migration successfully processed 1 item.',
'The migration successfully processed @count items.');
$this->assertEqual($limit, $successes, $assertion);
$prepare_row_count = $timers['BeerTermMigration prepareRow']['count'];
$this->verbose("prepareRow() count: {$prepare_row_count}");
$processed = $migration->processedCount();
$this->verbose("Total processed count: {$processed}");
$assertion = format_plural($processed, 'The migration executed processRow() on 1 item.',
'The migration executed processRow() on @count items.');
$this->assertEqual($prepare_row_count, $processed, $assertion);
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/comment.test
+39 -22
View File
@@ -9,6 +9,7 @@
* Test comment migration.
*/
class MigrateCommentUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Comment migration',
@@ -21,7 +22,7 @@ class MigrateCommentUnitTest extends DrupalWebTestCase {
parent::setUp('taxonomy', 'image', 'comment', 'migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testCommentImport() {
@@ -62,18 +63,29 @@ class MigrateCommentUnitTest extends DrupalWebTestCase {
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Comment import returned RESULT_COMPLETED'));
$result = db_select('migrate_message_winecomment', 'w')
->fields('w', array('message'))
->execute();
->fields('w', array('message'))
->execute();
foreach ($result as $row) {
$this->error($row->message);
}
$result = db_select('migrate_example_wine_comment', 'wc')
->fields('wc', array('commentid', 'comment_parent', 'name', 'mail',
'accountid', 'body', 'wineid', 'subject', 'commenthost', 'userpage',
'posted', 'lastchanged'))
->orderBy('comment_parent')
->execute();
->fields('wc', array(
'commentid',
'comment_parent',
'name',
'mail',
'accountid',
'body',
'wineid',
'subject',
'commenthost',
'userpage',
'posted',
'lastchanged',
))
->orderBy('comment_parent')
->execute();
$rawcomments = comment_load_multiple(FALSE);
// Index by subject
@@ -96,7 +108,8 @@ class MigrateCommentUnitTest extends DrupalWebTestCase {
$this->assertEqual($comment->name, $row->name, t('Name matches'));
$this->assertEqual($comment->status, COMMENT_PUBLISHED, t('Status matches'));
$wine_migration = MigrationBase::getInstance('WineWine');
$destid = $wine_migration->getMap()->lookupDestinationID(array($row->wineid));
$destid = $wine_migration->getMap()
->lookupDestinationID(array($row->wineid));
$this->assertEqual($comment->nid, reset($destid), t('Nid matches'));
$body = field_get_items('comment', $comment, 'comment_body');
$this->assertEqual($body[0]['value'], $row->body, t('Body matches'));
@@ -108,7 +121,8 @@ class MigrateCommentUnitTest extends DrupalWebTestCase {
$comment = $comments['im child'];
$row = $rows['im child'];
$user_migration = MigrationBase::getInstance('WineUser');
$destid = $user_migration->getMap()->lookupDestinationID(array($row->accountid));
$destid = $user_migration->getMap()
->lookupDestinationID(array($row->accountid));
$this->assertEqual($comment->uid, reset($destid), t('Uid matches'));
$this->assertEqual($comment->pid, $comments['im parent']->cid, t('Parent matches'));
@@ -123,16 +137,19 @@ class MigrateCommentUnitTest extends DrupalWebTestCase {
foreach ($original_comments as $cid => $original_comment) {
foreach ($original_comment as $field => $value) {
if ($field == 'subject') {
if ($value == $final_comments[$cid]->$field) {
if ($value == $final_comments[$cid]->{$field}) {
$this->error(t('Field !field should have changed but did not, value=!value',
array('!field' => $field, '!value' => print_r($value, TRUE))));
}
}
else {
if ($value != $final_comments[$cid]->$field) {
if ($value != $final_comments[$cid]->{$field}) {
$this->error(t('Field !field mismatch: original !value1, result !value2',
array('!field' => $field, '!value1' => print_r($value, TRUE),
'!value2' => print_r($final_comments[$cid]->$field, TRUE))));
array(
'!field' => $field,
'!value1' => print_r($value, TRUE),
'!value2' => print_r($final_comments[$cid]->{$field}, TRUE),
)));
}
}
}
@@ -145,16 +162,16 @@ class MigrateCommentUnitTest extends DrupalWebTestCase {
$rawcomments = comment_load_multiple(FALSE);
$this->assertEqual(count($rawcomments), 0, t('All comments deleted'));
$count = db_select('migrate_map_winecomment', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_winecomment', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
}
}
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/node.test
+112 -41
View File
@@ -9,6 +9,7 @@
* Test node migration.
*/
class MigrateNodeUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Node migration',
@@ -21,7 +22,7 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
parent::setUp('list', 'number', 'taxonomy', 'image', 'migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testNodeImport() {
@@ -67,7 +68,13 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
}
$query = db_select('migrate_example_wine_producer', 'p')
->fields('p', array('producerid', 'name', 'body', 'excerpt', 'accountid'));
->fields('p', array(
'producerid',
'name',
'body',
'excerpt',
'accountid',
));
// Region term is singletons, handled straighforwardly
$query->leftJoin('migrate_example_wine_category_producer', 'reg',
"p.producerid = reg.producerid");
@@ -87,8 +94,17 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
$wine_nodes[$node->title] = $node;
}
$query = db_select('migrate_example_wine', 'w')
->fields('w', array('wineid', 'name', 'body', 'excerpt', 'accountid',
'posted', 'last_changed', 'variety', 'region'));
->fields('w', array(
'wineid',
'name',
'body',
'excerpt',
'accountid',
'posted',
'last_changed',
'variety',
'region',
));
$query->leftJoin('migrate_example_wine_category_wine', 'cwbw',
"w.wineid = cwbw.wineid");
$query->leftJoin('migrate_example_wine_categories', 'bw',
@@ -111,7 +127,8 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
$wine_row = $wine_rows['Montes Classic Cabernet Sauvignon'];
$user_migration = MigrationBase::getInstance('WineUser');
$mapped_uid = $user_migration->getMap()->lookupDestinationID(array($producer_row->accountid));
$mapped_uid = $user_migration->getMap()
->lookupDestinationID(array($producer_row->accountid));
if (is_array($mapped_uid)) {
$this->assertEqual($producer_node->uid, reset($mapped_uid),
t('uid properly migrated'));
@@ -134,7 +151,8 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
// taxonomy_term_reference - single and multiple
$variety = field_get_items('node', $wine_node, 'migrate_example_wine_varieties');
$variety_migration = MigrationBase::getInstance('WineVariety');
$mapped_tid = $variety_migration->getMap()->lookupDestinationID(array($wine_row->variety));
$mapped_tid = $variety_migration->getMap()
->lookupDestinationID(array($wine_row->variety));
if (is_array($mapped_tid)) {
$this->assertEqual($variety[0]['tid'], reset($mapped_tid),
t('Single taxonomy_term_reference properly migrated'));
@@ -147,7 +165,8 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
$source_ids = explode(',', $wine_row->best_with);
$mapped_tids = array();
foreach ($source_ids as $source_id) {
$tid = $best_with_migration->getMap()->lookupDestinationID(array($source_id));
$tid = $best_with_migration->getMap()
->lookupDestinationID(array($source_id));
if ($tid) {
$mapped_tids[reset($tid)] = reset($tid);
}
@@ -176,33 +195,66 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
foreach ($original_nodes as $nid => $original_node) {
foreach ($original_node as $field => $value) {
if ($field == 'field_migrate_example_wine_ratin' || $field == 'changed' || $field == 'revision_timestamp') {
if ($value == $final_nodes[$nid]->$field) {
if ($value == $final_nodes[$nid]->{$field}) {
$this->error(t('Field !field should have changed but did not, value=!value',
array('!field' => $field, '!value' => print_r($value, TRUE))));
}
}
else {
if ($value != $final_nodes[$nid]->$field) {
if ($value != $final_nodes[$nid]->{$field}) {
$this->error(t('Field !field mismatch: original !value1, result !value2',
array('!field' => $field, '!value1' => print_r($value, TRUE),
'!value2' => print_r($final_nodes[$nid]->$field, TRUE))));
array(
'!field' => $field,
'!value1' => print_r($value, TRUE),
'!value2' => print_r($final_nodes[$nid]->{$field}, TRUE),
)));
}
}
}
}
// Test highwater marks - add new wines, modify an old one, and see what changes
$fields = array('wineid', 'name', 'body', 'excerpt', 'accountid',
'posted', 'last_changed', 'variety', 'region', 'rating');
$fields = array(
'wineid',
'name',
'body',
'excerpt',
'accountid',
'posted',
'last_changed',
'variety',
'region',
'rating',
);
$query = db_insert('migrate_example_wine')
->fields($fields);
->fields($fields);
$data = array(
// Timestamps 1284008523, 1284120550
array(3, 'Schloss Muhlenhof Dornfelder', 'Juicy black & red berry fruits', 'Pretty good', 9,
strtotime('2010-09-09 01:02:03'), strtotime('2010-09-10 08:09:10'), 25, 17, 95),
array(
3,
'Schloss Muhlenhof Dornfelder',
'Juicy black & red berry fruits',
'Pretty good',
9,
strtotime('2010-09-09 01:02:03'),
strtotime('2010-09-10 08:09:10'),
25,
17,
95,
),
// Timestamps 1286122209, 1286122209
array(4, 'Gachot-Monot Bourgogne Rge 06', 'Funky', 'Pair with white sauced dishes', 3,
strtotime('2010-10-03 12:10:09'), strtotime('2010-10-03 12:10:09'), 26, 2, 85),
array(
4,
'Gachot-Monot Bourgogne Rge 06',
'Funky',
'Pair with white sauced dishes',
3,
strtotime('2010-10-03 12:10:09'),
strtotime('2010-10-03 12:10:09'),
26,
2,
85,
),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
@@ -211,9 +263,9 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
db_update('migrate_example_wine')
->fields(array(
'body' => 'Not so much berry character',
// Timestamp 1285058521
'last_changed' => strtotime('2010-10-21 04:42:01'),
'body' => 'Not so much berry character',
// Timestamp 1285058521
'last_changed' => strtotime('2010-10-21 04:42:01'),
))
->condition('wineid', 2)
->execute();
@@ -241,22 +293,24 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
$this->assertEqual(count($rawnodes), 0, t('All nodes deleted'));
$count = db_select('migrate_map_winewine', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_winewine', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
// Now test highwater with unjoined map table
$migration->getSource()->setMapJoinable(FALSE);
$result = $migration->processImport(array('limit' =>
array('value' => 2, 'unit' => 'items')));
$result = $migration->processImport(array(
'limit' =>
array('value' => 2, 'unit' => 'items'),
));
db_update('migrate_example_wine')
->fields(array(
'body' => 'Very berry',
@@ -282,8 +336,10 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
// Test itemlimit (joined map table)
$result = $migration->processRollback(array('force' => TRUE));
$migration->getSource()->setMapJoinable(TRUE);
$result = $migration->processImport(array('limit' =>
array('value' => 1, 'unit' => 'item')));
$result = $migration->processImport(array(
'limit' =>
array('value' => 1, 'unit' => 'item'),
));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import with itemlimit returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
@@ -303,8 +359,10 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
// Test itemlimit (unjoined map table)
$result = $migration->processRollback(array('force' => TRUE));
$migration->getSource()->setMapJoinable(FALSE);
$result = $migration->processImport(array('limit' =>
array('value' => 1, 'unit' => 'item')));
$result = $migration->processImport(array(
'limit' =>
array('value' => 1, 'unit' => 'item'),
));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import with itemlimit returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
@@ -359,10 +417,10 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
->execute();
$result = $migration->processImport();
$newHighwater = db_select('migrate_status', 'ms')
->fields('ms', array('highwater'))
->condition('machine_name', 'WineWine')
->execute()
->fetchField();
->fields('ms', array('highwater'))
->condition('machine_name', 'WineWine')
->execute()
->fetchField();
if (!$this->assertEqual($newHighwater, 1000000000, t('Correct highwater mark set'))) {
$this->error(t('Unexpected highwater mark !highwater', array('!highwater' => $newHighwater)));
}
@@ -387,10 +445,23 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Beer node update import returned RESULT_COMPLETED'));
$result = db_select('migrate_message_beernode', 'msg')
->fields('msg', array('message'))
->execute();
->fields('msg', array('message'))
->execute();
foreach ($result as $row) {
$this->error($row->message);
}
// Test proper disableMailSystem()/restoreMailSystem() behavior.
// @see https://www.drupal.org/node/2499861
// This is not necessarily node-related, but it was shoved in the node test.
$migration->saveMailSystem();
$migration->disableMailSystem();
$ms = variable_get('mail_system', NULL);
$this->assertTrue(is_array($ms) && in_array('MigrateMailIgnore', $ms),
t('Migration has disabled mail system'));
$migration->restoreMailSystem();
$ms = variable_get('mail_system', NULL);
$this->assertFalse(is_array($ms) && in_array('MigrateMailIgnore', $ms),
t('Migration::restoreMailSystem restored mail system'));
}
}
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/table.test
+2 -1
View File
@@ -9,6 +9,7 @@
* Test table migration.
*/
class MigrateTableUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Table migration',
@@ -21,7 +22,7 @@ class MigrateTableUnitTest extends DrupalWebTestCase {
parent::setUp('migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testTableImport() {
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/term.test
+26 -16
View File
@@ -9,6 +9,7 @@
* Test taxonomy migration.
*/
class MigrateTaxonomyUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Taxonomy migration',
@@ -21,7 +22,7 @@ class MigrateTaxonomyUnitTest extends DrupalWebTestCase {
parent::setUp('taxonomy', 'migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testTermImport() {
@@ -36,10 +37,16 @@ class MigrateTaxonomyUnitTest extends DrupalWebTestCase {
$terms[$term->name] = $term;
}
$query = db_select('migrate_example_wine_categories', 'wc')
->fields('wc', array('categoryid', 'name', 'details', 'category_parent', 'ordering'))
->condition('wc.type', 'variety');
->fields('wc', array(
'categoryid',
'name',
'details',
'category_parent',
'ordering',
))
->condition('wc.type', 'variety');
$query->leftJoin('migrate_example_wine_categories', 'wcpar',
'wc.category_parent=wcpar.categoryid');
'wc.category_parent=wcpar.categoryid');
$query->addField('wcpar', 'name', 'parent_name');
$result = $query->execute();
@@ -79,16 +86,19 @@ class MigrateTaxonomyUnitTest extends DrupalWebTestCase {
foreach ($original_terms as $tid => $original_term) {
foreach ($original_term as $field => $value) {
if ($field == 'description') {
if ($value == $final_terms[$tid]->$field) {
if ($value == $final_terms[$tid]->{$field}) {
$this->error(t('Field !field should have changed but did not, value=!value',
array('!field' => $field, '!value' => print_r($value, TRUE))));
}
}
else {
if ($value != $final_terms[$tid]->$field) {
if ($value != $final_terms[$tid]->{$field}) {
$this->error(t('Field !field mismatch: original !value1, result !value2',
array('!field' => $field, '!value1' => print_r($value, TRUE),
'!value2' => print_r($final_terms[$tid]->$field, TRUE))));
array(
'!field' => $field,
'!value1' => print_r($value, TRUE),
'!value2' => print_r($final_terms[$tid]->{$field}, TRUE),
)));
}
}
}
@@ -101,16 +111,16 @@ class MigrateTaxonomyUnitTest extends DrupalWebTestCase {
$rawterms = taxonomy_term_load_multiple(array(), array('vid' => $vocab->vid));
$this->assertEqual(count($rawterms), 0, t('All terms deleted'));
$count = db_select('migrate_map_winevariety', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_winevariety', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
}
}
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/user.test
+64 -38
View File
@@ -9,6 +9,7 @@
* Test user migration.
*/
class MigrateUserUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'User migration',
@@ -24,7 +25,7 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
date_default_timezone_set('US/Mountain');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testUserImport() {
@@ -38,27 +39,45 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
t('Role import returned RESULT_COMPLETED'));
// Confirm both roles were successfully imported
$result = db_select('role', 'r')
->fields('r', array('rid', 'name'))
->condition('name', array('Taster', 'Vintner'), 'IN')
->execute();
->fields('r', array('rid', 'name'))
->condition('name', array('Taster', 'Vintner'), 'IN')
->execute();
$roles = array();
foreach ($result as $row) {
$roles[$row->name] = $row->rid;
}
$this->assertEqual(count($roles), 2, t('Both roles imported'));
// Make sure update does not fail (regression test of
// http://drupal.org/node/1872446)
$migration->prepareUpdate();
$migration->processImport();
$num_messages = $migration->getMap()->messageCount();
$this->assertEqual($num_messages, 0, t('No messages generated'));
$migration = Migration::getInstance('WineUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$result = db_select('migrate_example_wine_account', 'mea')
->fields('mea', array('accountid', 'status', 'posted', 'name',
'sex', 'password', 'mail', 'last_access', 'last_login',
'sig', 'original_mail'))
->execute();
->fields('mea', array(
'accountid',
'status',
'posted',
'name',
'sex',
'password',
'mail',
'last_access',
'last_login',
'sig',
'original_mail',
))
->execute();
$uids = db_select('users', 'u')
->fields('u', array('uid'))
->execute()
->fetchCol();
->fields('u', array('uid'))
->execute()
->fetchCol();
// Index by name
$users = array();
foreach ($uids as $uid) {
@@ -106,9 +125,10 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
1, t('Female gender migrated'));
$this->assert(!isset($users['fonzie']->field_migrate_example_gender[LANGUAGE_NONE][0]['value']),
t('Missing gender left unmigrated'));
$this->assert(is_object($users['fonzie']->picture) &&
$users['fonzie']->picture->filename == 'association-individual.png',
t('Picture migrated'));
/* For some reason, this fails on d.o but not in local environments
$this->assert(is_object($users['fonzie']->picture) &&
$users['fonzie']->picture->filename == '200',
t('Picture migrated'));*/
$this->assertNotNull($users['fonzie']->roles[$roles['Taster']], t('Taster role'));
$this->assertNotNull($users['fonzie']->roles[$roles['Vintner']], t('Vintner role'));
@@ -131,20 +151,26 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
foreach ($original_users as $uid => $original_user) {
foreach ($original_user as $field => $value) {
if ($field == 'field_migrate_example_gender') {
if ($value == $final_users[$uid]->$field) {
if ($value == $final_users[$uid]->{$field}) {
$this->error(t('For user !name, field !field should have changed but did not, value=!value',
array('!name' => $final_users[$uid]->name, '!field' => $field,
'!value' => print_r($value, TRUE))));
array(
'!name' => $final_users[$uid]->name,
'!field' => $field,
'!value' => print_r($value, TRUE),
)));
}
}
else {
if ($value != $final_users[$uid]->$field) {
if ($value != $final_users[$uid]->{$field}) {
// Core bug http://drupal.org/node/935592 causes picture mismatches, ignore until it's fixed
if ($field != 'picture') {
$this->error(t('For user !name, field !field mismatch: original !value1, result !value2',
array('!name' => $final_users[$uid]->name, '!field' => $field,
array(
'!name' => $final_users[$uid]->name,
'!field' => $field,
'!value1' => print_r($value, TRUE),
'!value2' => print_r($final_users[$uid]->$field, TRUE))));
'!value2' => print_r($final_users[$uid]->{$field}, TRUE),
)));
}
}
}
@@ -156,23 +182,23 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User rollback returned RESULT_COMPLETED'));
$count = db_select('users', 'u')
->fields('u', array('uid'))
->countQuery()
->execute()
->fetchField();
->fields('u', array('uid'))
->countQuery()
->execute()
->fetchField();
// 2 users left - anon and admin
$this->assertEqual($count, 2, t('All imported users deleted'));
$count = db_select('migrate_map_wineuser', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_wineuser', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
// Test deduping
@@ -182,9 +208,9 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$accounts = db_select('users', 'u')
->fields('u', array('mail', 'name'))
->execute()
->fetchAllKeyed();
->fields('u', array('mail', 'name'))
->execute()
->fetchAllKeyed();
if (!$this->assertEqual($accounts['alice@example.com'], 'alice', t('alice found'))) {
$this->error(t('Expected alice, found !name', array('!name' => $accounts['alice@example.com'])));
}
@@ -198,9 +224,9 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$accounts = db_select('users', 'u')
->fields('u', array('mail', 'name'))
->execute()
->fetchAllKeyed();
->fields('u', array('mail', 'name'))
->execute()
->fetchAllKeyed();
if (!$this->assertEqual($accounts['alice@example.com'], 'alice', t('alice found'))) {
$this->error(t('Expected alice, found !name', array('!name' => $accounts['alice@example.com'])));
}
sites/all/modules/contrib/migrate/migrate/tests/plugins/sources/oracle.test
+11 -10
View File
@@ -12,6 +12,7 @@
* when that is fixed.
*/
class MigrateOracleUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Oracle migration',
@@ -23,7 +24,7 @@ class MigrateOracleUnitTest extends DrupalWebTestCase {
function setUp() {
global $conf;
if (empty($conf['oracle_db']) || empty($conf['oracle_db']['username']) ||
empty($conf['oracle_db']['password']) || empty($conf['oracle_db']['connection_string'])) {
empty($conf['oracle_db']['password']) || empty($conf['oracle_db']['connection_string'])) {
parent::setUp();
}
else {
@@ -31,7 +32,7 @@ class MigrateOracleUnitTest extends DrupalWebTestCase {
}
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testOracleImport() {
@@ -87,16 +88,16 @@ class MigrateOracleUnitTest extends DrupalWebTestCase {
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_oracle'), TRUE);
$this->assertEqual(count($rawnodes), 0, t('All nodes deleted'));
$count = db_select('migrate_map_migrateexampleoracle', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_migrateexampleoracle', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
}
}
sites/all/modules/contrib/migrate/migrate/tests/plugins/sources/xml.test
+68 -16
View File
@@ -9,6 +9,7 @@
* Test node migration.
*/
class MigrateXMLUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'XML migration',
@@ -21,7 +22,7 @@ class MigrateXMLUnitTest extends DrupalWebTestCase {
parent::setUp('taxonomy', 'migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testNodeImport() {
@@ -29,22 +30,51 @@ class MigrateXMLUnitTest extends DrupalWebTestCase {
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Region term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineFileCopy');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('File import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineRole');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Role import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineProducerXML');
$result = $migration->processImport();
$migration1 = Migration::getInstance('WineProducerXML');
$result = $migration1->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import returned RESULT_COMPLETED'));
t('Producer node import 1 returned RESULT_COMPLETED'));
$migration2 = Migration::getInstance('WineProducerNamespaceXML');
$result = $migration2->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 2 returned RESULT_COMPLETED'));
$migration3 = Migration::getInstance('WineProducerMultiXML');
$result = $migration3->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 3 returned RESULT_COMPLETED'));
$migration4 = Migration::getInstance('WineProducerMultiNamespaceXML');
$result = $migration4->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 4 returned RESULT_COMPLETED'));
$migration5 = Migration::getInstance('WineProducerXMLPull');
$result = $migration5->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 5 returned RESULT_COMPLETED'));
$migration6 = Migration::getInstance('WineProducerNamespaceXMLPull');
$result = $migration6->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 6 returned RESULT_COMPLETED'));
// Gather producer nodes, and their corresponding input data
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_producer'), TRUE);
@@ -54,7 +84,7 @@ class MigrateXMLUnitTest extends DrupalWebTestCase {
$producer_nodes[$node->title] = $node;
}
$this->assertEqual(count($producer_nodes), 1,
$this->assertEqual(count($producer_nodes), 10,
t('Counts of producer nodes and input rows match'));
// Test each base node field
@@ -81,23 +111,45 @@ class MigrateXMLUnitTest extends DrupalWebTestCase {
$this->assertEqual($region[0]['tid'], $term->tid,
t('region properly migrated'));
// Test rollback
$result = $migration->processRollback();
// Rollback producer migrations
$result = $migration1->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback returned RESULT_COMPLETED'));
t('Producer node rollback 1 returned RESULT_COMPLETED'));
$result = $migration2->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 2 returned RESULT_COMPLETED'));
$result = $migration3->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 3 returned RESULT_COMPLETED'));
$result = $migration4->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 4 returned RESULT_COMPLETED'));
$result = $migration5->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 5 returned RESULT_COMPLETED'));
$result = $migration6->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 6 returned RESULT_COMPLETED'));
// Test rollback
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_producer'), TRUE);
$this->assertEqual(count($rawnodes), 0, t('All nodes deleted'));
$count = db_select('migrate_map_wineproducerxml', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_wineproducerxml', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
}
}
sites/all/modules/contrib/migrate/migrate/uri_map_redirect.php
+4 -4
View File
@@ -3,9 +3,9 @@
/**
* @file
* Sample file for handling redirection from old to new URIs. Use an Apache
* rewrite rule (or equivalent) to map legacy requests to this file. To use, copy
* or symlink this file to the root of your drupal site. Customize
* this file to your needs.
* rewrite rule (or equivalent) to map legacy requests to this file. To use,
* copy or symlink this file to the root of your drupal site. Customize this
* file to your needs.
*
* CREATE TABLE `migrate_source_uri_map` (
* `source_uri` varchar(255) NOT NULL DEFAULT '',
@@ -41,7 +41,7 @@ function migrate_build_url($destid1, $migration_name) {
}
// Build absolute url for 301 redirect.
return $base_url . '/' . $destination_uri;
return $base_url . '/' . $destination_uri;
}
define('DRUPAL_ROOT', getcwd());