bachir/materio-base-legacy
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

non security modules update

Browse Source
This commit is contained in:
Bachir Soussi Chiadmi
2015-04-20 16:32:07 +02:00
parent 6a8d30db08
commit 37fbabab56
466 changed files with 32690 additions and 9652 deletions
Download Patch File Download Diff File Expand all files Collapse all files

248
sites/all/modules/contrib/migrate/migrate/CHANGELOG.txt
View File

@@ -1,3 +1,235 @@
Migrate 2.7
===========
Bug fixes
- #2415597 - Make batching of SQL sources optional, and force map_joinable FALSE.
Migrate 2.7 Release Candidate 1
===============================
Features and enhancements
- #2296911 - Add a source handler for IBM DB2.
- #2256761 - Add a destination handler for variables.
- #2047815 - Support multi-column source keys in idlist.
- #1751438 - Add spreadsheet source plugin.
Bug fixes
- #2403593 - SQL batching messes up cases with altered queries, such as idlist.
- #2298969 - Verify wizard validation function exists.
- #2268863 - Fix drush --all option.
- #2410523 - Remove inconsistent escaping of migrate_drush_path.
Migrate 2.6
===========
IMPORTANT CHANGES SINCE MIGRATE 2.5
-----------------------------------
Migration developers will need to add the "advanced migration information"
permission to their roles to continue seeing all the info in the UI they're
used to.
Auto-registration (having classes be registered just based on their class name,
with no call to registerMigration or definition in hook_migrate_api()) is no
longer supported. Registration of classes defined in hook_migrate_api() is no
longer automatic - do a drush migrate-register or use the Register button in the
UI to register them.
Migration class constructors should now always accept an $arguments array as
the first parameter and pass it to its parent. This version does support legacy
migrations which pass a group object, or nothing, but these methods are
deprecated.
Features and enhancements
- #2390229 - Make literal strings monolithic.
- #2391789 - Add extender getter to better support extending wizards.
- #2363015 - Add support for modifying wizards defined by other modules.
- #2353527 - Add getter/setter for trackLastImported.
- #2302929 - Explicitly count IDs for JSON source counts.
- #2296187 - Batch MigrateSQLSource queries.
- #2260211 - Allow skipping of file contents in MigrateListFiles.
- #2259089 - Display actual query for SQL sources.
- #2224297 - Destination handler for custom blocks.
- #2370677 - Add removeStep() method to Wizard API.
- #2306953 - Give basic users a little more information.
- #2261357 - Add prepareCallback/completeCallback to table destination.
- #2306923 - Propagate message statuses to drupal_set_message in batch.
- #2301679 - Display all source key fields in the message view.
- #2312075 - Add --ignore-highwater option to drush imports
- #1961316 - Get data from all XML namespaces
- #1298724 - Add a node revision destination plugin.
- #1890610 - Add a MongoDB source plugin
- #2065295 - Add ability to disable other module's hooks during migration.
- #2051547 - Add ability to revert UI-defined field mappings.
- #1998632 - Extend MigrateItemsXML to handle an array of XML files.
- #1990612 - Add a row status argument to MigrateException, allowing rows to
be cleanly skipped by throwing exceptions.
- #2017835 - Add MigrateFileUriAsIs file class, and make file migrations more
flexible.
- #2004426 - UI support for editing dependencies; enable setting dependencies
through arguments.
- #1826112 - A new API has been added to support external modules in developing
easy-to-use wizard-based UIs.
- #1833380 - Major refactoring of the UI, breaking groups and migrations
(tasks) into separate pages, introducing an advanced permission and
presenting a simplified UI to those with only the basic permission.
- #1860450 - The UI now has the capability of spawning import/rollback operations
in drush, with email notifications of completion. The notification
ability is available when running drush at the command line, which
may be useful for running imports via cron.
- #1996602 - A default field handler is now provided that should handle many if
not most field types without custom handlers.
- #1901980 - Support encrypting particular arguments saved to the database, to
support wizard implementations that may be prompting for secure
database credentials.
- #1896096 - Add ability to define field mappings via arguments at registration
time, and use those to override mappings in code.
- #1961620 - When editing field mappings, allow destination or source fields to
be explicitly set DNM.
- #1996280 - Allow all field/subfields/options to be edited, and indent
subfields/options to make the relationship more clear.
- #1996350 - Allow sourceMigration to be edited.
- #1996736 - Add support for hook_migrate_api_alter().
- #1984568 - Add setters to enable constructing migrations according to a
dependency injection pattern.
- #1835822 - Hash source rows to detect changes.
- #1975180 - Explicitly check dependency existence so missing dependencies aren't
reported as circular.
- #1984534 - Updating/cleanup of examples to reflect current best practices.
- #1972600 - Add getter for the MigrateXMLReader member of MigrateSourceXML.
- #1856694 - Allow disabling of uri encoding.
- #1835142 - Add names-only option to drush ms command.
- #1892296 - Increase time limit for batch UI migrations.
- #1839644 - Groups have been extended to record titles and arguments in the db.
- #1896920 - Remove auto-registration, and make static registration expicit.
- #1928956 - Clean up registration/instantiation chicken-and-egg problems, by
supporting consistent constructor arguments.
- #1550878 - Added UI for deregistering migrations, including one-button
deregistration of orphans.
- #1792894 - Remove empty field values, permitting proper defaulting.
- #1903236 - Allow reset of migrate_migrations() cache.
- #1894344 - Support warn_on_override for bulk unmigrated methods.
- #1886404 - Allow subfields to be mapped before primary fields.
Bug fixes
- #2391891 - idlist with XML source causes exception.
- #2314077 - Don't merge group dependencies into migration dependencies.
- #2392683 - Set vocabulary name (bundle) explicitly for term reference fields.
- #2266395 - Check for valid picture file saving users.
- #2225551 - Add field validation to entity destination plugins.
- #2285966 - Added exception handling around constructors.
- #2370877 - Fix reloading of wizard steps after first.
- #2358767 - Topological sort to help avoid false circular dependency errors.
- #2019193 - Properly handle destination DNM status on field mapping save.
- #2250081 - Fix PostgreSQL failure updating node statistics on comments.
- #2157933 - Fix PostgreSQL error with empty source migrations.
- #2177313 - Make sure reused file entities are saved to pick up field changes.
- #2352971 - Treat empty default values as NULL in field mapping editor.
- #2347205 - Explicitly default the language for terms.
- #2307599 - Notice for non-advanced users on rollback.
- #2305105 - Remove pedantically-deprecated hyphens.
- #2325237 - Handle language arrays for file fields.
- #2258909 - Use migrated languages for file fields.
- #2323605 - Handle single-parent terms properly.
- #2313495 - Handle multilingual path aliases.
- #2261227 - Fixed missing subfields on edit mapping form.
- #2154385 - Add file/line context to source plugin exceptions.
- #2190255 - Remove duplicate warnings on missing XML source.
- #2129609 - Remove warnings on MS SQL counts.
- #2109931 - Pass drush global options to subprocess.
- #2236279 - Prevent ignore_case from causing created terms to be lower-cased.
- #2104149 - Sanity-check decrypted arguments.
- #2047523 - Consistently document destination handler fields() implementations.
- #2244759 - Rollback migrations in proper (reverse) order in UI.
- #2184641 - Make saveHighwater() work with PostgreSQL.
- #2076035 - Properly check key values in saveMessage().
- #2076035 - Validate that term names are not empty.
- #2225809 - Use proper API for registering wizard classes.
- #2105037 - Expose nid/uid destinations when is_new is on.
- #2145213 - Add SQLMap constructor documentation.
- #2213033 - Add batch callback documentation.
- #2095829 - Remove all variables on uninstall.
- #2072721 - Proper title for migrate_example_baseball feature.
- #1999228 - Removed AJAX from field mapping form for performance.
- #2237755 - Fix activeUrl handling in JSON source.
- #2237891 - Use drupal_register_shutdown_function().
- #2141409 - Handle drush spawning when Drupal in subdirectory.
- #2191335 - Remove unused columns from CSV source row.
- #2210533 - Properly pass directory by reference.
- #2141569 - Explicitly create group in baseball example.
- #2159291 - Add message for missing argument to drush migrate-messages.
- #2019193 - Fix unsetting of DNM source fields through the UI.
- #2227061 - Fix to subfields in a multi-value context.
- #2170177 - Properly handle multi-value fields when a subfield is mapped first.
- #2120205 - Fix bug when upgrading lastimported from D6 to D7.
- #2109821 - Handle multi-value subfields in the default field handler.
- #2039649 - Smarter setting of file 'type' field.
- #2106925 - Fix machine_name generation for legacy migrations.
- #2102087 - Don't pass --group to drush subprocess.
- #2099585 - Enable group option on drush migrate-stop.
- #2088255 - Eliminate notices on migrate_map_hash with track_changes.
- #2014849 - Make sure (again) statistics properly default to 0.
- #2042535 - Properly set file_usage for user pictures.
- #2060631 - Fix notices on poll vote import.
- #2049689 - Empty values for node is_new caused SQL errors.
- #2042399 - Node import did not respect revision flag.
- #2034885 - For XML sources, don't override values populated in prepareRow().
- #2041267 - Handle lack of potential dependencies on edit page.
- #2040101 - Breadcrumbs in migration UI missing sections
- #2033947 - Empty migration display names when matching group names.
- #2037265 - Static migrations were not registered on module enable.
- #2021457 - Coding standard improvements for xml.inc.
- #2030559 - Add deprecation messages for arguments().
- #1854382 - Fix handling of terms with leading spaces.
- #2025137 - Ignore --update in the presence of --idlist.
- #2023813 - Apply defaultValue() for empty XML values.
- #2023657 - Prevent duplicate aliases when updating nodes.
- #2021973 - Drush deregistration needs to handle mixed-case machine names.
- #2017443 - Properly hash XML source rows with track_changes on.
- #2020095 - Preserve arguments in legacy constructors.
- #2018841 - Default field handler needs to account for empty column
descriptions.
- #2016173 - Fix fatal error editing MigrationBase migrations.
- #2015327 - Remove broken automatic field mapping feature.
- #2014849 - Make sure statistics properly default to 0.
- #2011024 - Wildcard groups/tasks in the menu hierarchy, saving the need to
rebuild menus when migrations are registered/deregistered.
- #2010884 - Clean up empty default groups on dbupdate.
- #2004296 - Enforce map/message table names with prefixes.
- #2009222 - Properly update map/message tables on a non-default connection.
- #2006158 - Fix warnings on default field handler fields().
- #1999918 - Move permissions to core Migrate module.
- #1999290 - Check that an XMLMigration has actually populated $row->xml.
- #1989012 - Support preserve_files for MigrateFileFid.
- #1982564 - Properly handle messages for ignored rows.
- #1989022 - Strip HTML tags from drush fields output.
- #1988008 - Restore group dependency support.
- #1985750 - Add missing prepareRollback/completeRollback for file destinations.
- #1978702 - Check highwater marks against the starting highwater.
- #1974216 - Change field mapping table PK to a simple serial column.
- #1977578 - Bad exception handling in MigrateDestinationTableCopy.
- #1973030 - Handle zero-valued timestamps properly.
- #1973092 - Fix preservation of sourceMigration when editing mappings.
- #1968358 - Prevent unnecessary reimport when using track_changes.
- #1968014 - Fix menu notices on task list page.
- #1967946 - Missing check on existence of mapping.
- #1849350 - Make sure encoding of uris doesn't break query strings.
- #1844316 - Ensure rollback_action is added to all map tables.
- #1913462 - Fix update functions returning arrays.
- #1831940 - Allow empty string source keys in handleSourceMigration().
- #1896042 - Fix incorrect usage of originalQuery.
- #1954936 - Allow overriding source count in isComplete.
- #1952430 - Prevent bogus "no error provided" messages.
- #1931168 - Properly cache class info in _migrate_class_list().
- #1900914 - Disable email with no custom mail system configured.
- #1901648 - Missing bundle property on file migrations.
- #1885362 - Make access callback explicit, to avoid conflict with admin_views.
- #1880512 - Strip tags from descriptions on drush migrate-mappings.
- #1872446 - Properly handle updates on role migration.
- #1871764 - Pass zero values through handleSourceMigration.
- #1839534 - Handle missing chunk separator in MigrateItemFile.
- #1854382 - Prevent duplicate terms due to leading/trailing spaces.
- #1794236 - Namespace detail pages within menu system.
- #1836426 - Proper check on activeUrl for multiple XML files.
Migrate 2.5
===========
@@ -10,12 +242,6 @@ If your migrations are not explicitly registered, you must request auto-registra
with a "drush mar" (drush migrate-auto-register) command, or by clicking the
"Register" button at admin/content/migrate/registration.
Bug fixes
- #1827052 - Properly check for bad XML.
Migrate 2.5 Release Candidate 2
===============================
Features and enhancements
- #1824024 - Destination and field handlers may now be registered through
hook_migrate_api(). Automatic registration of all migration and
@@ -26,14 +252,6 @@ Features and enhancements
hook_migrate_api().
- #1819730 - Make migration of files in a field context non-fatal.
- #1816652 - Provide useful warning when file subfield arrays don't line up.
Bug fixes
- #1824118 - Make --force work for rollbacks.
Migrate 2.5 Release Candidate 1
===============================
Features and enhancements
- #1778952 - Enable registration of dynamic migrations via hook_migrate_api().
Add explicit auto-registration of non-dynamic migrations, remove
performing registration on cache clear.
@@ -54,6 +272,8 @@ Features and enhancements
- #1621906 - Support DESTINATION system-of-records for menu links.
Bug fixes
- #1827052 - Properly check for bad XML.
- #1824118 - Make --force work for rollbacks.
- #1659150 - Change 'ok' message types to 'status'.
- #1690080 - Deal with self-references in handleSourceMigration().
- #1797644 - Remove bogus assignment on term update.

4
sites/all/modules/contrib/migrate/migrate/README.txt
View File

@@ -6,8 +6,8 @@ users are included. Plugins permit migration of other types of content.
Usage
-----
Documentation is at http://drupal.org/node/415260. To get started, enable the
migrate_example module and browse to admin/content/migrate to see its dashboard.
Documentation is at http://drupal.org/migrate. To get started, enable the
migrate_example module and browse to admin/content/migrate to see its dashboard.
The code for this migration is in migrate_example/beer.inc (advanced examples are
in wine.inc). Mimic that file in order to specify your own migrations.

423
sites/all/modules/contrib/migrate/migrate/includes/base.inc
View File

@@ -37,9 +37,9 @@ abstract class MigrationBase {
}
/**
* The name of a migration group, used to collect related migrations.
* A migration group object, used to collect related migrations.
*
* @var string
* @var MigrateGroup
*/
protected $group;
public function getGroup() {
@@ -55,6 +55,9 @@ abstract class MigrationBase {
public function getDescription() {
return $this->description;
}
public function setDescription($description) {
$this->description = $description;
}
/**
* Save options passed to current operation
@@ -143,9 +146,21 @@ abstract class MigrationBase {
public function getHardDependencies() {
return $this->dependencies;
}
public function setHardDependencies(array $dependencies) {
$this->dependencies = $dependencies;
}
public function addHardDependencies(array $dependencies) {
$this->dependencies = array_merge($this->dependencies, $dependencies);
}
public function getSoftDependencies() {
return $this->softDependencies;
}
public function setSoftDependencies(array $dependencies) {
$this->softDependencies = $dependencies;
}
public function addSoftDependencies(array $dependencies) {
$this->softDependencies = array_merge($this->softDependencies, $dependencies);
}
public function getDependencies() {
return array_merge($this->dependencies, $this->softDependencies);
}
@@ -161,6 +176,13 @@ abstract class MigrationBase {
self::$displayFunction = $display_function;
}
/**
* Track whether or not we've already displayed an encryption warning
*
* @var bool
*/
protected static $showEncryptionWarning = TRUE;
/**
* The fraction of the memory limit at which an operation will be interrupted.
* Can be overridden by a Migration subclass if one would like to push the
@@ -193,6 +215,14 @@ abstract class MigrationBase {
*/
protected $timeLimit;
/**
* A time limit in seconds appropriate to be used in a batch
* import. Defaults to 240.
*
* @var int
*/
protected $batchTimeLimit = 240;
/**
* MigrateTeamMember objects representing people involved with this
* migration.
@@ -203,6 +233,9 @@ abstract class MigrationBase {
public function getTeam() {
return $this->team;
}
public function setTeam(array $team) {
$this->team = $team;
}
/**
* If provided, an URL for an issue tracking system containing :id where
@@ -214,6 +247,9 @@ abstract class MigrationBase {
public function getIssuePattern() {
return $this->issuePattern;
}
public function setIssuePattern($issue_pattern) {
$this->issuePattern = $issue_pattern;
}
/**
* If we set an error handler (during import), remember the previous one so
@@ -223,6 +259,22 @@ abstract class MigrationBase {
*/
protected $previousErrorHandler = NULL;
/**
* Arguments configuring a migration.
*
* @var array
*/
protected $arguments = array();
public function getArguments() {
return $this->arguments;
}
public function setArguments(array $arguments) {
$this->arguments = $arguments;
}
public function addArguments(array $arguments) {
$this->arguments = array_merge($this->arguments, $arguments);
}
/**
* Disabling a migration prevents it from running with --all, or individually
* without --force
@@ -233,6 +285,29 @@ abstract class MigrationBase {
public function getEnabled() {
return $this->enabled;
}
public function setEnabled($enabled) {
$this->enabled = $enabled;
}
/**
* Any module hooks which should be disabled during migration processes.
*
* @var array
* Key: Hook name (e.g., 'node_insert')
* Value: Array of modules for which to disable this hook (e.g., array('pathauto')).
*/
protected $disableHooks = array();
public function getDisableHooks() {
return $this->disableHooks;
}
/**
* Have we already warned about obsolete constructor argumentss on this request?
*
* @var bool
*/
static protected $groupArgumentWarning = FALSE;
static protected $emptyArgumentsWarning = FALSE;
/**
* Codes representing the result of a rollback or import process.
@@ -284,17 +359,51 @@ abstract class MigrationBase {
}
/**
* General initialization of a MigrationBase object.
* Construction of a MigrationBase instance.
*
* @param array $arguments
*/
public function __construct($group = NULL) {
$this->machineName = $this->generateMachineName();
if (empty($group)) {
$this->group = MigrateGroup::getInstance('default');
public function __construct($arguments = array()) {
// Support for legacy code passing a group object as the first parameter.
if (is_object($arguments) && is_a($arguments, 'MigrateGroup')) {
$this->group = $arguments;
$this->arguments['group_name'] = $arguments->getName();
if (!self::$groupArgumentWarning &&
variable_get('migrate_deprecation_warnings', 1)) {
self::displayMessage(t('Passing a group object to a migration constructor is now deprecated - pass through the arguments array passed to the leaf class instead.'));
self::$groupArgumentWarning = TRUE;
}
}
else {
$this->group = $group;
if (empty($arguments)) {
$this->arguments = array();
if (!self::$emptyArgumentsWarning &&
variable_get('migrate_deprecation_warnings', 1)) {
self::displayMessage(t('Passing an empty first parameter to a migration constructor is now deprecated - pass through the arguments array passed to the leaf class instead.'));
self::$emptyArgumentsWarning = TRUE;
}
}
else {
$this->arguments = $arguments;
}
if (empty($this->arguments['group_name'])) {
$this->arguments['group_name'] = 'default';
}
$this->group = MigrateGroup::getInstance($this->arguments['group_name']);
}
if (isset($this->arguments['machine_name'])) {
$this->machineName = $this->arguments['machine_name'];
}
else {
// Deprecated - this supports old code which does not pass the arguments
// array through to the base constructor. Remove in the next version.
$this->machineName = $this->machineFromClass(get_class($this));
}
// Make any group arguments directly accessible to the specific migration,
// other than group dependencies.
$group_arguments = $this->group->getArguments();
unset($group_arguments['dependencies']);
$this->arguments += $group_arguments;
// Record the memory limit in bytes
$limit = trim(ini_get('memory_limit'));
@@ -330,9 +439,18 @@ abstract class MigrationBase {
$conf['mail_system'][$system] = 'MigrateMailIgnore';
}
}
else {
$conf['mail_system']['default-system'] = 'MigrateMailIgnore';
}
// Make sure we clear our semaphores in case of abrupt exit
register_shutdown_function(array($this, 'endProcess'));
drupal_register_shutdown_function(array($this, 'endProcess'));
// Save any hook disablement information.
if (isset($this->arguments['disable_hooks']) &&
is_array($this->arguments['disable_hooks'])) {
$this->disableHooks = $this->arguments['disable_hooks'];
}
}
/**
@@ -358,7 +476,10 @@ abstract class MigrationBase {
* @param string $machine_name
* @param array $arguments
*/
static public function registerMigration($class_name, $machine_name = NULL, array $arguments = array()) {
static public function registerMigration($class_name, $machine_name = NULL,
array $arguments = array()) {
// Support for legacy migration code - in later releases, the machine_name
// should always be explicit.
if (!$machine_name) {
$machine_name = self::machineFromClass($class_name);
}
@@ -368,18 +489,28 @@ abstract class MigrationBase {
array('!name' => $machine_name)));
}
// Making sure the machine name is in the arguments array helps with
// chicken-and-egg problems in determining the machine name.
if (!isset($arguments['machine_name'])) {
$arguments['machine_name'] = $machine_name;
// We no longer have any need to store the machine_name in the arguments.
if (isset($arguments['machine_name'])) {
unset($arguments['machine_name']);
}
if (isset($arguments['group_name'])) {
$group_name = $arguments['group_name'];
unset($arguments['group_name']);
}
else {
$group_name = 'default';
}
$arguments = self::encryptArguments($arguments);
// Register the migration if it's not already there; if it is,
// update the class and arguments in case they've changed.
db_merge('migrate_status')
->key(array('machine_name' => $machine_name))
->fields(array(
'class_name' => $class_name,
'group_name' => $group_name,
'arguments' => serialize($arguments)
))
->execute();
@@ -395,17 +526,27 @@ abstract class MigrationBase {
$rows_deleted = db_delete('migrate_status')
->condition('machine_name', $machine_name)
->execute();
// Make sure the group gets deleted if we were the only member.
MigrateGroup::deleteOrphans();
}
/**
* By default, the migration machine name is the class name (with the
* Migration suffix, if present, stripped).
* The migration machine name is stored in the arguments.
*
* @return string
*/
protected function generateMachineName() {
$class_name = get_class($this);
return self::machineFromClass($class_name);
return $this->arguments['machine_name'];
}
/**
* Given only a class name, derive a machine name (the class name with the
* "Migration" suffix, if any, removed).
*
* @param $class_name
*
* @return string
*/
protected static function machineFromClass($class_name) {
if (preg_match('/Migration$/', $class_name)) {
$machine_name = drupal_substr($class_name, 0,
@@ -422,38 +563,74 @@ abstract class MigrationBase {
*
* @param string $machine_name
*/
/**
* Return the single instance of the given migration.
*
* @param $machine_name
* The unique machine name of the migration to retrieve.
* @param string $class_name
* Deprecated - no longer used, class name is retrieved from migrate_status.
* @param array $arguments
* Deprecated - no longer used, arguments are retrieved from migrate_status.
*
* @return MigrationBase
*/
static public function getInstance($machine_name, $class_name = NULL, array $arguments = array()) {
$migrations = &drupal_static(__FUNCTION__, array());
// Otherwise might miss cache hit on case difference
$machine_name_key = drupal_strtolower($machine_name);
if (!isset($migrations[$machine_name_key])) {
// Skip the query if our caller already made it
if (!$class_name) {
// See if we know about this migration
$row = db_select('migrate_status', 'ms')
->fields('ms', array('class_name', 'arguments'))
->condition('machine_name', $machine_name)
->execute()
->fetchObject();
if ($row) {
$class_name = $row->class_name;
$arguments = unserialize($row->arguments);
// See if we know about this migration
$row = db_select('migrate_status', 'ms')
->fields('ms', array('class_name', 'group_name', 'arguments'))
->condition('machine_name', $machine_name)
->execute()
->fetchObject();
if ($row) {
$class_name = $row->class_name;
$arguments = unserialize($row->arguments);
$arguments = self::decryptArguments($arguments);
$arguments['group_name'] = $row->group_name;
}
else {
// Can't find a migration with this name
self::displayMessage(t('No migration found with machine name !machine',
array('!machine' => $machine_name)));
return NULL;
}
$arguments['machine_name'] = $machine_name;
if (class_exists($class_name)) {
try {
$migrations[$machine_name_key] = new $class_name($arguments);
}
else {
// Can't find a migration with this name
throw new MigrateException(t('No migration found with machine name !machine',
catch (Exception $e) {
self::displayMessage(t('Migration !machine could not be constructed.',
array('!machine' => $machine_name)));
self::displayMessage($e->getMessage());
return NULL;
}
}
$migrations[$machine_name_key] = new $class_name($arguments);
else {
self::displayMessage(t('No migration class !class found',
array('!class' => $class_name)));
return NULL;
}
if (isset($arguments['dependencies'])) {
$migrations[$machine_name_key]->setHardDependencies(
$arguments['dependencies']);
}
if (isset($arguments['soft_dependencies'])) {
$migrations[$machine_name_key]->setSoftDependencies(
$arguments['soft_dependencies']);
}
}
return $migrations[$machine_name_key];
}
/**
* Identifies whether this migration is "dynamic" (that is, allows multiple
* instances distinguished by differing parameters). A dynamic class should
* override this with a return value of TRUE.
* @deprecated - No longer a useful distinction between "status" and "dynamic"
* migrations.
*/
static public function isDynamic() {
return FALSE;
@@ -625,9 +802,16 @@ abstract class MigrationBase {
if (!empty($this->highwaterField['type']) && $this->highwaterField['type'] == 'int') {
// If the highwater is an integer type, we need to force the DB server
// to treat the varchar highwater field as an integer (otherwise it will
// think '5' > '10'). CAST(highwater AS INTEGER) would be ideal, but won't
// work in MySQL. This hack is thought to be portable.
$query->where('(highwater+0) < :highwater', array(':highwater' => $highwater));
// think '5' > '10').
switch (Database::getConnection()->databaseType()) {
case 'pgsql':
$query->where('(CASE WHEN highwater=\'\' THEN 0 ELSE CAST(highwater AS INTEGER) END) < :highwater', array(':highwater' => intval($highwater)));
break;
default:
// CAST(highwater AS INTEGER) would be ideal, but won't
// work in MySQL. This hack is thought to be portable.
$query->where('(highwater+0) < :highwater', array(':highwater' => $highwater));
}
}
else {
$query->condition('highwater', $highwater, '<');
@@ -684,7 +868,7 @@ abstract class MigrationBase {
else {
foreach ($this->dependencies as $dependency) {
$migration = MigrationBase::getInstance($dependency);
if (!$migration->isComplete()) {
if (!$migration || !$migration->isComplete()) {
return FALSE;
}
}
@@ -699,7 +883,7 @@ abstract class MigrationBase {
$incomplete = array();
foreach ($this->getDependencies() as $dependency) {
$migration = MigrationBase::getInstance($dependency);
if (!$migration->isComplete()) {
if (!$migration || !$migration->isComplete()) {
$incomplete[] = $dependency;
}
}
@@ -752,6 +936,17 @@ abstract class MigrationBase {
))
->execute();
}
// If we're disabling any hooks, reset the static module_implements cache so
// it is rebuilt with the specified hooks removed by our
// hook_module_implements_alter(). By setting #write_cache to FALSE, we
// ensure that our munged version of the hooks array does not get written
// to the persistent cache and interfere with other Drupal processes.
if (!empty($this->disableHooks)) {
$implementations = &drupal_static('module_implements');
$implementations = array();
$implementations['#write_cache'] = FALSE;
}
}
/**
@@ -904,6 +1099,14 @@ abstract class MigrationBase {
return $return;
}
/**
* Set the PHP time limit. This method may be called from batch callbacks
* before calling the processImport method.
*/
public function setBatchTimeLimit() {
drupal_set_time_limit($this->batchTimeLimit);
}
/**
* A derived migration class does the actual rollback or import work in these
* methods - we cannot declare them abstract because some classes may define
@@ -999,6 +1202,132 @@ abstract class MigrationBase {
}
}
/**
* Encrypt an incoming value. Detects for existence of the Drupal 'Encrypt'
* module or the mcrypt PHP extension.
*
* @param string $value
* @return string The encrypted value.
*/
static public function encrypt($value) {
if (module_exists('encrypt')) {
$value = encrypt($value);
}
else if (extension_loaded('mcrypt')) {
// Mimic encrypt module to ensure compatibility
$key = drupal_substr(variable_get('drupal_private_key', 'no_key'), 0, 32);
$iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_256, MCRYPT_MODE_ECB);
$iv = mcrypt_create_iv($iv_size, MCRYPT_RAND);
$value = mcrypt_encrypt(MCRYPT_RIJNDAEL_256, $key, $value,
MCRYPT_MODE_ECB, $iv);
$encryption_array['text'] = $value;
// For forward compatibility with the encrypt module.
$encryption_array['method'] = 'mcrypt_rij_256';
$encryption_array['key_name'] = 'drupal_private_key';
$value = serialize($encryption_array);
}
else {
if (self::$showEncryptionWarning) {
MigrationBase::displayMessage(t('Encryption of secure migration information is not supported. Ensure the <a href="@encrypt">Encrypt module</a> or <a href="mcrypt">mcrypt PHP extension</a> is installed for this functionality.',
array(
'@encrypt' => 'http://drupal.org/project/encrypt',
'@mcrypt' => 'http://php.net/manual/en/book.mcrypt.php',
)
),
'warning');
self::$showEncryptionWarning = FALSE;
}
}
return $value;
}
/**
* Decrypt an incoming value.
*
* @param string $value
* @return string The encrypted value
*/
static public function decrypt($value) {
if (module_exists('encrypt')) {
$value = decrypt($value);
}
else if (extension_loaded('mcrypt')) {
// Mimic encrypt module to ensure compatibility
$encryption_array = unserialize($value);
$method = $encryption_array['method']; // Not used right now
$text = $encryption_array['text'];
$key_name = $encryption_array['key_name']; // Not used right now
$iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_256, MCRYPT_MODE_ECB);
$iv = mcrypt_create_iv($iv_size, MCRYPT_RAND);
$key = drupal_substr(variable_get('drupal_private_key', 'no_key'), 0, 32);
$value = mcrypt_decrypt(MCRYPT_RIJNDAEL_256, $key, $text,
MCRYPT_MODE_ECB, $iv);
}
else {
if (self::$showEncryptionWarning) {
MigrationBase::displayMessage(t('Encryption of secure migration information is not supported. Ensure the <a href="@encrypt">Encrypt module</a> or <a href="mcrypt">mcrypt PHP extension</a> is installed for this functionality.',
array(
'@encrypt' => 'http://drupal.org/project/encrypt',
'@mcrypt' => 'http://php.net/manual/en/book.mcrypt.php',
)
),
'warning');
self::$showEncryptionWarning = FALSE;
}
}
return $value;
}
/**
* Make sure any arguments we want to be encrypted get encrypted.
*
* @param array $arguments
*
* @return array
*/
static public function encryptArguments(array $arguments) {
if (isset($arguments['encrypted_arguments'])) {
foreach ($arguments['encrypted_arguments'] as $argument_name) {
if (isset($arguments[$argument_name])) {
$arguments[$argument_name] = self::encrypt(
serialize($arguments[$argument_name]));
}
}
}
return $arguments;
}
/**
* Make sure any arguments we want to be decrypted get decrypted.
*
* @param array $arguments
*
* @return array
*/
static public function decryptArguments(array $arguments) {
if (isset($arguments['encrypted_arguments'])) {
foreach ($arguments['encrypted_arguments'] as $argument_name) {
if (isset($arguments[$argument_name])) {
$decrypted_string = self::decrypt($arguments[$argument_name]);
// A decryption failure will return FALSE and issue a notice. We need
// to distinguish a failure from a serialized FALSE.
$unserialized_value = @unserialize($decrypted_string);
if ($unserialized_value === FALSE && $decrypted_string != serialize(FALSE)) {
self::displayMessage(t('Failed to decrypt argument %argument_name',
array('%argument_name' => $argument_name)));
unset($arguments[$argument_name]);
}
else {
$arguments[$argument_name] = $unserialized_value;
}
}
}
}
return $arguments;
}
/**
* Convert an incoming string (which may be a UNIX timestamp, or an arbitrarily-formatted
* date/time string) to a UNIX timestamp.
@@ -1006,16 +1335,16 @@ abstract class MigrationBase {
* @param string $value
*/
static public function timestamp($value) {
// Default empty values to now
if (empty($value)) {
return time();
}
// Does it look like it's already a timestamp? Just return it
if (is_numeric($value)) {
return $value;
}
// Default empty values to now
if (empty($value)) {
return time();
}
$date = new DateTime($value);
$time = $date->format('U');
if ($time == FALSE) {

17
sites/all/modules/contrib/migrate/migrate/includes/destination.inc
View File

@@ -105,11 +105,22 @@ abstract class MigrateDestination {
* All destination handlers should be derived from MigrateDestinationHandler
*/
abstract class MigrateDestinationHandler extends MigrateHandler {
// abstract function arguments(...)
// Any one or more of these methods may be implemented
/**
* Any one or more of these methods may be implemented
* Documentation of any fields added to the destination by this handler.
*
* @param $entity_type
* The entity type (node, user, etc.) for which to list fields.
* @param $bundle
* The bundle (article, blog, etc.), if any, for which to list fields.
* @param Migration $migration
* Optionally, the migration providing the context.
* @return array
* An array keyed by field name, with field descriptions as values.
*/
//abstract public function fields();
//abstract public function fields($entity_type, $bundle, $migration = NULL);
//abstract public function prepare($entity, stdClass $row);
//abstract public function complete($entity, stdClass $row);
}

19
sites/all/modules/contrib/migrate/migrate/includes/exception.inc
View File

@@ -6,13 +6,30 @@
*/
class MigrateException extends Exception {
/**
* The level of the error being reported (a Migration::MESSAGE_* constant)
* @var int
*/
protected $level;
public function getLevel() {
return $this->level;
}
public function __construct($message, $level = Migration::MESSAGE_ERROR) {
/**
* The status to record in the map table for the current item (a
* MigrateMap::STATUS_* constant)
*
* @var int
*/
protected $status;
public function getStatus() {
return $this->status;
}
public function __construct($message, $level = Migration::MESSAGE_ERROR,
$status = MigrateMap::STATUS_FAILED) {
$this->level = $level;
$this->status = $status;
parent::__construct($message);
}
}

17
sites/all/modules/contrib/migrate/migrate/includes/field_mapping.inc
View File

@@ -29,6 +29,19 @@ class MigrateFieldMapping {
return $this->sourceField;
}
/**
* @var int
*/
const MAPPING_SOURCE_CODE = 1;
const MAPPING_SOURCE_DB = 2;
protected $mappingSource = self::MAPPING_SOURCE_CODE;
public function getMappingSource() {
return $this->mappingSource;
}
public function setMappingSource($mapping_source) {
$this->mappingSource = $mapping_source;
}
/**
* Default value for simple mappings, when there is no source mapping or the
* source field is empty. If both this and the sourceField are omitted, the
@@ -96,6 +109,7 @@ class MigrateFieldMapping {
* 'source_field' - Name of the source field in the incoming row containing the
* value to be assigned
* 'default_value' - A constant value to be assigned in the absence of source_field
* Deprecated - subfield notation is now preferred.
*
* @var array
*/
@@ -173,6 +187,9 @@ class MigrateFieldMapping {
}
public function arguments($arguments) {
if (variable_get('migrate_deprecation_warnings', 1)) {
MigrationBase::displayMessage(t('The field mapping arguments() method is now deprecated - please use subfield notation instead.'));
}
$this->arguments = $arguments;
return $this;
}

145
sites/all/modules/contrib/migrate/migrate/includes/group.inc
View File

@@ -7,7 +7,7 @@
class MigrateGroup {
/**
* The name of the group - used to identify it in drush commands.
* The machine name of the group - used to identify it in drush commands.
*
* @var string
*/
@@ -16,6 +16,27 @@ class MigrateGroup {
return $this->name;
}
/**
* The user-visible title of the group.
*
* @var string
*/
protected $title;
public function getTitle() {
return $this->title;
}
/**
* Domain-specific arguments for the group (to be applied to all migrations
* in the group).
*
* @var array
*/
protected $arguments = array();
public function getArguments() {
return $this->arguments;
}
/**
* List of groups this group is dependent on.
*
@@ -35,9 +56,11 @@ class MigrateGroup {
static public function groups() {
$groups = array();
$dependent_groups = array();
$dependencies_list = array();
$required_groups = array();
foreach (self::$groupList as $name => $group) {
$dependencies = $group->getDependencies();
$dependencies_list[$name] = $dependencies;
if (count($dependencies) > 0) {
// Set groups with dependencies aside for reordering
$dependent_groups[$name] = $group;
@@ -48,29 +71,11 @@ class MigrateGroup {
$groups[$name] = $group;
}
}
$iterations = 0;
while (count($dependent_groups) > 0) {
if ($iterations++ > 20) {
$group_names = implode(',', array_keys($dependent_groups));
throw new MigrateException(t('Failure to sort migration list - most likely due ' .
'to circular dependencies involving groups !group_names',
array('!group_names' => $group_names)));
}
foreach ($dependent_groups as $name => $group) {
$ready = TRUE;
// Scan all the dependencies for this group and make sure they're all
// in the final list
foreach ($group->getDependencies() as $dependency) {
if (!isset($groups[$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this group to the final list
$groups[$name] = $group;
unset($dependent_groups[$name]);
}
$ordered_groups = migrate_order_dependencies($dependencies_list);
foreach ($ordered_groups as $name) {
if (!isset($groups[$name])) {
$groups[$name] = $dependent_groups[$name];
}
}
@@ -86,8 +91,10 @@ class MigrateGroup {
* @param array $dependencies
* List of dependent groups.
*/
public function __construct($name, $dependencies = array()) {
public function __construct($name, $dependencies = array(), $title = '', $arguments = array()) {
$this->name = $name;
$this->title = $title;
$this->arguments = $arguments;
$this->dependencies = $dependencies;
}
@@ -102,8 +109,94 @@ class MigrateGroup {
*/
static public function getInstance($name, $dependencies = array()) {
if (empty(self::$groupList[$name])) {
self::$groupList[$name] = new MigrateGroup($name, $dependencies);
$row = db_select('migrate_group', 'mg')
->fields('mg')
->condition('name', $name)
->execute()
->fetchObject();
if ($row) {
$arguments = unserialize($row->arguments);
$arguments = MigrationBase::decryptArguments($arguments);
if (empty($dependencies) && isset($arguments['dependencies'])) {
$dependencies = $arguments['dependencies'];
}
self::$groupList[$name] = new MigrateGroup($name, $dependencies,
$row->title, $arguments);
}
else {
self::register($name);
self::$groupList[$name] = new MigrateGroup($name, $dependencies);
}
}
return self::$groupList[$name];
}
/**
* Register a new migration group in the migrate_group table.
*
* @param string $name
* The machine name (unique identifier) for the group.
*
* @param string $title
* A user-visible title for the group. Defaults to the machine name.
*
* @param array $arguments
* An array of group arguments - generally data that applies to all migrations
* in the group.
*/
static public function register($name, $title = NULL, array $arguments = array()) {
if (!$title) {
$title = $name;
}
$arguments = MigrationBase::encryptArguments($arguments);
// Register the migration if it's not already there; if it is,
// update the class and arguments in case they've changed.
db_merge('migrate_group')
->key(array('name' => $name))
->fields(array(
'title' => $title,
'arguments' => serialize($arguments)
))
->execute();
}
/**
* Deregister a migration group - remove it from the database, and also
* remove migrations attached to it.
*
* @param string $name
* (Machine) name of the group to deregister.
*/
static public function deregister($name) {
$result = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name'))
->condition('group_name', $name)
->execute();
foreach ($result as $row) {
Migration::deregisterMigration($row->machine_name);
}
db_delete('migrate_group')
->condition('name', $name)
->execute();
}
/**
* Remove any groups which no longer contain any migrations.
*/
static public function deleteOrphans() {
$query = db_select('migrate_group', 'mg');
$query->addField('mg', 'name');
$query->leftJoin('migrate_status', 'ms', 'mg.name=ms.group_name');
$query->isNull('ms.machine_name');
$result = $query->execute();
foreach ($result as $row) {
db_delete('migrate_group')
->condition('name', $row->name)
->execute();
}
}
}

19
sites/all/modules/contrib/migrate/migrate/includes/map.inc
View File

@@ -43,12 +43,27 @@ abstract class MigrateMap implements Iterator {
*/
protected $sourceKeyMap, $destinationKeyMap;
/**
* Get the source key map.
*/
public function getSourceKeyMap() {
return $this->sourceKeyMap;
}
/**
* Boolean determining whether to track last_imported times in map tables
*
* @var boolean
*/
protected $trackLastImported = FALSE;
public function getTrackLastImported() {
return $this->trackLastImported;
}
public function setTrackLastImported($trackLastImported) {
if (is_bool($trackLastImported)) {
$this->trackLastImported = $trackLastImported;
}
}
/**
* Save a mapping from the key values in the source row to the destination
@@ -58,9 +73,11 @@ abstract class MigrateMap implements Iterator {
* @param $dest_ids
* @param $status
* @param $rollback_action
* @param $hash
*/
abstract public function saveIDMapping(stdClass $source_row, array $dest_ids,
$status = MigrateMap::STATUS_IMPORTED, $rollback_action = MigrateMap::ROLLBACK_DELETE);
$status = MigrateMap::STATUS_IMPORTED,
$rollback_action = MigrateMap::ROLLBACK_DELETE, $hash = NULL);
/**
* Record a message related to a source record

248
sites/all/modules/contrib/migrate/migrate/includes/migration.inc
View File

@@ -22,6 +22,9 @@ abstract class Migration extends MigrationBase {
public function getSource() {
return $this->source;
}
public function setSource(MigrateSource $source) {
$this->source = $source;
}
/**
* Destination object for the migration, derived from MigrateDestination.
@@ -32,6 +35,9 @@ abstract class Migration extends MigrationBase {
public function getDestination() {
return $this->destination;
}
public function setDestination(MigrateDestination $destination) {
$this->destination = $destination;
}
/**
* Map object tracking relationships between source and destination data
@@ -42,6 +48,9 @@ abstract class Migration extends MigrationBase {
public function getMap() {
return $this->map;
}
public function setMap(MigrateMap $map) {
$this->map = $map;
}
/**
* Indicate whether the primary system of record for this migration is the
@@ -59,6 +68,9 @@ abstract class Migration extends MigrationBase {
public function getSystemOfRecord() {
return $this->systemOfRecord;
}
public function setSystemOfRecord($system_of_record) {
$this->systemOfRecord = $system_of_record;
}
/**
* Specify value of needs_update for current map row. Usually set by
@@ -75,6 +87,12 @@ abstract class Migration extends MigrationBase {
* @var int
*/
protected $defaultRollbackAction = MigrateMap::ROLLBACK_DELETE;
public function getDefaultRollbackAction() {
return $this->defaultRollbackAction;
}
public function setDefaultRollbackAction($rollback_action) {
$this->defaultRollbackAction = $rollback_action;
}
/**
* The rollback action to be saved for the current row.
@@ -84,13 +102,62 @@ abstract class Migration extends MigrationBase {
public $rollbackAction;
/**
* Simple mappings between destination fields (keys) and source fields (values).
* Field mappings defined in code.
*
* @var array
*/
protected $fieldMappings = array();
protected $storedFieldMappings = array();
protected $storedFieldMappingsRetrieved = FALSE;
public function getStoredFieldMappings() {
if (!$this->storedFieldMappingsRetrieved) {
$this->loadFieldMappings();
$this->storedFieldMappingsRetrieved = TRUE;
}
return $this->storedFieldMappings;
}
/**
* Field mappings retrieved from storage.
*
* @var array
*/
protected $codedFieldMappings = array();
public function getCodedFieldMappings() {
return $this->codedFieldMappings;
}
/**
* All field mappings, with those retrieved from the database overriding those
* defined in code.
*
* @var array
*/
protected $allFieldMappings = array();
public function getFieldMappings() {
return $this->fieldMappings;
if (empty($allFieldMappings)) {
$this->allFieldMappings = array_merge($this->getCodedFieldMappings(),
$this->getStoredFieldMappings());
// If there are multiple mappings of a given source field to no
// destination field, keep only the last (so the UI can override a source
// field DNM that was defined in code).
$no_destination = array();
foreach ($this->allFieldMappings as $destination_field => $mapping) {
// If the source field is not mapped to a destination field, the
// array index is integer.
if (is_int($destination_field)) {
$source_field = $mapping->getSourceField();
if (isset($no_destination[$source_field])) {
unset($this->allFieldMappings[$no_destination[$source_field]]);
unset($no_destination[$source_field]);
}
$no_destination[$source_field] = $destination_field;
}
}
// Make sure primary fields come before their subfields
ksort($this->allFieldMappings);
}
return $this->allFieldMappings;
}
/**
@@ -119,6 +186,9 @@ abstract class Migration extends MigrationBase {
public function getHighwaterField() {
return $this->highwaterField;
}
public function setHighwaterField(array $highwater_field) {
$this->highwaterField = $highwater_field;
}
/**
* The object currently being constructed
@@ -143,8 +213,29 @@ abstract class Migration extends MigrationBase {
/**
* General initialization of a Migration object.
*/
public function __construct($group = NULL) {
parent::__construct($group);
public function __construct($arguments = array()) {
parent::__construct($arguments);
}
/**
* Register a new migration process in the migrate_status table. This will
* generally be used in two contexts - by the class detection code for
* static (one instance per class) migrations, and by the module implementing
* dynamic (parameterized class) migrations.
*
* @param string $class_name
* @param string $machine_name
* @param array $arguments
*/
static public function registerMigration($class_name, $machine_name = NULL,
array $arguments = array()) {
// Record any field mappings provided via arguments.
if (isset($arguments['field_mappings'])) {
self::saveFieldMappings($machine_name, $arguments['field_mappings']);
unset($arguments['field_mappings']);
}
parent::registerMigration($class_name, $machine_name, $arguments);
}
/**
@@ -161,9 +252,16 @@ abstract class Migration extends MigrationBase {
try {
// Remove map and message tables
$migration = self::getInstance($machine_name);
$migration->map->destroy();
if ($migration && method_exists($migration, 'getMap')) {
$migration->getMap()->destroy();
}
// TODO: Clear log entries? Or keep for historical purposes?
// @todo: Clear log entries? Or keep for historical purposes?
// Remove stored field mappings for this migration
$rows_deleted = db_delete('migrate_field_mapping')
->condition('machine_name', $machine_name)
->execute();
// Call the parent deregistration (which clears migrate_status) last, the
// above will reference it.
@@ -174,6 +272,51 @@ abstract class Migration extends MigrationBase {
}
}
/**
* Record an array of field mappings to the database.
*
* @param $machine_name
* @param array $field_mappings
*/
static public function saveFieldMappings($machine_name, array $field_mappings) {
// Clear existing field mappings
db_delete('migrate_field_mapping')
->condition('machine_name', $machine_name)
->execute();
foreach ($field_mappings as $field_mapping) {
$destination_field = $field_mapping->getDestinationField();
$source_field = $field_mapping->getSourceField();
db_insert('migrate_field_mapping')
->fields(array(
'machine_name' => $machine_name,
'destination_field' => is_null($destination_field) ? '' : $destination_field,
'source_field' => is_null($source_field) ? '' : $source_field,
'options' => serialize($field_mapping)
))
->execute();
}
}
/**
* Load any stored field mappings from the database.
*/
public function loadFieldMappings() {
$result = db_select('migrate_field_mapping', 'mfm')
->fields('mfm', array('destination_field', 'source_field', 'options'))
->condition('machine_name', $this->machineName)
->execute();
foreach ($result as $row) {
$field_mapping = unserialize($row->options);
$field_mapping->setMappingSource(MigrateFieldMapping::MAPPING_SOURCE_DB);
if (empty($row->destination_field)) {
$this->storedFieldMappings[] = $field_mapping;
}
else {
$this->storedFieldMappings[$row->destination_field] = $field_mapping;
}
}
}
////////////////////////////////////////////////////////////////////
// Processing
@@ -193,25 +336,25 @@ abstract class Migration extends MigrationBase {
$warn_on_override = TRUE) {
// Warn of duplicate mappings
if ($warn_on_override && !is_null($destination_field) &&
isset($this->fieldMappings[$destination_field])) {
isset($this->codedFieldMappings[$destination_field])) {
self::displayMessage(
t('!name addFieldMapping: !dest was previously mapped from !source, overridden',
array('!name' => $this->machineName, '!dest' => $destination_field,
'!source' => $this->fieldMappings[$destination_field]->getSourceField())),
'!source' => $this->codedFieldMappings[$destination_field]->getSourceField())),
'warning');
}
$mapping = new MigrateFieldMapping($destination_field, $source_field);
if (is_null($destination_field)) {
$this->fieldMappings[] = $mapping;
$this->codedFieldMappings[] = $mapping;
}
else {
$this->fieldMappings[$destination_field] = $mapping;
$this->codedFieldMappings[$destination_field] = $mapping;
}
return $mapping;
}
/**
* Remove any existing mappings for a given destination or source field.
* Remove any existing coded mappings for a given destination or source field.
*
* @param string $destination_field
* Name of the destination field.
@@ -220,12 +363,12 @@ abstract class Migration extends MigrationBase {
*/
public function removeFieldMapping($destination_field, $source_field = NULL) {
if (isset($destination_field)) {
unset($this->fieldMappings[$destination_field]);
unset($this->codedFieldMappings[$destination_field]);
}
if (isset($source_field)) {
foreach ($this->fieldMappings as $key => $mapping) {
foreach ($this->codedFieldMappings as $key => $mapping) {
if ($mapping->getSourceField() == $source_field) {
unset($this->fieldMappings[$key]);
unset($this->codedFieldMappings[$key]);
}
}
}
@@ -254,12 +397,12 @@ abstract class Migration extends MigrationBase {
* @param string $issue_group
* Issue group name to apply to the generated mappings (defaults to 'DNM').
*/
public function addUnmigratedDestinations(array $fields, $issue_group = NULL) {
public function addUnmigratedDestinations(array $fields, $issue_group = NULL, $warn_on_override = TRUE) {
if (!$issue_group) {
$issue_group = t('DNM');
}
foreach ($fields as $field) {
$this->addFieldMapping($field)
$this->addFieldMapping($field, NULL, $warn_on_override)
->issueGroup($issue_group);
}
}
@@ -274,12 +417,12 @@ abstract class Migration extends MigrationBase {
* @param string $issue_group
* Issue group name to apply to the generated mappings (defaults to 'DNM').
*/
public function addUnmigratedSources(array $fields, $issue_group = NULL) {
public function addUnmigratedSources(array $fields, $issue_group = NULL, $warn_on_override = TRUE) {
if (!$issue_group) {
$issue_group = t('DNM');
}
foreach ($fields as $field) {
$this->addFieldMapping(NULL, $field)
$this->addFieldMapping(NULL, $field, $warn_on_override)
->issueGroup($issue_group);
}
}
@@ -289,7 +432,7 @@ abstract class Migration extends MigrationBase {
* source rows have been processed).
*/
public function isComplete() {
$total = $this->source->count(TRUE);
$total = $this->sourceCount(TRUE);
// If the source is uncountable, we have no way of knowing if it's
// complete, so stipulate that it is.
if ($total < 0) {
@@ -551,8 +694,8 @@ abstract class Migration extends MigrationBase {
}
catch (Exception $e) {
self::displayMessage(
t('Migration failed with source plugin exception: !e',
array('!e' => $e->getMessage())));
t('Migration failed with source plugin exception: %e, in %file:%line',
array('%e' => $e->getMessage(), '%file' => $e->getFile(), '%line' => $e->getLine())));
return MigrationBase::RESULT_FAILED;
}
while ($this->source->valid()) {
@@ -570,28 +713,33 @@ abstract class Migration extends MigrationBase {
$ids = $this->destination->import($this->destinationValues, $this->sourceValues);
migrate_instrument_stop('destination import');
if ($ids) {
$this->map->saveIDMapping($this->sourceValues, $ids, $this->needsUpdate,
$this->rollbackAction);
$this->map->saveIDMapping($this->sourceValues, $ids,
$this->needsUpdate, $this->rollbackAction,
$data_row->migrate_map_hash);
$this->successes_since_feedback++;
$this->total_successes++;
}
else {
$this->map->saveIDMapping($this->sourceValues, array(),
MigrateMap::STATUS_FAILED, $this->rollbackAction);
$message = t('New object was not saved, no error provided');
$this->saveMessage($message);
self::displayMessage($message);
MigrateMap::STATUS_FAILED, $this->rollbackAction,
$data_row->migrate_map_hash);
if ($this->map->messageCount() == 0) {
$message = t('New object was not saved, no error provided');
$this->saveMessage($message);
self::displayMessage($message);
}
}
}
catch (MigrateException $e) {
$this->map->saveIDMapping($this->sourceValues, array(),
MigrateMap::STATUS_FAILED, $this->rollbackAction);
$e->getStatus(), $this->rollbackAction, $data_row->migrate_map_hash);
$this->saveMessage($e->getMessage(), $e->getLevel());
self::displayMessage($e->getMessage());
}
catch (Exception $e) {
$this->map->saveIDMapping($this->sourceValues, array(),
MigrateMap::STATUS_FAILED, $this->rollbackAction);
MigrateMap::STATUS_FAILED, $this->rollbackAction,
$data_row->migrate_map_hash);
$this->handleException($e);
}
$this->total_processed++;
@@ -624,8 +772,8 @@ abstract class Migration extends MigrationBase {
}
catch (Exception $e) {
self::displayMessage(
t('Migration failed with source plugin exception: !e',
array('!e' => $e->getMessage())));
t('Migration failed with source plugin exception: %e, in %file:%line',
array('%e' => $e->getMessage(), '%file' => $e->getFile(), '%line' => $e->getLine())));
return MigrationBase::RESULT_FAILED;
}
}
@@ -679,7 +827,7 @@ abstract class Migration extends MigrationBase {
$row = $this->source->current();
// Cheat for XML migrations, which don't pick up the source values
// until applyMappings() applies the xpath()
if (is_a($this, 'XMLMigration')) {
if (is_a($this, 'XMLMigration') && isset($row->xml)) {
$this->sourceValues = $row;
$this->applyMappings();
$row = $this->sourceValues;
@@ -1038,7 +1186,7 @@ abstract class Migration extends MigrationBase {
*/
protected function applyMappings() {
$this->destinationValues = new stdClass;
foreach ($this->fieldMappings as $mapping) {
foreach ($this->getFieldMappings() as $mapping) {
$destination = $mapping->getDestinationField();
// Skip mappings with no destination (source fields marked DNM)
if ($destination) {
@@ -1055,7 +1203,7 @@ abstract class Migration extends MigrationBase {
// If there's a source mapping, and a source value in the data row, copy
// to the destination
if ($source && property_exists($this->sourceValues, $source)) {
if ($source && isset($this->sourceValues->{$source})) {
$destination_values = $this->sourceValues->$source;
}
// Otherwise, apply the default value (if any)
@@ -1141,7 +1289,8 @@ abstract class Migration extends MigrationBase {
}
// We've seen a subfield, so add as an array value.
else {
$this->destinationValues->{$destination_field}[] = $destination_values;
$this->destinationValues->{$destination_field} = array_merge(
(array)$destination_values, $this->destinationValues->{$destination_field});
}
}
}
@@ -1202,15 +1351,17 @@ abstract class Migration extends MigrationBase {
$results = array();
// Each $source_key will be an array of key values
foreach ($source_keys as $source_key) {
// If any source keys are empty, skip this set
// If any source keys are NULL, skip this set
$continue = FALSE;
foreach ($source_key as $value) {
if (empty($value) && $value !== 0 && $value !== '0') {
if (!isset($value)) {
$continue = TRUE;
break;
}
}
if ($continue || empty($source_key)) {
// Occasionally $source_key comes through with an empty string.
$sanity_check = array_filter($source_key);
if ($continue || empty($source_key) || empty($sanity_check)) {
continue;
}
// Loop through each source migration, checking for an existing dest ID.
@@ -1261,7 +1412,7 @@ abstract class Migration extends MigrationBase {
}
else {
$value = reset($results);
return empty($value) ? NULL : $value;
return empty($value) && $value !== 0 && $value !== '0' ? NULL : $value;
}
}
@@ -1372,7 +1523,7 @@ abstract class Migration extends MigrationBase {
/**
* Save any messages we've queued up to the message table.
*/
protected function saveQueuedMessages() {
public function saveQueuedMessages() {
foreach ($this->queuedMessages as $queued_message) {
$this->saveMessage($queued_message['message'], $queued_message['level']);
}
@@ -1391,12 +1542,19 @@ abstract class Migration extends MigrationBase {
}
/**
* Convenience class - deriving from this rather than directory from Migration
* ensures that a class will not be registered as a migration itself - it is
* the implementor's responsibility to register each instance of a dynamic
* migration class.
* @deprecated - This class is no longer necessary, inherit directly from
* Migration instead.
*/
abstract class DynamicMigration extends Migration {
static $deprecationWarning = FALSE;
public function __construct($arguments) {
parent::__construct($arguments);
if (variable_get('migrate_deprecation_warnings', 1) &&
!self::$deprecationWarning) {
self::displayMessage(t('The DynamicMigration class is no longer necessary and is now deprecated - please derive your migration classes directly from Migration.'));
self::$deprecationWarning = TRUE;
}
}
/**
* Overrides default of FALSE
*/

149
sites/all/modules/contrib/migrate/migrate/includes/source.inc
View File

@@ -78,6 +78,20 @@ abstract class MigrateSource implements Iterator {
*/
protected $highwaterField;
/**
* The highwater mark at the beginning of the import operation.
*
* @var
*/
protected $originalHighwater = '';
/**
* Used in the case of multiple key sources that need to use idlist.
*
* @var string
*/
protected $multikeySeparator = ':';
/**
* List of source IDs to process.
*
@@ -116,6 +130,14 @@ abstract class MigrateSource implements Iterator {
*/
protected $skipCount = FALSE;
/**
* If TRUE, we will maintain hashed source rows to determine whether incoming
* data has changed.
*
* @var bool
*/
protected $trackChanges = FALSE;
/**
* By default, next() will directly read the map row and add it to the data
* row. A source plugin implementation may do this itself (in particular, the
@@ -186,6 +208,9 @@ abstract class MigrateSource implements Iterator {
if (!empty($options['cache_key'])) {
$this->cacheKey = $options['cache_key'];
}
if (!empty($options['track_changes'])) {
$this->trackChanges = $options['track_changes'];
}
}
/**
@@ -229,6 +254,9 @@ abstract class MigrateSource implements Iterator {
$this->numProcessed = 0;
$this->numIgnored = 0;
$this->highwaterField = $this->activeMigration->getHighwaterField();
if (!empty($this->highwaterField)) {
$this->originalHighwater = $this->activeMigration->getHighwater();
}
if ($this->activeMigration->getOption('idlist')) {
$this->idList = explode(',', $this->activeMigration->getOption('idlist'));
}
@@ -248,9 +276,11 @@ abstract class MigrateSource implements Iterator {
public function next() {
$this->currentKey = NULL;
$this->currentRow = NULL;
migrate_instrument_start(get_class($this) . ' getNextRow');
while ($row = $this->getNextRow()) {
migrate_instrument_stop(get_class($this) . ' getNextRow');
// Populate the source key for this row
$this->currentKey = $this->activeMigration->prepareKey(
$this->activeMap->getSourceKey(), $row);
@@ -267,23 +297,22 @@ abstract class MigrateSource implements Iterator {
}
}
// First, determine if this row should be passed to prepareRow(), or skipped
// entirely. The rules are:
// First, determine if this row should be passed to prepareRow(), or
// skipped entirely. The rules are:
// 1. If there's an explicit idlist, that's all we care about (ignore
// highwaters and map rows).
$prepared = FALSE;
if (!empty($this->idList)) {
if (in_array(reset($this->currentKey), $this->idList)) {
// In the list, fall through.
}
else {
// Not in the list, skip it
$this->currentRow = NULL;
continue;
if (!in_array(reset($this->currentKey), $this->idList)) {
$compoundKey = implode($this->multikeySeparator, $this->currentKey);
if (count($this->currentKey) > 1 && !in_array($compoundKey, $this->idList)) {
// Could not find the key, skip.
continue;
}
}
}
// 2. If the row is not in the map (we have never tried to import it before),
// we always want to try it.
// 2. If the row is not in the map (we have never tried to import it
// before), we always want to try it.
elseif (!isset($row->migrate_map_sourceid1)) {
// Fall through
}
@@ -293,36 +322,52 @@ abstract class MigrateSource implements Iterator {
}
// 4. At this point, we have a row which has previously been imported and
// not marked for update. If we're not using highwater marks, then we
// will not take this row.
// will not take this row. Except, if we're looking for changes in the
// data, we need to go through prepareRow() before we can decide to
// skip it.
elseif (empty($this->highwaterField)) {
// No highwater, skip
$this->currentRow = NULL;
continue;
if ($this->trackChanges) {
if ($this->prepareRow($row) !== FALSE) {
if ($this->dataChanged($row)) {
// This is a keeper
$this->currentRow = $row;
break;
}
else {
// No change, skip it.
continue;
}
}
else {
// prepareRow() told us to skip it.
continue;
}
}
else {
// No highwater and not tracking changes, skip.
continue;
}
}
// 5. The initial highwater mark, before anything is migrated, is ''. We
// want to make sure we don't mistakenly skip rows with a highwater
// field value of 0, so explicitly handle '' here.
elseif ($this->activeMigration->getHighwater() === '') {
elseif ($this->originalHighwater === '') {
// Fall through
}
// 6. So, we are using highwater marks. Take the row if its highwater field
// value is greater than the saved mark, otherwise skip it.
// 6. So, we are using highwater marks. Take the row if its highwater
// field value is greater than the saved mark, otherwise skip it.
else {
// Call prepareRow() here, in case the highwaterField needs preparation
if ($this->prepareRow($row) !== FALSE) {
if ($row->{$this->highwaterField['name']} > $this->activeMigration->getHighwater()) {
if ($row->{$this->highwaterField['name']} > $this->originalHighwater) {
$this->currentRow = $row;
break;
}
else {
// Skip
$this->currentRow = NULL;
continue;
}
}
else {
$this->currentRow = NULL;
}
$prepared = TRUE;
}
@@ -358,6 +403,10 @@ abstract class MigrateSource implements Iterator {
migrate_instrument_stop(get_class($this->activeMigration) . ' prepareRow');
// We're explicitly skipping this row - keep track in the map table
if ($return === FALSE) {
// Make sure we replace any previous messages for this item with any
// new ones.
$this->activeMigration->getMap()->delete($this->currentKey, TRUE);
$this->activeMigration->saveQueuedMessages();
$this->activeMigration->getMap()->saveIDMapping($row, array(),
MigrateMap::STATUS_IGNORED, $this->activeMigration->rollbackAction);
$this->numIgnored++;
@@ -366,8 +415,62 @@ abstract class MigrateSource implements Iterator {
}
else {
$return = TRUE;
// When tracking changed data, We want to quietly skip (rather than
// "ignore") rows with changes. The caller needs to make that decision,
// so we need to provide them with the necessary information (before and
// after hashes).
if ($this->trackChanges) {
$unhashed_row = clone ($row);
// Remove all map data, otherwise we'll have a false positive on the
// second import (attempt) on a row.
foreach ($unhashed_row as $field => $data) {
if (strpos($field, 'migrate_map_') === 0) {
unset($unhashed_row->$field);
}
}
$row->migrate_map_original_hash = isset($row->migrate_map_hash) ?
$row->migrate_map_hash : '';
$row->migrate_map_hash = $this->hash($unhashed_row);
}
else {
$row->migrate_map_hash = '';
}
}
$this->numProcessed++;
return $return;
}
/**
* Determine whether this row has changed, and therefore whether it should
* be processed.
*
* @param $row
*
* @return bool
*/
protected function dataChanged($row) {
if ($row->migrate_map_original_hash != $row->migrate_map_hash) {
$return = TRUE;
}
else {
$return = FALSE;
}
return $return;
}
/**
* Generate a hash of the source row.
*
* @param $row
*
* @return string
*/
protected function hash($row) {
migrate_instrument_start('MigrateSource::hash');
$hash = md5(serialize($row));
migrate_instrument_stop('MigrateSource::hash');
return $hash;
}
}

81
sites/all/modules/contrib/migrate/migrate/migrate.api.php
View File

@@ -6,17 +6,96 @@
*/
/**
* Registers your module as an implementor of Migrate-based classes.
* Registers your module as an implementor of Migrate-based classes and provides
* default configuration for migration processes.
*
* @return
* An associative array with the following keys (of which only 'api' is
* required):
* - api: Always 2 for any module implementing the Migrate 2 API.
* - groups: An associative array, keyed by group machine name, defining one
* or more migration groups. Each value is an associative array - the 'title'
* key defines a user-visible name for the group; any other values are
* passed as arguments to all migrations in the group.
* - migrations: An associative array, keyed by migration machine name,
* defining one or more migrations. Each value is an associative array - any
* keys other than the following are passed as arguments to the migration
* constructor:
* - class_name (required): The name of the class implementing the migration.
* - group_name: The machine name of the group containing the migration.
* - disable_hooks: An associative array, keyed by hook name, listing hook
* implementations to be disabled during migration. Each value is an
* array of module names whose implementations of the hook in the key is
* to be disabled.
* - destination handlers: An array of classes implementing destination
* handlers.
* - field handlers: An array of classes implementing field handlers.
* - wizard classes: An array of classes that provide Migrate UI wizards.
* - wizard extenders: An array of classes that extend Migrate UI wizards.
* Keys are the wizard classes, values are arrays of extender classes.
*
* See system_hook_info() for all hook groups defined by Drupal core.
*
* @see hook_migrate_api_alter().
*/
function hook_migrate_api() {
$api = array(
'api' => 2,
'groups' => array(
'legacy' => array(
'title' => t('Import from legacy system'),
// Default format for all content migrations
'default_format' => 'filtered_html',
),
),
'migrations' => array(
'ExampleUser' => array(
'class_name' => 'ExampleUserMigration',
'group_name' => 'legacy',
'default_role' => 'member', // Added to constructor $arguments
),
'ExampleNode' => array(
'class_name' => 'ExampleNodeMigration',
'group_name' => 'legacy',
'default_uid' => 1, // Added to constructor $arguments
'disable_hooks' => array(
// Improve migration performance, and prevent accidental emails.
'node_insert' => array(
'expensive_module',
'email_notification_module',
),
'node_update' => array(
'expensive_module',
'email_notification_module',
),
),
),
),
);
return $api;
}
/**
* Alter information from all implementations of hook_migrate_api().
*
* @param array $info
* An array of results from hook_migrate_api(), keyed by module name.
*
* @see hook_migrate_api().
*/
function hook_migrate_api_alter(array &$info) {
// Override the class for another module's migration - say, to add some
// additional preprocessing in prepareRow().
if (isset($info['MODULE_NAME']['migrations']['ExampleNode'])) {
$info['MODULE_NAME']['migrations']['ExampleNode']['class_name'] = 'MyBetterExampleNodeMigration';
}
}
/**
* Provides text to be displayed at the top of the dashboard page (migrate_ui).
*
* @return
* Translated text for display on the dashboard page.
*/
function hook_migrate_overview() {
return t('<p>Listed below are all the migration processes defined for migration

304
sites/all/modules/contrib/migrate/migrate/migrate.drush.inc
View File

@@ -17,12 +17,14 @@ function migrate_drush_command() {
'instrument' => 'Capture performance information (timer, memory, or all)',
'force' => 'Force an operation to run, even if all dependencies are not satisfied',
'group' => 'Name of the migration group to run',
'notify' => 'Send email notification upon completion of operation',
);
$items['migrate-status'] = array(
'description' => 'List all migrations with current status.',
'options' => array(
'refresh' => 'Recognize new migrations and update counts',
'group' => 'Name of the migration group to list',
'names-only' => 'Only return names, not all the details (faster)',
),
'arguments' => array(
'migration' => 'Restrict to a single migration. Optional',
@@ -129,8 +131,6 @@ function migrate_drush_command() {
$items['migrate-rollback'] = array(
'description' => 'Roll back the destination objects from a given migration',
'options' => $migration_options,
// We will bootstrap to login from within the command callback.
'bootstrap' => DRUSH_BOOTSTRAP_DRUPAL_FULL,
'arguments' => array(
'migration' => 'Name of migration(s) to roll back. Delimit multiple using commas.',
),
@@ -144,12 +144,13 @@ function migrate_drush_command() {
'drupal dependencies' => array('migrate'),
'aliases' => array('mr'),
);
$migration_options['update'] = 'In addition to processing unimported items from the source, update previously-imported items with new data';
$migration_options['update'] = 'In addition to processing unprocessed items from the source, update previously-imported items with new data';
$migration_options['needs-update'] =
'Reimport up to 10K records where needs_update=1. This option is only needed when your Drupal DB is on a different DB server from your source data. Otherwise, these records get migrated with just migrate-import.';
$migration_options['stop'] = 'Stop specified migration(s) if applicable.';
$migration_options['rollback'] = 'Rollback specified migration(s) if applicable.';
$migration_options['file_function'] = 'Override file function to use when migrating images.';
$migration_options['ignore-highwater'] = 'Ignore the highwater field during migration';
$items['migrate-import'] = array(
'description' => 'Perform one or more migration processes',
'options' => $migration_options,
@@ -172,7 +173,8 @@ function migrate_drush_command() {
);
$items['migrate-stop'] = array(
'description' => 'Stop an active migration operation',
'options' => array('all' => 'Stop all active migration operations'),
'options' => array('all' => 'Stop all active migration operations',
'group' => 'Name of a specific migration group to stop'),
'arguments' => array(
'migration' => 'Name of migration to stop',
),
@@ -198,21 +200,30 @@ function migrate_drush_command() {
);
$items['migrate-deregister'] = array(
'description' => 'Remove all tracking of a migration',
'options' => array('orphans' => 'Remove tracking for any migrations whose implementing class no longer exists'),
'options' => array(
'orphans' => 'Remove tracking for any migrations whose implementing class no longer exists',
'group' => 'Remove tracking of a migration group, and any migrations assigned to it',
),
'arguments' => array(
'migration' => 'Name of migration to deregister',
),
'examples' => array(
'migrate-deregister Article' => 'Deregister the Article migration',
'migrate-deregister --orphans' => 'Deregister any no-longer-implemented migrations',
'migrate-deregister --group=myblog' => 'Deregister the myblog group and all migrations within it',
),
'drupal dependencies' => array('migrate'),
);
$items['migrate-auto-register'] = array(
'description' => 'Register any newly-defined migration classes',
'description' => 'Register any newly defined migration classes',
'drupal dependencies' => array('migrate'),
'aliases' => array('mar'),
);
$items['migrate-register'] = array(
'description' => 'Register or reregister any statically defined migrations',
'drupal dependencies' => array('migrate'),
'aliases' => array('mreg'),
);
$items['migrate-wipe'] = array(
'description' => 'Delete all nodes from specified content types.',
'examples' => array(
@@ -236,9 +247,11 @@ function migrate_drush_command() {
*/
function drush_migrate_get_options() {
$options = array();
$blacklist = array('stop', 'rollback', 'update', 'all');
$blacklist = array('stop', 'rollback', 'update', 'all', 'group');
$command = drush_parse_command();
foreach ($command['options'] as $key => $value) {
$global_options = drush_get_global_options();
$opts = array_merge($command['options'], $global_options);
foreach ($opts as $key => $value) {
// Strip leading --
$key = ltrim($key, '-');
if (!in_array($key, $blacklist)) {
@@ -281,6 +294,7 @@ function drush_migrate_status($name = NULL) {
try {
$refresh = drush_get_option('refresh');
$group_option = drupal_strtolower(drush_get_option('group'));
$names_only = drush_get_option('names-only');
// Validate input and load Migration(s).
if ($name) {
@@ -311,56 +325,69 @@ function drush_migrate_status($name = NULL) {
if ($group_members_count == 1) {
// An empty line and the headers.
$table[] = array('');
$table[] = array(dt('Group: !name', array('!name' => $group->getName())), dt('Total'), dt('Imported'), dt('Unimported'), dt('Status'), dt('Last imported'));
if ($names_only) {
$table[] = array(dt('Group: !name',
array('!name' => $group->getName())));
}
else {
$table[] = array(dt('Group: !name',
array('!name' => $group->getName())), dt('Total'), dt('Imported'),
dt('Unprocessed'), dt('Status'), dt('Last imported'));
}
}
$has_counts = TRUE;
if (method_exists($migration, 'sourceCount')) {
$total = $migration->sourceCount($refresh);
if ($total < 0) {
if (!$names_only) {
$has_counts = TRUE;
if (method_exists($migration, 'sourceCount')) {
$total = $migration->sourceCount($refresh);
if ($total < 0) {
$has_counts = FALSE;
$total = dt('N/A');
}
}
else {
$has_counts = FALSE;
$total = dt('N/A');
}
if (method_exists($migration, 'importedCount')) {
$imported = $migration->importedCount();
$processed = $migration->processedCount();
}
else {
$has_counts = FALSE;
$imported = dt('N/A');
}
if ($has_counts) {
$unimported = $total - $processed;
}
else {
$unimported = dt('N/A');
}
$status = $migration->getStatus();
switch ($status) {
case MigrationBase::STATUS_IDLE:
$status = dt('Idle');
break;
case MigrationBase::STATUS_IMPORTING:
$status = dt('Importing');
break;
case MigrationBase::STATUS_ROLLING_BACK:
$status = dt('Rolling back');
break;
case MigrationBase::STATUS_STOPPING:
$status = dt('Stopping');
break;
case MigrationBase::STATUS_DISABLED:
$status = dt('Disabled');
break;
default:
$status = dt('Unknown');
break;
}
$table[] = array($migration->getMachineName(), $total, $imported, $unimported, $status, $migration->getLastImported());
}
else {
$has_counts = FALSE;
$total = dt('N/A');
$table[] = array($migration->getMachineName());
}
if (method_exists($migration, 'importedCount')) {
$imported = $migration->importedCount();
$processed = $migration->processedCount();
}
else {
$has_counts = FALSE;
$imported = dt('N/A');
}
if ($has_counts) {
$unimported = $total - $processed;
}
else {
$unimported = dt('N/A');
}
$status = $migration->getStatus();
switch ($status) {
case MigrationBase::STATUS_IDLE:
$status = dt('Idle');
break;
case MigrationBase::STATUS_IMPORTING:
$status = dt('Importing');
break;
case MigrationBase::STATUS_ROLLING_BACK:
$status = dt('Rolling back');
break;
case MigrationBase::STATUS_STOPPING:
$status = dt('Stopping');
break;
case MigrationBase::STATUS_DISABLED:
$status = dt('Disabled');
break;
default:
$status = dt('Unknown');
break;
}
$table[] = array($migration->getMachineName(), $total, $imported, $unimported, $status, $migration->getLastImported());
}
}
drush_print_table($table);
@@ -382,7 +409,7 @@ function drush_migrate_fields_destination($args = NULL) {
if (method_exists($destination, 'fields')) {
$table = array();
foreach ($destination->fields($migration) as $machine_name => $description) {
$table[] = array($description, $machine_name);
$table[] = array(strip_tags($description), $machine_name);
}
drush_print_table($table);
}
@@ -407,7 +434,7 @@ function drush_migrate_fields_source($args = NULL) {
if (method_exists($source, 'fields')) {
$table = array();
foreach ($source->fields() as $machine_name => $description) {
$table[] = array($description, $machine_name);
$table[] = array(strip_tags($description), $machine_name);
}
drush_print_table($table);
}
@@ -438,14 +465,20 @@ function drush_migrate_mappings($args = NULL) {
$dest_descriptions = array();
if (method_exists($destination, 'fields')) {
foreach ($destination->fields($migration) as $machine_name => $description) {
$dest_descriptions[$machine_name] = $description;
if (is_array($description)) {
$description = reset($description);
}
$dest_descriptions[$machine_name] = strip_tags($description);
}
}
$source = $migration->getSource();
$src_descriptions = array();
if (method_exists($source, 'fields')) {
foreach ($source->fields() as $machine_name => $description) {
$src_descriptions[$machine_name] = $description;
if (is_array($description)) {
$description = reset($description);
}
$src_descriptions[$machine_name] = strip_tags($description);
}
}
}
@@ -545,6 +578,10 @@ function drush_migrate_mappings($args = NULL) {
* Display messages for a migration.
*/
function drush_migrate_messages($migration_name) {
if (!trim($migration_name)) {
drush_log(dt('You must specify a migration name'), 'status');
return;
}
try {
$migration = MigrationBase::getInstance($migration_name);
if (is_a($migration, 'Migration')) {
@@ -783,6 +820,21 @@ function drush_migrate_audit($args = NULL) {
*/
function drush_migrate_rollback($args = NULL) {
try {
if (drush_get_option('notify', FALSE)) {
// Capture non-informational output for mailing
ob_start();
ob_implicit_flush(FALSE);
// Save original mail setup, which Migrate will disable, so we can
// restore it later.
global $conf;
if (!empty($conf['mail_system'])) {
$mail_system = $conf['mail_system'];
}
else {
$mail_system = NULL;
}
}
$migrations = drush_migrate_get_migrations($args);
// Rollback in reverse order
@@ -873,6 +925,37 @@ function drush_migrate_rollback($args = NULL) {
if ($_migrate_track_timer && !drush_get_context('DRUSH_DEBUG')) {
drush_print_timers();
}
// Notify user
if (drush_get_option('notify')) {
if (is_null($mail_system)) {
unset($conf['mail_system']);
}
else {
$conf['mail_system'] = $mail_system;
}
_drush_migrate_notify();
}
}
/**
* Send email notification to the user running the operation.
*/
function _drush_migrate_notify() {
global $user;
if ($user->uid) {
$uid = $user->uid;
}
else {
$uid = 1;
}
$account = user_load($uid);
$params['account'] = $account;
$params['output'] = ob_get_contents();
drush_print_r(ob_get_status());
ob_end_flush();
drupal_mail('migrate_ui', 'import_complete', $account->mail,
user_preferred_language($account), $params);
}
function drush_migrate_get_migrations($args) {
@@ -883,7 +966,7 @@ function drush_migrate_get_migrations($args) {
$seen = $start === TRUE ? TRUE : FALSE;
foreach ($migration_objects as $name => $migration) {
if (!$seen && (drupal_strtolower($start) . 'migration' == drupal_strtolower($name))) {
if (!$seen && (drupal_strtolower($start) == drupal_strtolower($name))) {
// We found our starting migration. $seen is always TRUE now.
$seen = TRUE;
}
@@ -1040,6 +1123,20 @@ function drush_migrate_pre_migrate_import($args = NULL) {
*/
function drush_migrate_import($args = NULL) {
try {
if (drush_get_option('notify', FALSE)) {
// Capture non-informational output for mailing
ob_start();
ob_implicit_flush(FALSE);
// Save original mail setup, which Migrate will disable, so we can
// restore it later.
global $conf;
if (!empty($conf['mail_system'])) {
$mail_system = $conf['mail_system'];
}
else {
$mail_system = NULL;
}
}
$migrations = drush_migrate_get_migrations($args);
$options = array();
if ($idlist = drush_get_option('idlist', FALSE)) {
@@ -1101,8 +1198,12 @@ function drush_migrate_import($args = NULL) {
foreach ($migrations as $machine_name => $migration) {
drush_log(dt("Importing '!description' migration",
array('!description' => $machine_name)));
if (drush_get_option('update')) {
if (drush_get_option('update') && !$idlist) {
$migration->prepareUpdate();
if (drush_get_option('ignore-highwater')) {
$migration->setHighwaterField(array());
}
}
if (drush_get_option('needs-update')) {
$map_rows = $migration->getMap()->getRowsNeedingUpdate(10000);
@@ -1190,14 +1291,25 @@ function drush_migrate_import($args = NULL) {
if ($_migrate_track_timer && !drush_get_context('DRUSH_DEBUG')) {
drush_print_timers();
}
// Notify user
if (drush_get_option('notify')) {
if (is_null($mail_system)) {
unset($conf['mail_system']);
}
else {
$conf['mail_system'] = $mail_system;
}
_drush_migrate_notify();
}
}
//**
// * Stop clearing or importing a given content set.
// *
// * @param $content_set
// * The name of the Migration
// */
/**
* Stop clearing or importing a given content set.
*
* @param $content_set
* The name of the Migration
*/
function drush_migrate_stop($args = NULL) {
try {
$migrations = drush_migrate_get_migrations($args);
@@ -1231,31 +1343,40 @@ function drush_migrate_reset_status($args = NULL) {
}
/**
* Deregister a given migration, or all orphaned migrations. Note that the
* migration might no longer "exist" (the class implementation might be gone),
* so we can't count on being able to instantiate it, or use migrate_migrations().
* Deregister a given migration, migration group, or all orphaned migrations.
* Note that the migration might no longer "exist" (the class implementation
* might be gone), so we can't count on being able to instantiate it, or use
* migrate_migrations().
*/
function drush_migrate_deregister($args = NULL) {
try {
$orphans = drush_get_option('orphans');
if ($orphans) {
$migrations = array();
$result = db_select('migrate_status', 'ms')
->fields('ms', array('class_name', 'machine_name'))
->execute();
foreach ($result as $row) {
if (!class_exists($row->class_name)) {
$migrations[] = $row->machine_name;
}
}
$group = drush_get_option('group');
if ($group) {
MigrateGroup::deregister($group);
drush_log(dt("Deregistered group '!description' and all its migrations",
array('!description' => $group)), 'success');
}
else {
$migrations = explode(',', $args);
}
foreach ($migrations as $machine_name) {
drush_migrate_deregister_migration($machine_name);
drush_log(dt("Deregistered '!description' migration",
array('!description' => $machine_name)), 'success');
if ($orphans) {
$migrations = array();
$result = db_select('migrate_status', 'ms')
->fields('ms', array('class_name', 'machine_name'))
->execute();
foreach ($result as $row) {
if (!class_exists($row->class_name)) {
$migrations[] = $row->machine_name;
}
}
}
else {
$migrations = explode(',', $args);
}
foreach ($migrations as $machine_name) {
drush_migrate_deregister_migration(drupal_strtolower($machine_name));
drush_log(dt("Deregistered '!description' migration",
array('!description' => $machine_name)), 'success');
}
}
}
catch (MigrateException $e) {
@@ -1277,13 +1398,28 @@ function drush_migrate_deregister_migration($machine_name) {
db_delete('migrate_status')
->condition('machine_name', $machine_name)
->execute();
db_delete('migrate_field_mapping')
->condition('machine_name', $machine_name)
->execute();
}
/**
* Register any previously-unrecognized non-dynamic migrations.
* Auto-registration is no longer supported. This command should be removed
* entirely in a future point release.
*
* @deprecated
*/
function drush_migrate_auto_register($args = NULL) {
migrate_autoregister();
drush_log(dt('The auto-registration feature has been removed. Migrations '
. 'must now be explicitly registered.'), 'error');
}
/**
* Register any migrations defined in hook_migrate_api().
*/
function drush_migrate_register($args = NULL) {
migrate_static_registration();
drush_log(dt('All statically defined migrations have been (re)registered.'), 'success');
}
/**

13
sites/all/modules/contrib/migrate/migrate/migrate.info
View File

@@ -1,6 +1,6 @@
name = "Migrate"
description = "Import content from external sources"
package = "Development"
package = "Migration"
core = 7.x
files[] = includes/base.inc
@@ -14,6 +14,7 @@ files[] = includes/map.inc
files[] = includes/source.inc
files[] = includes/team.inc
files[] = migrate.mail.inc
files[] = plugins/destinations/block_custom.inc
files[] = plugins/destinations/entity.inc
files[] = plugins/destinations/term.inc
files[] = plugins/destinations/user.inc
@@ -28,15 +29,19 @@ files[] = plugins/destinations/table_copy.inc
files[] = plugins/destinations/menu.inc
files[] = plugins/destinations/menu_links.inc
files[] = plugins/destinations/statistics.inc
files[] = plugins/destinations/variable.inc
files[] = plugins/sources/csv.inc
files[] = plugins/sources/db2.inc
files[] = plugins/sources/files.inc
files[] = plugins/sources/json.inc
files[] = plugins/sources/list.inc
files[] = plugins/sources/mongodb.inc
files[] = plugins/sources/multiitems.inc
files[] = plugins/sources/sql.inc
files[] = plugins/sources/sqlmap.inc
files[] = plugins/sources/mssql.inc
files[] = plugins/sources/oracle.inc
files[] = plugins/sources/spreadsheet.inc
files[] = plugins/sources/xml.inc
files[] = tests/import/options.test
files[] = tests/plugins/destinations/comment.test
@@ -46,9 +51,9 @@ files[] = tests/plugins/destinations/term.test
files[] = tests/plugins/destinations/user.test
files[] = tests/plugins/sources/xml.test
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2015-02-09
version = "7.x-2.7"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1423521491"

242
sites/all/modules/contrib/migrate/migrate/migrate.install
View File

@@ -9,6 +9,8 @@ function migrate_schema() {
$schema = array();
$schema['migrate_status'] = migrate_schema_status();
$schema['migrate_log'] = migrate_schema_log();
$schema['migrate_group'] = migrate_schema_group();
$schema['migrate_field_mapping'] = migrate_schema_field_mapping();
return $schema;
}
@@ -28,6 +30,12 @@ function migrate_schema_status() {
'not null' => TRUE,
'description' => 'Name of class to instantiate for this migration',
),
'group_name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Name of group containing migration',
),
'status' => array(
'type' => 'int',
'size' => 'tiny',
@@ -115,6 +123,74 @@ function migrate_schema_log() {
);
}
function migrate_schema_group() {
return array(
'description' => 'Information on migration groups',
'fields' => array(
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Unique machine name for a migration group',
),
'title' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Display name for a migration group',
),
'arguments' => array(
'type' => 'blob',
'not null' => FALSE,
'size' => 'big',
'serialize' => TRUE,
'description' => 'A serialized array of arguments to the migration group',
),
),
'primary key' => array('name'),
);
}
function migrate_schema_field_mapping() {
return array(
'description' => 'History of migration processes',
'fields' => array(
'fmid' => array(
'type' => 'serial',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Unique ID for the field mapping row',
),
'machine_name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Parent migration for the field mapping',
),
'destination_field' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Destination field for the field mapping',
),
'source_field' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Source field for the field mapping',
),
'options' => array(
'type' => 'blob',
'not null' => FALSE,
'size' => 'big',
'serialize' => TRUE,
'description' => 'A serialized MigrateFieldMapping object holding all options',
),
),
'primary key' => array('fmid'),
);
}
/**
* Implements hook_uninstall().
* Drop map/message tables, in case implementing classes did not.
@@ -137,13 +213,17 @@ function migrate_uninstall() {
->condition('module', 'migrate')
->execute();
}
// Remove variables
variable_del('migrate_disable_autoregistration');
variable_del('migrate_disabled_handlers');
variable_del('migrate_deprecation_warnings');
}
/**
* Add highwater mark
*/
function migrate_update_7001() {
$ret = array();
if (!db_field_exists('migrate_status', 'highwater')) {
db_add_field('migrate_status', 'highwater', array(
'type' => 'varchar',
@@ -155,7 +235,7 @@ function migrate_update_7001() {
);
}
$ret[] = t('Added highwater column to migrate_status table');
$ret = t('Added highwater column to migrate_status table');
return $ret;
}
@@ -163,7 +243,6 @@ function migrate_update_7001() {
* Add last_imported field to all map tables
*/
function migrate_update_7002() {
$ret = array();
foreach (db_find_tables('migrate_map_%') as $tablename) {
if (!db_field_exists($tablename, 'last_imported')) {
db_add_field($tablename, 'last_imported', array(
@@ -175,7 +254,7 @@ function migrate_update_7002() {
));
}
}
$ret[] = t('Added last_imported column to all map tables');
$ret = t('Added last_imported column to all map tables');
return $ret;
}
@@ -183,7 +262,7 @@ function migrate_update_7002() {
* Add lastthroughput column to migrate_status
*/
function migrate_update_7003() {
$ret = array();
$ret = '';
if (!db_field_exists('migrate_status', 'lastthroughput')) {
db_add_field('migrate_status', 'lastthroughput', array(
'type' => 'int',
@@ -194,7 +273,7 @@ function migrate_update_7003() {
);
}
$ret[] = t('Added lastthroughput column to migrate_status table');
$ret = t('Added lastthroughput column to migrate_status table');
return $ret;
}
@@ -202,7 +281,7 @@ function migrate_update_7003() {
* Convert lastimported datetime field to lastimportedtime int field.
*/
function migrate_update_7004() {
$ret = array();
$ret = '';
if (!db_field_exists('migrate_status', 'lastimportedtime')) {
db_add_field('migrate_status', 'lastimportedtime', array(
'type' => 'int',
@@ -212,7 +291,7 @@ function migrate_update_7004() {
)
);
if (!db_field_exists('migrate_status', 'lastimported')) {
if (db_field_exists('migrate_status', 'lastimported')) {
$result = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name', 'lastimported'))
->execute();
@@ -226,7 +305,7 @@ function migrate_update_7004() {
db_drop_field('migrate_status', 'lastimported');
$ret[] = t('Converted lastimported datetime field to lastimportedtime int field');
$ret .= "\n" . t('Converted lastimported datetime field to lastimportedtime int field');
}
}
return $ret;
@@ -236,11 +315,11 @@ function migrate_update_7004() {
* Add support for history logging
*/
function migrate_update_7005() {
$ret = array();
$ret = '';
if (!db_table_exists('migrate_log')) {
$ret[] = t('Create migrate_log table');
$ret .= "\n" . t('Create migrate_log table');
db_create_table('migrate_log', migrate_schema_log());
$ret[] = t('Remove historic columns from migrate_status table');
$ret .= "\n" . t('Remove historic columns from migrate_status table');
db_drop_field('migrate_status', 'lastthroughput');
db_drop_field('migrate_status', 'lastimportedtime');
}
@@ -252,7 +331,7 @@ function migrate_update_7005() {
* dependencies or sourceMigration() must be changed! See CHANGELOG.txt.
*/
function migrate_update_7006() {
$ret = array();
$ret = '';
if (!db_field_exists('migrate_status', 'class_name')) {
db_add_field('migrate_status', 'class_name', array(
'type' => 'varchar',
@@ -266,7 +345,7 @@ function migrate_update_7006() {
db_query("UPDATE {migrate_status}
SET class_name = CONCAT(machine_name, 'Migration')
");
$ret[] = t('Added class_name column to migrate_status table');
$ret = t('Added class_name column to migrate_status table');
}
return $ret;
}
@@ -275,7 +354,7 @@ function migrate_update_7006() {
* Add arguments field to migrate_status table.
*/
function migrate_update_7007() {
$ret = array();
$ret = '';
if (!db_field_exists('migrate_status', 'arguments')) {
db_add_field('migrate_status', 'arguments', array(
'type' => 'blob',
@@ -286,7 +365,7 @@ function migrate_update_7007() {
)
);
$ret[] = t('Added arguments column to migrate_status table');
$ret = t('Added arguments column to migrate_status table');
}
return $ret;
}
@@ -298,10 +377,9 @@ function migrate_update_7008() {
// Updates can be run when the module is disabled, which would mean the
// call to migrate_migrations() will fail. Just bail in that case...
if (!module_exists('migrate')) {
throw new DrupalUpdateException(t('This update cannot be run while the Migrate ' .
'module is disabled - you must enable Migrate to run this update.'));
throw new DrupalUpdateException(t('This update cannot be run while the Migrate module is disabled - you must enable Migrate to run this update.'));
}
$ret = array();
$ret = '';
foreach (migrate_migrations() as $migration) {
if (is_a($migration, 'Migration')) {
// Since we're now tracking failed/ignored rows in the map table,
@@ -317,7 +395,7 @@ function migrate_update_7008() {
$field_schema['not null'] = FALSE;
$map_connection->schema()->changeField($map_table, $field, $field,
$field_schema);
$ret[] = t('Changed !table.!field to be non-null',
$ret .= "\n" . t('Changed !table.!field to be non-null',
array('!table' => $map_table, '!field' => $field));
}
@@ -343,7 +421,7 @@ function migrate_update_7008() {
$msg_marked = TRUE;
}
if ($msg_marked) {
$ret[] = t('Marked failures in !table', array('!table' => $map_table));
$ret .= "\n" . t('Marked failures in !table', array('!table' => $map_table));
}
}
}
@@ -363,10 +441,11 @@ function migrate_update_7201() {
}
/**
* Add rollback_action field to all map tables
* Add rollback_action field to all map tables in the Drupal database.
*/
function migrate_update_7202() {
$ret = array();
// Note this won't catch any prefixed tables, or any stored in the source
// database - ensureTables() will take care of those.
foreach (db_find_tables('migrate_map_%') as $tablename) {
if (!db_field_exists($tablename, 'rollback_action')) {
db_add_field($tablename, 'rollback_action', array(
@@ -379,6 +458,121 @@ function migrate_update_7202() {
));
}
}
$ret[] = t('Added rollback_action column to all map tables');
$ret = t('Added rollback_action column to all map tables');
return $ret;
}
/**
* Add database tracking of per-group info.
*/
function migrate_update_7203() {
$ret = '';
if (!db_table_exists('migrate_group')) {
$ret .= t('Create migrate_group table') . "\n";
db_create_table('migrate_group', migrate_schema_group());
}
if (!db_field_exists('migrate_status', 'group_name')) {
$ret .= t('Add group relationship to migrate_status table'). "\n";
db_add_field('migrate_status', 'group_name', array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => 'default',
'description' => 'Name of group containing migration',
)
);
// Populate each migration's group_name field
$groups = array();
foreach (migrate_migrations() as $machine_name => $migration) {
$group_name = $migration->getGroup()->getName();
if (empty($group_name)) {
$group_name = 'default';
}
$groups[$group_name] = $group_name;
db_update('migrate_status')
->fields(array('group_name' => $group_name))
->condition('machine_name', $machine_name)
->execute();
}
// Populate the migrate_group table
foreach ($groups as $group_name) {
$title = db_select('migrate_group', 'mg')
->fields('mg', array('title'))
->condition('name', $group_name)
->execute()
->fetchField();
if (!$title) {
db_insert('migrate_group')
->fields(array(
'name' => $group_name,
'title' => $group_name,
'arguments' => serialize(array()),
))
->execute();
}
}
}
return $ret;
}
/**
* Add database tracking of field mappings.
*/
function migrate_update_7204() {
$ret = '';
if (!db_table_exists('migrate_field_mapping')) {
$ret = t('Create migrate_field_mapping table');
db_create_table('migrate_field_mapping', migrate_schema_field_mapping());
}
return $ret;
}
/**
* Remove obsolete autoregistration disablement.
*/
function migrate_update_7205() {
variable_del('migrate_disable_autoregistration');
}
/**
* Replace three-column PK with a simple serial.
*/
function migrate_update_7206() {
if (!db_field_exists('migrate_field_mapping', 'fmid')) {
db_drop_primary_key('migrate_field_mapping');
db_add_field('migrate_field_mapping', 'fmid',
array(
'type' => 'serial',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Unique ID for the field mapping row',
),
array(
'primary key' => array('fmid'),
)
);
}
}
/**
* Make sure we remove an empty 'default' group created by the previous updates.
*/
function migrate_update_7207() {
$rows = db_select('migrate_group', 'mg')
->fields('mg', array('name'))
->condition('name', 'default')
->execute()
->rowCount();
if ($rows > 0) {
$rows = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name'))
->condition('group_name', 'default')
->execute()
->rowCount();
if ($rows == 0) {
db_delete('migrate_group')
->condition('name', 'default')
->execute();
}
}
}

373
sites/all/modules/contrib/migrate/migrate/migrate.module
View File

@@ -15,16 +15,24 @@
define('MIGRATE_API_VERSION', 2);
define('MIGRATE_ACCESS_BASIC', 'migration information');
define('MIGRATE_ACCESS_ADVANCED', 'advanced migration information');
/**
* Retrieve a list of all active migrations, ordered by dependencies. To be
* recognized, a class must be non-abstract, and derived from MigrationBase.
*
* @param $reset
* If TRUE, the static cache of migrations will be flushed before attempting to
* reinstantiate all active migrations. This can be important for script runs
* where migration classes may be dynamically registered.
*
* @return
* Array of migration objects, keyed by the machine name.
*/
function migrate_migrations() {
function migrate_migrations($reset = NULL) {
static $migrations = array();
if (!empty($migrations)) {
if (!empty($migrations) && empty($reset)) {
return $migrations;
}
@@ -32,31 +40,30 @@ function migrate_migrations() {
// make sure any dynamic migrations defined in hook_migrate_api() get registered.
migrate_get_module_apis(TRUE);
$dependencies_list = array();
$dependent_migrations = array();
$required_migrations = array();
$result = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name', 'class_name', 'arguments'))
->fields('ms', array('machine_name', 'class_name'))
->execute();
foreach ($result as $row) {
if (class_exists($row->class_name)) {
$reflect = new ReflectionClass($row->class_name);
if (!$reflect->isAbstract() && $reflect->isSubclassOf('MigrationBase')) {
$arguments = unserialize($row->arguments);
if (!$arguments || !is_array($arguments)) {
$arguments = array();
}
$migration = MigrationBase::getInstance($row->machine_name,
$row->class_name, $arguments);
$dependencies = $migration->getDependencies();
if (count($dependencies) > 0) {
// Set classes with dependencies aside for reordering
$dependent_migrations[$row->machine_name] = $migration;
$required_migrations += $dependencies;
}
else {
// No dependencies, just add
$migrations[$row->machine_name] = $migration;
$migration = MigrationBase::getInstance($row->machine_name);
if ($migration) {
$dependencies = $migration->getDependencies();
$dependencies_list[$row->machine_name] = $dependencies;
if (count($dependencies) > 0) {
// Set classes with dependencies aside for reordering
$dependent_migrations[$row->machine_name] = $migration;
$required_migrations += $dependencies;
}
else {
// No dependencies, just add
$migrations[$row->machine_name] = $migration;
}
}
}
else {
@@ -70,32 +77,10 @@ function migrate_migrations() {
}
}
// Scan modules with dependencies - we'll take 20 passes at it before
// giving up
// TODO: Can we share code with _migrate_class_list()?
$iterations = 0;
while (count($dependent_migrations) > 0) {
if ($iterations++ > 20) {
$migration_names = implode(',', array_keys($dependent_migrations));
throw new MigrateException(t('Failure to sort migration list - most likely due ' .
'to circular dependencies involving !migration_names',
array('!migration_names' => $migration_names)));
}
foreach ($dependent_migrations as $name => $migration) {
$ready = TRUE;
// Scan all the dependencies for this class and make sure they're all
// in the final list
foreach ($migration->getDependencies() as $dependency) {
if (!isset($migrations[$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this class to the final list
$migrations[$name] = $migration;
unset($dependent_migrations[$name]);
}
$ordered_migrations = migrate_order_dependencies($dependencies_list);
foreach ($ordered_migrations as $name) {
if (!isset($migrations[$name])) {
$migrations[$name] = $dependent_migrations[$name];
}
}
@@ -123,59 +108,6 @@ function migrate_migrations() {
return $migrations;
}
/**
* On request, scan the Drupal code registry for any new migration classes
* for us to register in migrate_status.
*/
function migrate_autoregister() {
// Make sure the registry is up-to-date on all available classes.
require_once 'includes/registry.inc';
_registry_update();
// Get list of modules implementing Migrate API
$modules = array_keys(migrate_get_module_apis(TRUE));
// Get list of classes we already know about
$existing_classes = db_select('migrate_status', 'ms')
->fields('ms', array('class_name'))
->execute()
->fetchCol();
// Discover class names registered with Drupal by modules implementing our API
$result = db_select('registry', 'r')
->fields('r', array('name'))
->condition('type', 'class')
->condition('module', $modules, 'IN')
->condition('filename', '%.test', 'NOT LIKE')
->execute();
foreach ($result as $record) {
$class_name = $record->name;
// If we already know about this class, skip it
if (isset($existing_classes[$class_name])) {
continue;
}
// Validate it's an implemented subclass of the parent class
// Ignore errors
try {
$class = new ReflectionClass($class_name);
}
catch (Exception $e) {
continue;
}
if (!$class->isAbstract() && $class->isSubclassOf('MigrationBase')) {
// Verify that it's not a dynamic class (the implementor will be responsible
// for registering those).
$dynamic = call_user_func(array($class_name, 'isDynamic'));
if (!$dynamic) {
// OK, this is a new non-dynamic migration class, register it
MigrationBase::registerMigration($class_name);
}
}
}
}
/**
* Invoke any available handlers attached to a given destination type.
* If any handlers have dependencies defined, they will be invoked after
@@ -216,8 +148,6 @@ function migrate_handler_invoke_all($destination, $method) {
/**
* Invoke any available handlers attached to a given field type.
* If any handlers have dependencies defined, they will be invoked after
* the specified handlers.
*
* @param $entity
* The object we are building up before calling example_save().
@@ -231,18 +161,21 @@ function migrate_handler_invoke_all($destination, $method) {
* @param $method
* Handler method to call (defaults to prepare()).
*/
function migrate_field_handler_invoke_all($entity, array $field_info, array $instance,
array $values, $method = 'prepare') {
function migrate_field_handler_invoke_all($entity, array $field_info,
array $instance, array $values, $method = 'prepare') {
$return = array();
$type = $field_info['type'];
$class_list = _migrate_class_list('MigrateFieldHandler');
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
$disabled = unserialize(variable_get('migrate_disabled_handlers',
serialize(array())));
$handler_called = FALSE;
foreach ($class_list as $class_name => $handler) {
if (!in_array($class_name, $disabled) && $handler->handlesType($type)
&& method_exists($handler, $method)) {
migrate_instrument_start($class_name . '->' . $method);
$result = call_user_func_array(array($handler, $method),
array($entity, $field_info, $instance, $values));
$handler_called = TRUE;
migrate_instrument_stop($class_name . '->' . $method);
if (isset($result) && is_array($result)) {
$return = array_merge_recursive($return, $result);
@@ -252,6 +185,20 @@ function migrate_field_handler_invoke_all($entity, array $field_info, array $ins
}
}
}
if (!$handler_called && $method == 'prepare') {
$handler = new MigrateDefaultFieldHandler();
migrate_instrument_start('MigrateDefaultFieldHandler->prepare');
$result = call_user_func_array(array($handler, 'prepare'),
array($entity, $field_info, $instance, $values));
migrate_instrument_stop('MigrateDefaultFieldHandler->prepare');
if (isset($result) && is_array($result)) {
$return = array_merge_recursive($return, $result);
}
elseif (isset($result)) {
$return[] = $result;
}
}
return $return;
}
@@ -269,10 +216,9 @@ function migrate_field_handler_invoke_all($entity, array $field_info, array $ins
function _migrate_class_list($parent_class) {
// Get info on modules implementing Migrate API
static $module_info;
if (!isset($modules)) {
if (!isset($module_info)) {
$module_info = migrate_get_module_apis();
}
$modules = array_keys($module_info);
static $class_lists = array();
if (!isset($class_lists[$parent_class])) {
@@ -291,90 +237,6 @@ function _migrate_class_list($parent_class) {
}
}
}
// Avoid scrounging the registry for handler classes if possible.
if (variable_get('migrate_disable_autoregistration', FALSE)) {
return $class_lists[$parent_class];
}
$dependent_classes = array();
$required_classes = array();
// Discover class names registered with Drupal by modules implementing our API
$result = db_select('registry', 'r')
->fields('r', array('name'))
->condition('type', 'class')
->condition('module', $modules, 'IN')
->condition('filename', '%.test', 'NOT LIKE')
->execute();
foreach ($result as $record) {
// Validate it's an implemented subclass of the parent class
// We can get funky errors here, ignore them (and the class that caused them)
try {
$class = new ReflectionClass($record->name);
}
catch (Exception $e) {
continue;
}
if (!$class->isAbstract() && $class->isSubclassOf($parent_class)) {
// If the constructor has required parameters, this may fail. We will
// silently ignore - it is up to the implementor of such a class to
// instantiate it in hook_migrations_alter().
try {
$object = new $record->name;
}
catch (Exception $e) {
unset($object);
}
if (isset($object)) {
$dependencies = $object->getDependencies();
if (count($dependencies) > 0) {
// Set classes with dependencies aside for reordering
$dependent_classes[$record->name] = $object;
$required_classes += $dependencies;
}
else {
// No dependencies, just add
$class_lists[$parent_class][$record->name] = $object;
}
}
}
}
// Validate that each depended-on class at least exists
foreach ($required_classes as $class_name) {
if ((!isset($dependent_classes[$class_name])) && !isset($class_lists[$parent_class][$class_name])) {
throw new MigrateException(t('Dependency on non-existent class !class - make sure ' .
'you have added the file defining !class to the .info file.',
array('!class' => $class_name)));
}
}
// Scan modules with dependencies - we'll take 20 passes at it before
// giving up
$iterations = 0;
while (count($dependent_classes) > 0) {
if ($iterations++ > 20) {
$class_names = implode(',', array_keys($dependent_classes));
throw new MigrateException(t('Failure to sort class list - most likely due ' .
'to circular dependencies involving !class_names.',
array('!class_names' => $class_names)));
}
foreach ($dependent_classes as $name => $object) {
$ready = TRUE;
// Scan all the dependencies for this class and make sure they're all
// in the final list
foreach ($object->getDependencies() as $dependency) {
if (!isset($class_lists[$parent_class][$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this class to the final list
$class_lists[$parent_class][$name] = $object;
unset($dependent_classes[$name]);
}
}
}
}
return $class_lists[$parent_class];
}
@@ -387,9 +249,26 @@ function migrate_hook_info() {
$hooks['migrate_api'] = array(
'group' => 'migrate',
);
$hooks['migrate_api_alter'] = array(
'group' => 'migrate',
);
return $hooks;
}
/**
* Implementation of hook_permission().
*/
function migrate_permission() {
return array(
MIGRATE_ACCESS_BASIC => array(
'title' => t('Access to basic migration information'),
),
MIGRATE_ACCESS_ADVANCED => array(
'title' => t('Access to advanced migration information'),
),
);
}
/**
* Get a list of modules that support the current migrate API.
*/
@@ -405,13 +284,6 @@ function migrate_get_module_apis($reset = FALSE) {
$info = $function();
if (isset($info['api']) && $info['api'] == MIGRATE_API_VERSION) {
$cache[$module] = $info;
// Register any migrations defined via the hook.
if (isset($info['migrations']) && is_array($info['migrations'])) {
foreach ($info['migrations'] as $machine_name => $arguments) {
MigrationBase::registerMigration($arguments['class_name'],
$machine_name, $arguments);
}
}
}
else {
drupal_set_message(t('%function supports Migrate API version %modversion,
@@ -420,11 +292,97 @@ function migrate_get_module_apis($reset = FALSE) {
'%version' => MIGRATE_API_VERSION)));
}
}
// Allow modules to alter the migration information.
drupal_alter('migrate_api', $cache);
}
return $cache;
}
/**
* Register any migrations defined in hook_migrate_api().
*
* @param array $machine_names
* If populated, only (re)register the specified migrations.
*/
function migrate_static_registration($machine_names = array()) {
$module_info = migrate_get_module_apis(TRUE);
foreach ($module_info as $module => $info) {
// Register any groups defined via the hook.
if (isset($info['groups']) && is_array($info['groups'])) {
foreach ($info['groups'] as $name => $arguments) {
$title = $arguments['title'];
unset($arguments['title']);
MigrateGroup::register($name, $title, $arguments);
}
}
// Register any migrations defined via the hook.
if (isset($info['migrations']) && is_array($info['migrations'])) {
foreach ($info['migrations'] as $machine_name => $arguments) {
// If we have an explicit list to register, skip any not in the list.
if (!empty($machine_names) && !in_array($machine_name, $machine_names)) {
continue;
}
$class_name = $arguments['class_name'];
unset($arguments['class_name']);
// Call the right registerMigration implementation. Note that this means
// that classes that override registerMigration() must always call it
// directly, they cannot register those classes by defining them in
// hook_migrate_api() and expect their extension to be called.
if (is_subclass_of($class_name, 'Migration')) {
Migration::registerMigration($class_name, $machine_name, $arguments);
}
else {
MigrationBase::registerMigration($class_name, $machine_name, $arguments);
}
}
}
}
}
/**
* Do a topological sort on our dependencies graph.
*/
function migrate_order_dependencies($dependencies) {
$visited = array();
$list = array();
foreach (array_keys($dependencies) as $name) {
$visited[$name] = FALSE;
}
foreach (array_keys($dependencies) as $name) {
migrate_visit_dependent($dependencies, $name, $list, $visited);
}
return $list;
}
/**
* Depth-first search for independent migrations.
*/
function migrate_visit_dependent($dependencies, $name, &$list, &$visited) {
if ($visited[$name]) {
if ($list[$name]) {
return;
}
else {
throw new MigrateException(t('Failure to sort migration list due to circular dependencies involving %name.', array('%name' => $name)));
}
}
$visited[$name] = TRUE;
if (isset($dependencies[$name])) {
foreach ($dependencies[$name] as $dependent) {
migrate_visit_dependent($dependencies, $dependent, $list, $visited);
}
}
$list[$name] = $name;
}
/**
* Implements hook_watchdog().
* Find the migration that is currently running and notify it.
@@ -579,3 +537,28 @@ function migrate_overview() {
}
return $overview;
}
/**
* Implements hook_modules_enabled.
*/
function migrate_modules_enabled($modules) {
if (array_intersect($modules, module_implements('migrate_api'))) {
migrate_static_registration();
}
}
/**
* Implements hook_module_implements_alter().
*/
function migrate_module_implements_alter(&$implementation, $hook) {
// Ensure that the Migration class exists, as different bootstrap phases may
// not have included migration.inc yet.
if (class_exists('Migration') && $migration = Migration::currentMigration()) {
$disable_hooks = $migration->getDisableHooks();
if (isset($disable_hooks[$hook])) {
foreach ($disable_hooks[$hook] as $module) {
unset($implementation[$module]);
}
}
}
}

335
sites/all/modules/contrib/migrate/migrate/migrate_example/beer.inc
View File

@@ -16,34 +16,41 @@
* - Comments to be attached to the beer nodes are described in the source
* migrate_example_beer_comment table.
*
* We will use the Migrate API to import and transform this data and turn it into
* a working Drupal system.
* We will use the Migrate API to import and transform this data and turn it
* into a working Drupal site.
*/
/**
* To define a migration process from a set of source data to a particular
* kind of Drupal object (for example, a specific node type), you define
* a class derived from Migration. You must define a constructor to initialize
* your migration object. By default, your class name will be the "machine name"
* of the migration, by which you refer to it. Note that the machine name is
* case-sensitive.
* your migration object.
*
* For your classes to be instantiated so they can be used to import content,
* you must register them - look at migrate_example.migrate.inc to see how
* registration works. Right now, it's important to understand that each
* migration will have a unique "machine name", which is displayed in the UI
* and is used to reference the migration in drush commands.
*
* In any serious migration project, you will find there are some options
* which are common to the individual migrations you're implementing. You can
* define an abstract intermediate class derived from Migration, then derive your
* individual migrations from that, to share settings, utility functions, etc.
*/
abstract class BasicExampleMigration extends DynamicMigration {
public function __construct() {
// Always call the parent constructor first for basic setup
parent::__construct();
abstract class BasicExampleMigration extends Migration {
// A Migration constructor takes an array of arguments as its first parameter.
// The arguments must be passed through to the parent constructor.
public function __construct($arguments) {
parent::__construct($arguments);
// With migrate_ui enabled, migration pages will indicate people involved in
// the particular migration, with their role and contact info. We default the
// list in the shared class; it can be overridden for specific migrations.
$this->team = array(
new MigrateTeamMember('Liz Taster', 'ltaster@example.com', t('Product Owner')),
new MigrateTeamMember('Larry Brewer', 'lbrewer@example.com', t('Implementor')),
new MigrateTeamMember('Liz Taster', 'ltaster@example.com',
t('Product Owner')),
new MigrateTeamMember('Larry Brewer', 'lbrewer@example.com',
t('Implementor')),
);
// Individual mappings in a migration can be linked to a ticket or issue
@@ -62,23 +69,45 @@ abstract class BasicExampleMigration extends DynamicMigration {
* this will receive data that originated from the source and has been mapped
* by the Migration class, and create Drupal objects.
* $this->map - An instance of a class derived from MigrateMap, this will keep
* track of which source items have been imported and what destination objects
* they map to.
* Mappings - Use $this->addFieldMapping to tell the Migration class what source
* fields correspond to what destination fields, and additional information
* associated with the mappings.
* track of which source items have been imported and what destination
* objects they map to.
* Field mappings - Use $this->addFieldMapping to tell the Migration class what
* source fields correspond to what destination fields, and additional
* information associated with the mappings.
*/
class BeerTermMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
// Human-friendly description of your migration process. Be as detailed as you
// like.
$this->description = t('Migrate styles from the source database to taxonomy terms');
public function __construct($arguments) {
parent::__construct($arguments);
// Human-friendly description of your migration process. Be as detailed as
// you like.
$this->description =
t('Migrate styles from the source database to taxonomy terms');
// In this example, we're using tables that have been added to the existing
// Drupal database but which are not Drupal tables. You can examine the
// various tables (starting here with migrate_example_beer_topic) using a
// database browser such as phpMyAdmin.
// First, we set up a query for this data. Note that by ordering on
// style_parent, we guarantee root terms are migrated first, so the
// parent_name mapping below will find that the parent exists.
$query = db_select('migrate_example_beer_topic', 'met')
->fields('met', array('style', 'details', 'style_parent', 'region',
'hoppiness'))
// This sort assures that parents are saved before children.
->orderBy('style_parent', 'ASC');
// Create a MigrateSource object, which manages retrieving the input data.
$this->source = new MigrateSourceSQL($query);
// Set up our destination - terms in the migrate_example_beer_styles
// vocabulary (note that we pass the machine name of the vocabulary).
$this->destination =
new MigrateDestinationTerm('migrate_example_beer_styles');
// Create a map object for tracking the relationships between source rows
// and their resulting Drupal objects. Usually, you'll use the MigrateSQLMap
// class, which uses database tables for tracking. Pass the machine name
// (BeerTerm) of this migration to use in generating map and message tables.
// and their resulting Drupal objects. We will use the MigrateSQLMap class,
// which uses database tables for tracking. Pass the machine name (BeerTerm)
// of this migration to use in generating map and message table names.
// And, pass schema definitions for the primary keys of the source and
// destination - we need to be explicit for our source, but the destination
// class knows its schema already.
@@ -93,28 +122,11 @@ class BeerTermMigration extends BasicExampleMigration {
MigrateDestinationTerm::getKeySchema()
);
// In this example, we're using tables that have been added to the existing
// Drupal database but which are not Drupal tables. You can examine the
// various tables (starting here with migrate_example_beer_topic) using a
// database browser like phpMyAdmin.
// First, we set up a query for this data. Note that by ordering on
// style_parent, we guarantee root terms are migrated first, so the
// parent_name mapping below will find that the parent exists.
$query = db_select('migrate_example_beer_topic', 'met')
->fields('met', array('style', 'details', 'style_parent', 'region', 'hoppiness'))
// This sort assures that parents are saved before children.
->orderBy('style_parent', 'ASC');
// Create a MigrateSource object, which manages retrieving the input data.
$this->source = new MigrateSourceSQL($query);
// Set up our destination - terms in the migrate_example_beer_styles vocabulary
$this->destination = new MigrateDestinationTerm('migrate_example_beer_styles');
// Assign mappings TO destination fields FROM source fields. To discover
// the names used in these calls, use the drush commands
// drush migrate-fields-destination BeerTerm
// drush migrate-fields-source BeerTerm
// drush migrate-fields-destination BeerTerm
// drush migrate-fields-source BeerTerm
// or review the detail pages in the UI.
$this->addFieldMapping('name', 'style');
$this->addFieldMapping('description', 'details');
@@ -127,12 +139,13 @@ class BeerTermMigration extends BasicExampleMigration {
// migration info page when the migrate_ui module is enabled. The default
// is 'Done', indicating active mappings which need no attention. A
// suggested practice is to use groups of:
// Do Not Migrate (or DNM) to indicate source fields which are not being used,
// or destination fields not to be populated by migration.
// Do Not Migrate (or DNM) to indicate source fields which are not being
// used, or destination fields not to be populated by migration.
// Client Issues to indicate input from the client is needed to determine
// how a given field is to be migrated.
// Implementor Issues to indicate that the client has provided all the
// necessary information, and now the implementor needs to complete the work.
// necessary information, and now the implementor needs to complete the
// work.
$this->addFieldMapping(NULL, 'hoppiness')
->description(t('This info will not be maintained in Drupal'))
->issueGroup(t('DNM'));
@@ -141,10 +154,11 @@ class BeerTermMigration extends BasicExampleMigration {
// MigrateFieldMapping::ISSUE_PRIORITY_OK). If you're using an issue
// tracking system, and have defined issuePattern (see BasicExampleMigration
// above), you can specify a ticket/issue number in the system on the
// mapping and migrate_ui will link directory to it.
// mapping and migrate_ui will link directly to it.
$this->addFieldMapping(NULL, 'region')
->description('Will a field be added to the vocabulary for this?')
->issueGroup(t('Client Issues'))
// This priority wil cause the mapping to be highlighted in the UI.
->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_MEDIUM)
->issueNumber(770064);
@@ -152,8 +166,8 @@ class BeerTermMigration extends BasicExampleMigration {
// explicitly - this makes sure that everyone understands exactly what is
// being migrated and what is not. Also, migrate_ui highlights unmapped
// fields, or mappings involving fields not in the source and destination,
// so if (for example) a new field is added to the destination field it's
// immediately visible, and you can find out if anything needs to be
// so if (for example) a new field is added to the destination typ it's
// immediately visible, and you can decide if anything needs to be
// migrated into it.
$this->addFieldMapping('format')
->issueGroup(t('DNM'));
@@ -163,11 +177,12 @@ class BeerTermMigration extends BasicExampleMigration {
->issueGroup(t('DNM'));
// We conditionally DNM these fields, so your field mappings will be clean
// whether or not you have path and or pathauto enabled
if (module_exists('path')) {
// whether or not you have path and/or pathauto enabled.
$destination_fields = $this->destination->fields();
if (isset($destination_fields['path'])) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
if (isset($destination_fields['pathauto'])) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
}
@@ -186,10 +201,15 @@ class BeerTermMigration extends BasicExampleMigration {
* transformations of the data.
*/
class BeerUserMigration extends BasicExampleMigration {
public function __construct() {
public function __construct($arguments) {
// The basic setup is similar to BeerTermMigraiton
parent::__construct();
parent::__construct($arguments);
$this->description = t('Beer Drinkers of the world');
$query = db_select('migrate_example_beer_account', 'mea')
->fields('mea', array('aid', 'status', 'posted', 'name',
'nickname', 'password', 'mail', 'sex', 'beers'));
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationUser();
$this->map = new MigrateSQLMap($this->machineName,
array('aid' => array(
'type' => 'int',
@@ -199,18 +219,15 @@ class BeerUserMigration extends BasicExampleMigration {
),
MigrateDestinationUser::getKeySchema()
);
$query = db_select('migrate_example_beer_account', 'mea')
->fields('mea', array('aid', 'status', 'posted', 'name', 'nickname', 'password', 'mail', 'sex', 'beers'));
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationUser();
// One good way to organize your mappings is in three groups - mapped fields,
// unmapped source fields, and unmapped destination fields
// One good way to organize your mappings in the constructor is in three
// groups - mapped fields, unmapped source fields, and unmapped destination
// fields.
// Mapped fields
// This is a shortcut you can use when the source and destination field
// names are identical (i.e., the email address field is named 'mail' in
// names are identical (e.g., the email address field is named 'mail' in
// both the source table and in Drupal).
$this->addSimpleMappings(array('status', 'mail'));
@@ -224,7 +241,8 @@ class BeerUserMigration extends BasicExampleMigration {
$this->addFieldMapping('name', 'name')
->dedupe('users', 'name');
// The migrate module automatically converts date/time strings to UNIX timestamps.
// The migrate module automatically converts date/time strings to UNIX
// timestamps.
$this->addFieldMapping('created', 'posted');
$this->addFieldMapping('pass', 'password');
@@ -246,6 +264,10 @@ class BeerUserMigration extends BasicExampleMigration {
$this->addFieldMapping('field_migrate_example_favbeers', 'beers')
->separator('|');
}
else {
$this->addFieldMapping(NULL, 'beers')
->issueGroup(t('DNM'));
}
// Unmapped source fields
$this->addFieldMapping(NULL, 'nickname')
@@ -254,10 +276,20 @@ class BeerUserMigration extends BasicExampleMigration {
// Unmapped destination fields
// This is a shortcut you can use to mark several destination fields as DNM
// at once
$this->addUnmigratedDestinations(array('theme', 'signature', 'access', 'login',
'timezone', 'language', 'picture', 'is_new', 'signature_format', 'role_names',
'init', 'data'));
// at once.
$this->addUnmigratedDestinations(array(
'access',
'data',
'is_new',
'language',
'login',
'picture',
'role_names',
'signature',
'signature_format',
'theme',
'timezone',
));
// Oops, we made a typo - this should have been 'init'! If you have
// migrate_ui enabled, look at the BeerUser info page - you'll see that it
@@ -267,10 +299,11 @@ class BeerUserMigration extends BasicExampleMigration {
$this->addFieldMapping('int')
->issueGroup(t('DNM'));
if (module_exists('path')) {
$destination_fields = $this->destination->fields();
if (isset($destination_fields['path'])) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
if (isset($destination_fields['pathauto'])) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
}
@@ -283,35 +316,19 @@ class BeerUserMigration extends BasicExampleMigration {
* and creates Drupal nodes of type 'Beer' as destination.
*/
class BeerNodeMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
public function __construct($arguments) {
parent::__construct($arguments);
$this->description = t('Beers of the world');
// You may optionally declare dependencies for your migration - other migrations
// which should run first. In this case, terms assigned to our nodes and
// the authors of the nodes should be migrated before the nodes themselves.
$this->dependencies = array('BeerTerm', 'BeerUser');
$this->map = new MigrateSQLMap($this->machineName,
array(
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID.',
'alias' => 'b',
)
),
MigrateDestinationNode::getKeySchema()
);
// We have a more complicated query. The Migration class fundamentally
// depends on taking a single source row and turning it into a single
// Drupal object, so how do we deal with zero or more terms attached to
// each node? One way (demonstrated for MySQL) is to pull them into a single
// each node? One way (valid for MySQL only) is to pull them into a single
// comma-separated list.
$query = db_select('migrate_example_beer_node', 'b')
->fields('b', array('bid', 'name', 'body', 'excerpt', 'aid', 'countries',
'image', 'image_alt', 'image_title', 'image_description'));
->fields('b', array('bid', 'name', 'body', 'excerpt', 'aid',
'countries', 'image', 'image_alt', 'image_title',
'image_description'));
$query->leftJoin('migrate_example_beer_topic_node', 'tb', 'b.bid = tb.bid');
// Gives a single comma-separated list of related terms
$query->groupBy('tb.bid');
@@ -331,6 +348,18 @@ class BeerNodeMigration extends BasicExampleMigration {
// Set up our destination - nodes of type migrate_example_beer
$this->destination = new MigrateDestinationNode('migrate_example_beer');
$this->map = new MigrateSQLMap($this->machineName,
array(
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID.',
'alias' => 'b',
)
),
MigrateDestinationNode::getKeySchema()
);
// Mapped fields
$this->addFieldMapping('title', 'name')
->description(t('Mapping beer name in source to node title'));
@@ -340,15 +369,6 @@ class BeerNodeMigration extends BasicExampleMigration {
->issueNumber(765736)
->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_LOW);
// To maintain node identities between the old and new systems (i.e., have
// the same unique IDs), map the ID column from the old system to nid and
// set is_new to TRUE. This works only if we're importing into a system that
// has no existing nodes with the nids being imported.
$this->addFieldMapping('nid', 'bid')
->description(t('Preserve old beer ID as nid in Drupal'));
$this->addFieldMapping('is_new')
->defaultValue(TRUE);
// References to related objects (such as the author of the content) are
// most likely going to be identifiers from the source data, not Drupal
// identifiers (such as uids). You can use the mapping from the relevant
@@ -357,25 +377,37 @@ class BeerNodeMigration extends BasicExampleMigration {
// find a corresponding uid for the aid, the owner will be the administrative
// account.
$this->addFieldMapping('uid', 'aid')
// Note this is the machine name of the user migration.
->sourceMigration('BeerUser')
->defaultValue(1);
// This is a multi-value text field
// This is a multi-value text field - in the source data the values are
// separated by |, so we tell migrate to split it by that character.
$this->addFieldMapping('field_migrate_example_country', 'countries')
->separator('|');
// These are related terms, which by default will be looked up by name
// These are related terms, which by default will be looked up by name.
$this->addFieldMapping('migrate_example_beer_styles', 'terms')
->separator(',');
// Some fields may have subfields such as text formats or summaries
// (equivalent to teasers in previous Drupal versions).
// These can be individually mapped as we see here.
// Some fields may have subfields such as text formats or summaries. These
// can be individually mapped as we see here.
$this->addFieldMapping('body', 'body');
$this->addFieldMapping('body:summary', 'excerpt');
// Copy an image file, write DB record to files table, and save in Field storage.
// We map the filename (relative to the source_dir below) to the field itself.
// File fields are more complex - the file needs to be copied, a Drupal
// file entity (file_managed table row) created, and the field populated.
// There are several different options involved. It's usually best to do
// migrate the files themselves in their own migration (see wine.inc for an
// example), but they can also be brought over through the field mapping.
// We map the filename (relative to the source_dir below) to the field
// itself.
$this->addFieldMapping('field_migrate_example_image', 'image');
// The file_class determines how the 'image' value is interpreted, and what
// other options are available. In this case, MigrateFileUri indicates that
// the 'image' value is a URI.
$this->addFieldMapping('field_migrate_example_image:file_class')
->defaultValue('MigrateFileUri');
// Here we specify the directory containing the source files.
$this->addFieldMapping('field_migrate_example_image:source_dir')
->defaultValue(drupal_get_path('module', 'migrate_example'));
@@ -387,24 +419,46 @@ class BeerNodeMigration extends BasicExampleMigration {
$this->addUnmigratedSources(array('image_description'));
// Unmapped destination fields
$this->addUnmigratedDestinations(array('created', 'changed', 'status',
'promote', 'revision', 'language', 'revision_uid', 'log', 'tnid',
'body:format', 'body:language', 'migrate_example_beer_styles:source_type',
'migrate_example_beer_styles:create_term', 'field_migrate_example_image:destination_dir',
'field_migrate_example_image:language', 'field_migrate_example_image:file_replace',
'field_migrate_example_image:preserve_files', 'field_migrate_example_country:language', 'comment',
'field_migrate_example_image:file_class', 'field_migrate_example_image:destination_file'));
// Some conventions we use here: with a long list of fields to ignore, we
// arrange them alphabetically, one distinct field per line (although
// subfields of the same field may be grouped on the same line), and indent
// subfields to distinguish them from top-level fields.
$this->addUnmigratedDestinations(array(
'body:format', 'body:language',
'changed',
'comment',
'created',
'field_migrate_example_country:language',
'field_migrate_example_image:destination_dir',
'field_migrate_example_image:destination_file',
'field_migrate_example_image:file_replace',
'field_migrate_example_image:language',
'field_migrate_example_image:preserve_files',
'field_migrate_example_image:urlencode',
'is_new',
'language',
'log',
'migrate_example_beer_styles:source_type',
'migrate_example_beer_styles:create_term',
'promote',
'revision',
'revision_uid',
'status',
'tnid',
));
if (module_exists('path')) {
$destination_fields = $this->destination->fields();
if (isset($destination_fields['path'])) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
if (isset($destination_fields['pathauto'])) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
}
}
if (module_exists('statistics')) {
$this->addUnmigratedDestinations(array('totalcount', 'daycount', 'timestamp'));
$this->addUnmigratedDestinations(
array('totalcount', 'daycount', 'timestamp'));
}
}
}
@@ -414,10 +468,21 @@ class BeerNodeMigration extends BasicExampleMigration {
* Drupal comment objects.
*/
class BeerCommentMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
public function __construct($arguments) {
parent::__construct($arguments);
$this->description = 'Comments about beers';
$this->dependencies = array('BeerUser', 'BeerNode');
$query = db_select('migrate_example_beer_comment', 'mec')
->fields('mec', array('cid', 'cid_parent', 'name', 'mail', 'aid',
'body', 'bid', 'subject'))
->orderBy('cid_parent', 'ASC');
$this->source = new MigrateSourceSQL($query);
// Note that the machine name passed for comment migrations is
// 'comment_node_' followed by the machine name of the node type these
// comments are attached to.
$this->destination =
new MigrateDestinationComment('comment_node_migrate_example_beer');
$this->map = new MigrateSQLMap($this->machineName,
array('cid' => array(
'type' => 'int',
@@ -426,19 +491,14 @@ class BeerCommentMigration extends BasicExampleMigration {
),
MigrateDestinationComment::getKeySchema()
);
$query = db_select('migrate_example_beer_comment', 'mec')
->fields('mec', array('cid', 'cid_parent', 'name', 'mail', 'aid', 'body', 'bid', 'subject'))
->orderBy('cid_parent', 'ASC');
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationComment('comment_node_migrate_example_beer');
// Mapped fields
$this->addSimpleMappings(array('name', 'subject', 'mail'));
$this->addFieldMapping('status')
->defaultValue(COMMENT_PUBLISHED);
// We preserved bid => nid ids during BeerNode import so simple mapping suffices.
$this->addFieldMapping('nid', 'bid');
$this->addFieldMapping('nid', 'bid')
->sourceMigration('BeerNode');
$this->addFieldMapping('uid', 'aid')
->sourceMigration('BeerUser')
@@ -453,7 +513,24 @@ class BeerCommentMigration extends BasicExampleMigration {
// No unmapped source fields
// Unmapped destination fields
$this->addUnmigratedDestinations(array('hostname', 'created', 'changed',
'thread', 'homepage', 'language', 'comment_body:format', 'comment_body:language'));
$this->addUnmigratedDestinations(array(
'changed',
'comment_body:format', 'comment_body:language',
'created',
'homepage',
'hostname',
'language',
'thread',
));
$destination_fields = $this->destination->fields();
if (isset($destination_fields['path'])) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (isset($destination_fields['pathauto'])) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
}
}
}
}

5
sites/all/modules/contrib/migrate/migrate/migrate_example/beer.install.inc
View File

@@ -47,10 +47,7 @@ function migrate_example_beer_uninstall() {
}
function migrate_example_beer_disable() {
Migration::deregisterMigration('BeerTerm');
Migration::deregisterMigration('BeerUser');
Migration::deregisterMigration('BeerNode');
Migration::deregisterMigration('BeerComment');
MigrateGroup::deregister('beer');
}
function migrate_example_beer_schema_node() {

9
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example.info
View File

@@ -1,6 +1,6 @@
name = "Migrate Example"
description = "Example migration data."
package = "Development"
package = "Migration"
core = 7.x
dependencies[] = taxonomy
dependencies[] = image
@@ -12,16 +12,15 @@ dependencies[] = number
;node_reference is useful but not required
;dependencies[] = node_reference
files[] = migrate_example.module
files[] = beer.inc
files[] = wine.inc
; For testing table_copy plugin. Since is infrequently used, we comment it out.
; files[] = example.table_copy.inc
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2015-02-09
version = "7.x-2.7"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1423521491"

1
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example.install
View File

@@ -34,6 +34,7 @@ function migrate_example_install() {
);
$example_format = (object) $example_format;
filter_format_save($example_format);
migrate_static_registration();
}
function migrate_example_uninstall() {

255
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example.migrate.inc
View File

@@ -9,39 +9,238 @@
/*
* You must implement hook_migrate_api(), setting the API level to 2, if you are
* implementing any migration classes. As of Migrate 2.5, you should also
* register your migration and handler classes explicitly here - the former
* method of letting them get automatically registered on a cache clear will
* break in certain environments (see http://drupal.org/node/1778952).
* implementing any migration classes. If your migration application is static -
* that is, you know at implementation time exactly what migrations must be
* instantiated - then you should register your migrations here. If your
* application is more dynamic (for example, if selections in the UI determine
* exactly what migrations need to be instantiated), then you would register
* your migrations using registerMigration() - see migrate_example_baseball for
* more information.
*/
function migrate_example_migrate_api() {
// Usually field mappings are established by code in the migration constructor -
// a call to addFieldMapping(). They may also be passed as arguments when
// registering a migration - in this case, they are stored in the database
// and override any mappings for the same field in the code. To do this,
// construct the field mapping object and configure it similarly to when
// you call addFieldMapping, and pass your mappings as an array below.
$translate_mapping = new MigrateFieldMapping('translate', NULL);
$translate_mapping->defaultValue(0);
$ignore_mapping = new MigrateFieldMapping('migrate_example_beer_styles:ignore_case', NULL);
$ignore_mapping->defaultValue(1);
$api = array(
// Required - tells the Migrate module that you are implementing version 2
// of the Migrate API.
'api' => 2,
// Migrations can be organized into groups. The key used here will be the
// machine name of the group, which can be used in Drush:
// drush migrate-import --group=wine
// The title is a required argument which is displayed for the group in the
// UI. You may also have additional arguments for any other data which is
// common to all migrations in the group.
'groups' => array(
'beer' => array(
'title' => t('Beer Imports'),
),
'wine' => array(
'title' => t('Wine Imports'),
),
),
// Here we register the individual migrations. The keys (BeerTerm, BeerUser,
// etc.) are the machine names of the migrations, and the class_name
// argument is required. The group_name is optional (defaulting to 'default')
// but specifying it is a best practice.
'migrations' => array(
'BeerTerm' => array('class_name' => 'BeerTermMigration'),
'BeerUser' => array('class_name' => 'BeerUserMigration'),
'BeerNode' => array('class_name' => 'BeerNodeMigration'),
'BeerComment' => array('class_name' => 'BeerCommentMigration'),
'WinePrep' => array('class_name' => 'WinePrepMigration'),
'WineVariety' => array('class_name' => 'WineVarietyMigration'),
'WineRegion' => array('class_name' => 'WineRegionMigration'),
'WineBestWith' => array('class_name' => 'WineBestWithMigration'),
'WineFileCopy' => array('class_name' => 'WineFileCopyMigration'),
'WineFileBlob' => array('class_name' => 'WineFileBlobMigration'),
'WineRole' => array('class_name' => 'WineRoleMigration'),
'WineUser' => array('class_name' => 'WineUserMigration'),
'WineProducer' => array('class_name' => 'WineProducerMigration'),
'WineProducerXML' => array('class_name' => 'WineProducerXMLMigration'),
'WineProducerMultiXML' => array('class_name' => 'WineProducerMultiXMLMigration'),
'WineProducerXMLPull' => array('class_name' => 'WineProducerXMLPullMigration'),
'WineWine' => array('class_name' => 'WineWineMigration'),
'WineComment' => array('class_name' => 'WineCommentMigration'),
'WineTable' => array('class_name' => 'WineTableMigration'),
'WineFinish' => array('class_name' => 'WineFinishMigration'),
'WineUpdates' => array('class_name' => 'WineUpdatesMigration'),
'WineCommentUpdates' => array('class_name' => 'WineCommentUpdatesMigration'),
'WineVarietyUpdates' => array('class_name' => 'WineVarietyUpdatesMigration'),
'WineUserUpdates' => array('class_name' => 'WineUserUpdatesMigration'),
'BeerTerm' => array(
'class_name' => 'BeerTermMigration',
'group_name' => 'beer',
),
'BeerUser' => array(
'class_name' => 'BeerUserMigration',
'group_name' => 'beer',
),
'BeerNode' => array(
'class_name' => 'BeerNodeMigration',
'group_name' => 'beer',
// You may optionally declare dependencies for your migration - other
// migrations which should run first. In this case, terms assigned to our
// nodes and the authors of the nodes should be migrated before the nodes
// themselves.
'dependencies' => array(
'BeerTerm',
'BeerUser',
),
// Here is where we add field mappings which may override those
// specified in the group constructor.
'field_mappings' => array(
$translate_mapping,
$ignore_mapping,
),
),
'BeerComment' => array(
'class_name' => 'BeerCommentMigration',
'group_name' => 'beer',
'dependencies' => array(
'BeerUser',
'BeerNode',
),
),
'WinePrep' => array(
'class_name' => 'WinePrepMigration',
'group_name' => 'wine',
),
'WineVariety' => array(
'class_name' => 'WineVarietyMigration',
'group_name' => 'wine',
),
'WineRegion' => array(
'class_name' => 'WineRegionMigration',
'group_name' => 'wine',
),
'WineBestWith' => array(
'class_name' => 'WineBestWithMigration',
'group_name' => 'wine',
),
'WineFileCopy' => array(
'class_name' => 'WineFileCopyMigration',
'group_name' => 'wine',
'dependencies' => array('WinePrep'),
),
'WineFileBlob' => array(
'class_name' => 'WineFileBlobMigration',
'group_name' => 'wine',
'dependencies' => array('WinePrep'),
),
'WineRole' => array(
'class_name' => 'WineRoleMigration',
'group_name' => 'wine',
// TIP: Regular dependencies, besides enforcing (in the absence of
// --force) the run order of migrations, affect the sorting of
// migrations on display. You can use soft dependencies to affect just
// the display order when the migrations aren't technically required to
// run in a certain order. In this case, we want the role migration to
// appear after the file migrations.
'soft_dependencies' => array('WineFileCopy'),
),
'WineUser' => array(
'class_name' => 'WineUserMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineFileCopy',
'WineRole',
),
),
'WineProducer' => array(
'class_name' => 'WineProducerMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerXML' => array(
'class_name' => 'WineProducerXMLMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerNamespaceXML' => array(
'class_name' => 'WineProducerNamespaceXMLMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerMultiXML' => array(
'class_name' => 'WineProducerMultiXMLMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerMultiNamespaceXML' => array(
'class_name' => 'WineProducerMultiNamespaceXMLMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerXMLPull' => array(
'class_name' => 'WineProducerXMLPullMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineProducerNamespaceXMLPull' => array(
'class_name' => 'WineProducerNamespaceXMLPullMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineUser',
),
),
'WineWine' => array(
'class_name' => 'WineWineMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineRegion',
'WineVariety',
'WineBestWith',
'WineUser',
'WineProducer',
),
),
'WineComment' => array(
'class_name' => 'WineCommentMigration',
'group_name' => 'wine',
'dependencies' => array(
'WineUser',
'WineWine',
),
),
'WineTable' => array(
'class_name' => 'WineTableMigration',
'group_name' => 'wine',
'soft_dependencies' => array('WineComment'),
),
'WineFinish' => array(
'class_name' => 'WineFinishMigration',
'group_name' => 'wine',
'dependencies' => array('WineComment'),
),
'WineUpdates' => array(
'class_name' => 'WineUpdatesMigration',
'group_name' => 'wine',
'dependencies' => array('WineWine'),
'soft_dependencies' => array('WineFinish'),
),
'WineCommentUpdates' => array(
'class_name' => 'WineCommentUpdatesMigration',
'group_name' => 'wine',
'dependencies' => array('WineComment'),
'soft_dependencies' => array('WineUpdates'),
),
'WineVarietyUpdates' => array(
'class_name' => 'WineVarietyUpdatesMigration',
'group_name' => 'wine',
'dependencies' => array('WineVariety'),
'soft_dependencies' => array('WineUpdates'),
),
'WineUserUpdates' => array(
'class_name' => 'WineUserUpdatesMigration',
'group_name' => 'wine',
'dependencies' => array('WineUser'),
'soft_dependencies' => array('WineUpdates'),
),
),
);
return $api;

8
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example.module
View File

@@ -2,13 +2,13 @@
/**
* @file
* THIS SPACE INTENTIONALLY LEFT BLANK.
* THIS FILE INTENTIONALLY LEFT BLANK.
*
* Yes, there is no code in the .module file. Migrate operates almost entirely
* through classes, and by adding any files containing class definitions to the
* .info file, those files are automatically included only when the classes they
* contain are referenced. The one non-class piece you need to implement is
* hook_migrate_api(), but because .migrate.inc is registered using hook_hook_info
* by defining your implementation of that hook in mymodule.migrate.inc, it is
* automatically invoked only when needed.
* hook_migrate_api(), but because .migrate.inc is registered using
* hook_hook_info, by defining your implementation of that hook in
* example.migrate.inc, it is automatically invoked only when needed.
*/

8
sites/all/modules/contrib/migrate/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.info
View File

@@ -8,12 +8,12 @@ features[field][] = "node-migrate_example_oracle-field_mainimage"
features[node][] = "migrate_example_oracle"
files[] = "migrate_example_oracle.migrate.inc"
name = "Migrate example - Oracle"
package = "Migrate Examples"
package = "Migration"
project = "migrate_example_oracle"
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2015-02-09
version = "7.x-2.7"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1423521491"

1500
sites/all/modules/contrib/migrate/migrate/migrate_example/wine.inc
View File

File diff suppressed because it is too large Load Diff

23
sites/all/modules/contrib/migrate/migrate/migrate_example/wine.install.inc
View File

@@ -65,26 +65,7 @@ function migrate_example_wine_uninstall() {
}
function migrate_example_wine_disable() {
MigrationBase::deregisterMigration('WinePrep');
Migration::deregisterMigration('WineFileCopy');
Migration::deregisterMigration('WineFileBlob');
Migration::deregisterMigration('WineRegion');
Migration::deregisterMigration('WineUser');
Migration::deregisterMigration('WineVariety');
Migration::deregisterMigration('WineBestWith');
Migration::deregisterMigration('WineProducer');
Migration::deregisterMigration('WineProducerXML');
Migration::deregisterMigration('WineProducerXMLPull');
Migration::deregisterMigration('WineProducerMultiXML');
Migration::deregisterMigration('WineWine');
Migration::deregisterMigration('WineComment');
MigrationBase::deregisterMigration('WineFinish');
Migration::deregisterMigration('WineUpdates');
Migration::deregisterMigration('WineUserUpdates');
Migration::deregisterMigration('WineVarietyUpdates');
Migration::deregisterMigration('WineCommentUpdates');
Migration::deregisterMigration('WineRole');
Migration::deregisterMigration('WineTable');
MigrateGroup::deregister('wine');
}
function migrate_example_wine_schema_wine() {
@@ -1212,7 +1193,7 @@ function migrate_example_wine_data_files() {
$query = db_insert('migrate_example_wine_files')
->fields($fields);
$data = array(
array(1, 'http://drupal.org/sites/all/modules/drupalorg/drupalorg/images/association-individual.png', NULL, NULL, NULL),
array(1, 'http://placekitten.com/200/200', NULL, NULL, NULL),
array(2, 'http://cyrve.com/files/penguin.jpeg', 'Penguin alt', 'Penguin title', 1),
array(3, 'http://cyrve.com/files/rioja.jpeg', 'Rioja alt', 'Rioja title', 2),
array(4, 'http://cyrve.com/files/boutisse_0.jpeg', 'Boutisse alt', 'Boutisse title', 2),

7
sites/all/modules/contrib/migrate/migrate/migrate_example/xml/0002.xml Normal file
View File

@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<pr:producer xmlns:pr="http://www.wine.org/wine-producers">
<pr:name>Château Latour</pr:name>
<pr:description>Makers of grand vin Chateau Latour, Les Forts de Latour and Pauillac</pr:description>
<pr:authorid>3</pr:authorid>
<pr:region>Bordeaux</pr:region>
</pr:producer>

4
sites/all/modules/contrib/migrate/migrate/migrate_example/xml/index2.xml Normal file
View File

@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
<wn:content xmlns:wn="http://www.wine.org/wine">
<wn:sourceid>0002</wn:sourceid>
</wn:content>

17
sites/all/modules/contrib/migrate/migrate/migrate_example/xml/producers3.xml Normal file
View File

@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<pr:producers xmlns:pr="http://www.wine.org/wine-producers">
<pr:producer>
<pr:sourceid>0009</pr:sourceid>
<pr:name>Château Feytit Clinet</pr:name>
<pr:description>Good things, they say, come in small packages; this is certainly the case at Château Feyit Clinet, the maker of thelabel Pomerol.</pr:description>
<pr:authorid>1</pr:authorid>
<pr:region>Pomerol</pr:region>
</pr:producer>
<pr:producer>
<pr:sourceid>0010</pr:sourceid>
<pr:name>Château Doisy-Védrines</pr:name>
<pr:description>Blessed with ancient vines, the fruit here is often subject to high levels of Botrytis (or noble rot), which shrinks the grapes and concentrates sugar levels in the remaining juice.</pr:description>
<pr:authorid>3</pr:authorid>
<pr:region>Barsac</pr:region>
</pr:producer>
</pr:producers>

17
sites/all/modules/contrib/migrate/migrate/migrate_example/xml/producers4.xml Normal file
View File

@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<pr:producers xmlns:pr="http://www.wine.org/wine-producers">
<pr:producer>
<pr:sourceid>0009</pr:sourceid>
<pr:name>Château Bourgneuf</pr:name>
<pr:description>Showing attractive red and dark fruits, cherry and plum, it is rich and even slightly racy, before concluding with classic Pomerol concentration and focus.</pr:description>
<pr:authorid>3</pr:authorid>
<pr:region>Pomerol</pr:region>
</pr:producer>
<pr:producer>
<pr:sourceid>0010</pr:sourceid>
<pr:name>Château Doisy-Daëne</pr:name>
<pr:description>Medium bodied, with elegance rather than density there is a wide spectrum of flavours; apples, peaches, lemongrass and a touch of white spice. Delicious now.</pr:description>
<pr:authorid>9</pr:authorid>
<pr:region>Barsac</pr:region>
</pr:producer>
</pr:producers>

2
sites/all/modules/contrib/migrate/migrate/migrate_example_baseball/migrate_example_baseball.features.inc
View File

@@ -6,7 +6,7 @@
function migrate_example_baseball_node_info() {
$items = array(
'migrate_example_baseball' => array(
'name' => t('migrate_example_baseball'),
'name' => t('Migrate example - Baseball'),
'base' => 'node_content',
'description' => t('A baseball box score'),
'has_title' => '1',

8
sites/all/modules/contrib/migrate/migrate/migrate_example_baseball/migrate_example_baseball.info
View File

@@ -21,12 +21,12 @@ features[field][] = "node-migrate_example_baseball-field_visiting_team"
features[node][] = "migrate_example_baseball"
files[] = "migrate_example_baseball.migrate.inc"
name = "migrate_example_baseball"
package = "Migrate Examples"
package = "Migration"
php = "5.2.4"
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2015-02-09
version = "7.x-2.7"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1423521491"

13
sites/all/modules/contrib/migrate/migrate/migrate_example_baseball/migrate_example_baseball.install
View File

@@ -9,10 +9,10 @@ function migrate_example_baseball_enable() {
$path = dirname(__FILE__) . '/data';
migrate_example_baseball_get_files($path);
for ($i=0; $i<=9; $i++) {
$file = 'gl200' . $i . '.txt';
$file = 'GL200' . $i . '.TXT';
Migration::registerMigration('GameBaseball',
pathinfo($file, PATHINFO_FILENAME),
array('source_file' => $path . '/' . $file));
array('source_file' => $path . '/' . $file, 'group_name' => 'baseball'));
}
}
@@ -53,14 +53,7 @@ function migrate_example_baseball_uninstall() {
}
function migrate_example_baseball_disable() {
for ($i=0; $i<=9; $i++) {
$file = 'gl200' . $i . '.txt';
Migration::deregisterMigration(pathinfo($file, PATHINFO_FILENAME));
$filename = dirname(__FILE__) . '/data/' . $file;
if (file_exists($filename)) {
unlink($filename);
}
}
MigrateGroup::deregister('baseball');
}
/**

22
sites/all/modules/contrib/migrate/migrate/migrate_example_baseball/migrate_example_baseball.migrate.inc
View File

@@ -13,17 +13,21 @@
function migrate_example_baseball_migrate_api() {
$api = array(
'api' => 2,
'groups' => array(
'baseball' => array(
'title' => t('Baseball'),
),
),
);
return $api;
}
/**
* A dynamic migration that is reused for each source CSV file.
* A migration that is reused for each source CSV file.
*/
class GameBaseball extends DynamicMigration {
public function __construct(array $arguments) {
$this->arguments = $arguments;
parent::__construct();
class GameBaseball extends Migration {
public function __construct($arguments) {
parent::__construct($arguments);
$this->description = t('Import box scores from CSV file.');
// Create a map object for tracking the relationships between source rows
@@ -81,7 +85,7 @@ class GameBaseball extends DynamicMigration {
}
}
function csvcolumns() {
protected function csvcolumns() {
// Note: Remember to subtract 1 from column number at http://www.retrosheet.org/gamelogs/glfields.txt
$columns[0] = array('start_date', 'Date of game');
$columns[3] = array('visiting_team', 'Visiting team');
@@ -102,7 +106,7 @@ class GameBaseball extends DynamicMigration {
return $columns;
}
function prepareRow($row) {
public function prepareRow($row) {
// Collect all the batters into one multi-value field.
for ($i=1; $i <= 9; $i++ ) {
$key = "visiting_batter_$i";
@@ -123,11 +127,11 @@ class GameBaseball extends DynamicMigration {
PATHINFO_FILENAME));
}
function fields() {
public function fields() {
return array(
'title' => 'A composite field made during prepareRow().',
'home_batters' => 'An array of batters, populated during prepareRow().',
'visiting_batters' => 'An array of batters, populated during prepareRow().',
);
}
}
}

12
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.info
View File

@@ -1,14 +1,14 @@
name = "Migrate UI"
description = "UI for managing migration processes"
package = "Development"
;configure = admin/config/development/migrate
package = "Migration"
configure = admin/content/migrate/configure
core = 7.x
dependencies[] = migrate
files[] = migrate_ui.module
files[] = migrate_ui.wizard.inc
; Information added by drupal.org packaging script on 2012-11-07
version = "7.x-2.5"
; Information added by Drupal.org packaging script on 2015-02-09
version = "7.x-2.7"
core = "7.x"
project = "migrate"
datestamp = "1352299007"
datestamp = "1423521491"

71
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.install Normal file
View File

@@ -0,0 +1,71 @@
<?php
/**
* @file
* Install/update function for migrate_ui.
*/
/**
* Implements hook_install().
*/
function migrate_ui_install() {
migrate_ui_set_weight();
}
/**
* Implements hook_uninstall().
*/
function migrate_ui_uninstall() {
variable_del('migrate_import_method');
variable_del('migrate_drush_path');
variable_del('migrate_drush_mail');
variable_del('migrate_drush_mail_subject');
variable_del('migrate_drush_mail_body');
}
/**
* Make sure we have a higher weight than node.
*/
function migrate_ui_update_7201() {
migrate_ui_set_weight();
}
/**
* Sets the weight of migrate_ui higher than node, so Import links come after
* "Add content" at admin/content.
*/
function migrate_ui_set_weight() {
$node_weight = db_select('system', 's')
->fields('s', array('weight'))
->condition('name', 'node')
->execute()
->fetchField();
db_update('system')
->fields(array('weight' => $node_weight + 1))
->condition('name', 'migrate_ui')
->execute();
}
/**
* If WordPress Migrate has background imports via drush enabled, copy the
* configuration to the new general Migrate support.
*/
function migrate_ui_update_7202() {
$drush_command = variable_get('wordpress_migrate_drush', '');
if ($drush_command) {
variable_set('migrate_drush_path', $drush_command);
// Consolidate these two variables into import method - 0 means immediate
// only, 1 means drush only, 2 means offer both options.
$import_method = variable_get('wordpress_migrate_import_method', 0);
$force_drush = variable_get('wordpress_migrate_force_drush', FALSE);
if (!$force_drush) {
$import_method = 2;
}
variable_set('migrate_import_method', $import_method);
variable_set('migrate_drush_mail',
variable_get('wordpress_migrate_notification', 0));
variable_set('migrate_drush_mail_subject',
variable_get('wordpress_migrate_notification_subject', ''));
variable_set('migrate_drush_mail_body',
variable_get('wordpress_migrate_notification_body', ''));
}
}

215
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.module
View File

@@ -1,25 +1,12 @@
<?php
define('MIGRATE_ACCESS_BASIC', 'migration information');
function migrate_ui_help($path, $arg) {
switch ($path) {
case 'admin/migrate':
return t('The current status of each migration defined in this system. Click on a migration name for details on its configuration.');
case 'admin/content/migrate':
return t('The current status of each migration group defined in this system. Click on a group name for details on its configuration.');
}
}
/**
* Implementation of hook_permission().
*/
function migrate_ui_permission() {
return array(
MIGRATE_ACCESS_BASIC => array(
'title' => t('Basic access to migration information'),
),
);
}
/**
* Implementation of hook_menu().
*/
@@ -27,67 +14,171 @@ function migrate_ui_menu() {
$items = array();
$items['admin/content/migrate'] = array(
'title' => 'Migrate',
'type' => MENU_LOCAL_TASK | MENU_NORMAL_ITEM,
'description' => 'Monitor the creation of Drupal content from source data',
'page callback' => 'migrate_ui_dashboard',
'title' => 'Migrate',
'description' => 'Manage importing of data into your Drupal site',
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_ui_migrate_dashboard'),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'access callback' => 'user_access',
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/dashboard'] = array(
'title' => 'Migrate',
$items['admin/content/migrate/groups'] = array(
'title' => 'Dashboard',
'type' => MENU_DEFAULT_LOCAL_TASK,
'weight' => -10,
);
$items['admin/content/migrate/registration'] = array(
'title' => 'Registration',
$items['admin/content/migrate/configure'] = array(
'title' => 'Configuration',
'type' => MENU_LOCAL_TASK,
'description' => 'Configure class registration',
'page callback' => 'migrate_ui_registration',
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
'weight' => 5,
);
$items['admin/content/migrate/handlers'] = array(
'title' => 'Handlers',
'type' => MENU_LOCAL_TASK,
'description' => 'Configure migration handlers',
'page callback' => 'migrate_ui_handlers',
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
'weight' => 10,
);
$items['admin/content/migrate/messages/%migration'] = array(
'title callback' => '_migrate_ui_title',
'title arguments' => array(4),
'description' => 'View messages from a migration',
'page callback' => 'migrate_ui_messages',
'page arguments' => array(4),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/%migration'] = array(
'title callback' => '_migrate_ui_title',
'title arguments' => array(3),
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_migration_info', 3),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'description' => 'Configure migration settings',
'page callback' => 'migrate_ui_configure',
'access arguments' => array(MIGRATE_ACCESS_ADVANCED),
'file' => 'migrate_ui.pages.inc',
'weight' => 100,
);
// Add tabs for each implemented migration wizard.
$wizards = migrate_ui_wizards();
foreach ($wizards as $wizard_class => $wizard) {
$items["admin/content/migrate/new/$wizard_class"] = array(
'type' => MENU_LOCAL_TASK,
'title' => 'Import from @source_title',
'title arguments' => array('@source_title' => $wizard->getSourceName()),
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_ui_wizard', $wizard_class),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.wizard.inc',
);
}
$items['admin/content/migrate/groups/%'] =
array(
'title callback' => 'migrate_ui_migrate_group_title',
'title arguments' => array(4),
'type' => MENU_NORMAL_ITEM,
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_ui_migrate_group', 4),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/groups/%/%'] =
array(
'title callback' => 'migrate_ui_migrate_migration_title',
'title arguments' => array(5),
'type' => MENU_NORMAL_ITEM,
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_migration_info', 4, 5),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/groups/%/%/view'] =
array(
'title' => 'View',
'type' => MENU_DEFAULT_LOCAL_TASK,
'weight' => -10,
);
$items['admin/content/migrate/groups/%/%/edit'] =
array(
'type' => MENU_LOCAL_TASK,
'title' => 'Edit',
'description' => 'Edit migration mappings',
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_ui_edit_mappings', 4, 5),
'access arguments' => array(MIGRATE_ACCESS_ADVANCED),
'file' => 'migrate_ui.pages.inc',
);
$items['admin/content/migrate/groups/%/%/messages'] =
array(
'type' => MENU_LOCAL_TASK,
'title' => 'Messages',
'description' => 'View messages from a migration',
'page callback' => 'migrate_ui_messages',
'page arguments' => array(4, 5),
'access arguments' => array(MIGRATE_ACCESS_ADVANCED),
'file' => 'migrate_ui.pages.inc',
);
return $items;
}
// A menu load callback.
function migration_load($machine_name) {
if ($machine_name) {
return Migration::getInstance($machine_name);
}
/**
* Title callback for the migrate group view page.
*/
function migrate_ui_migrate_group_title($group_name) {
$group = MigrateGroup::getInstance($group_name);
return $group->getTitle();
}
function _migrate_ui_title($migration) {
if (is_string($migration)) {
$migration = migration_load($migration);
}
return $migration->getMachineName();
/**
* Title callback for the migration view page.
*/
function migrate_ui_migrate_migration_title($migration_name) {
return $migration_name;
}
/**
* Implements hook_theme()
*
* @return array
*/
function migrate_ui_theme() {
return array(
'migrate_ui_field_mapping_form' => array(
'arguments' => array('field_mappings' => NULL),
'render element' => 'field_mappings',
'file' => '/migrate_ui.pages.inc',
),
'migrate_ui_field_mapping_dependencies' => array(
'arguments' => array('dependencies' => NULL),
'render element' => 'dependencies',
'file' => '/migrate_ui.pages.inc',
),
);
}
/**
* Implementation of hook_mail().
*
* @param $key
* @param $message
* @param $params
*/
function migrate_ui_mail($key, &$message, $params) {
$options['language'] = $message['language'];
user_mail_tokens($variables, array(), $options);
$langcode = $message['language']->language;
$subject = variable_get('migrate_drush_mail_subject', '');
$message['subject'] = t($subject, array(), array('langcode' => $langcode));
$body = variable_get('migrate_drush_mail_body', '');
$body .= "\n" . $params['output'];
$message['body'][] = t($body, array(), array('langcode' => $langcode));
}
/**
* Get info on all modules supporting a migration wizard.
*
* @return array
* key: machine_name for a particular wizard implementation. Used in the menu
* link.
* value: Wizard configuration array containing:
* source_title -
*/
function migrate_ui_wizards() {
$module_apis = migrate_get_module_apis();
$wizards = array();
foreach ($module_apis as $info) {
if (isset($info['wizard classes']) && is_array($info['wizard classes'])) {
foreach ($info['wizard classes'] as $wizard_class) {
$wizard_class = strtolower($wizard_class);
$wizards[$wizard_class] = new $wizard_class;
}
}
}
return $wizards;
}

1493
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.pages.inc
View File

File diff suppressed because it is too large Load Diff

663
sites/all/modules/contrib/migrate/migrate/migrate_ui/migrate_ui.wizard.inc Normal file
View File

@@ -0,0 +1,663 @@
<?php
/**
* @file
* Migration wizard framework.
*/
/**
* The primary formbuilder function for the wizard form.
*
* This form has two defined submit handlers to process the different steps:
* - Previous: handles the way to get back one step in the wizard.
* - Next: handles each step form submission,
*
* The third handler, the finish button handler, is the default form _submit
* handler used to process the information.
*
* @param string $class_name
* Name of the MigrateUIWizard clsas for this wizard.
*/
function migrate_ui_wizard($form, &$form_state, $class_name = '') {
// Rather than track state in $form_state, we simply keep our wizard
// instance there, and it encapsulates all the state. We just need
// to create the instance the first time in, and it will be serialized
// between steps.
/** @var MigrateUIWizard $wizard */
if (empty($form_state['wizard'])) {
$wizard = new $class_name();
// Add any extenders.
$module_apis = migrate_get_module_apis();
// Need a second pass at this to add wizard extenders.
foreach ($module_apis as $module => $info) {
// Add any extenders.
// @todo: consider allowing extender classes to declare dependencies on
// other extender classes, to ensure they work in the correct order?
if (isset($info['wizard extenders'])) {
foreach ($info['wizard extenders'] as $wizard_class => $extender_classes) {
// Note that $class_name is in lower case, so we can't just use isset()
// to find our wizard.
if (strtolower($wizard_class) == $class_name) {
foreach ($extender_classes as $extender_class) {
$wizard->addExtender($extender_class);
}
}
}
}
}
$form_state['wizard'] = $wizard;
}
else {
$wizard = $form_state['wizard'];
}
// Fetch the form for the wizard's current step.
$form = $wizard->form($form_state);
return $form;
}
/**
* Submit handler for the "previous" button. Moves the wizard back to the
* previous step, and retrieves the values that were submitted on that step.
*
* @todo: Can we remove steps that were dynamically added?
*/
function migrate_ui_wizard_previous_submit($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->gotoPreviousStep($form_state);
}
/**
* Validate handler for the 'next' button. Dispatches to the wizard's current
* step for validation.
*/
function migrate_ui_wizard_next_validate($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->formValidate($form_state);
}
/**
* Submit handler for the 'next' button. Saves the form values for the step
* we're leaving, so Previous can pick them up, and moves the wizard to the
* next step.
*/
function migrate_ui_wizard_next_submit($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->gotoNextStep($form_state);
}
/**
* Submit handler for the Save settings button. Register the migrations that were
* (implicitly) defined along the way and redirect to the Migrate dashboard.
*/
function migrate_ui_wizard_submit($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->formSaveSettings();
$form_state['redirect'] = 'admin/content/migrate/groups/' .
$wizard->getGroupName();
}
/**
* Submit handler for the "Save settings and import" button. Register the
* migrations that were (implicitly) defined along the way, run the import, and
* redirect to the Migrate dashboard.
*/
function migrate_ui_wizard_migrate_submit($form, &$form_state) {
/** @var MigrateUIWizard $wizard */
$wizard = $form_state['wizard'];
$wizard->formSaveSettings();
$wizard->formPerformImport();
$form_state['redirect'] = 'admin/content/migrate/groups/' .
$wizard->getGroupName();
}
/**
* The base class for migration wizards. Extend this class to implement a
* wizard UI for importing into Drupal from a given source format (Drupal,
* WordPress, etc.).
*/
abstract class MigrateUIWizard {
/**
* We maintain a doubly-linked list of wizard steps, both to support
* previous/next, and to easily insert steps dynamically.
*
* The first step of the wizard, which has no predecessor. Will generally be
* an overview/introductory page.
*
* @var MigrateUIStep
*/
protected $firstStep;
/**
* The last step of the wizard, which has no successor. Will generally be a
* review page.
*
* @var MigrateUIStep
*/
protected $lastStep;
/**
* Get the list of steps currently defined.
*
* @return
* An array of MigrateUIStep objects, in the order defined, keyed by the step
* name.
*/
protected function getSteps() {
$steps = array();
$steps[$this->firstStep->getName()] = $this->firstStep;
$next_step = $this->firstStep->nextStep;
while (!is_null($next_step)) {
$steps[$next_step->getName()] = $next_step;
$next_step = $next_step->nextStep;
}
return $steps;
}
/**
* The current step of the wizard (the one being shown in the UI, and the one
* whose button is being clicked on).
*
* @var MigrateUIStep
*/
protected $currentStep;
/**
* The step number, used in the page title.
*
* @var int
*/
protected $stepNumber = 1;
/**
* The group name to assign to any Migration instances created.
*
* @var string
*/
protected $groupName = 'default';
public function getGroupName() {
return $this->groupName;
}
/**
* The user-visible title of the group.
*
* @var string
*/
protected $groupTitle = 'default';
/**
* Any arguments that apply to all migrations in the group.
*
* @var array
*/
protected $groupArguments = array();
/**
* Array of Migration argument arrays, keyed by machine name. On Finish, used
* to register Migrations.
*
* @var array
*/
protected $migrations = array();
/**
* Array of MigrateUIWizardExtender objects that extend this wizard.
*
* @var array
*/
protected $extenders = array();
public function getExtender($extender_class) {
if (isset($this->extenders[$extender_class])) {
return $this->extenders[$extender_class];
}
else {
return NULL;
}
}
/**
* Returns the translatable name representing the source of the data (e.g.,
* "Drupal", "WordPress", etc.).
*
* @return string
*/
abstract public function getSourceName();
public function __construct() {}
/**
* Add a wizard extender.
*
* This initializes the new extender and adds it to our internal list.
*
* @param $extender_class
* The name of an extender class.
*/
public function addExtender($extender_class) {
$steps = $this->getSteps();
$extender = new $extender_class($this, $steps);
$this->extenders[$extender_class] = $extender;
}
/**
* Add a step to the wizard, using a step name and method.
*
* @param string $name
* Translatable name for the step, to be used in the page title.
* @param callable $form_method
* Callable returning the form array for the step. This can be either the
* name of a MigrateUIWizard method, or a callable array specifying a method
* on a wizard extender. The validation method is formed from the method's
* name with the suffix 'Validate' added.
* @param MigrateUIStep $after
* Optional step after which to insert the new step. If omitted, add it at
* the end.
* @param mixed $context
* Optional data to be used by this step's form.
*
* @return MigrateUIStep
*/
public function addStep($name, $form_method, MigrateUIStep $after = NULL, $context = NULL) {
if (!is_array($form_method)) {
$form_method = array($this, $form_method);
}
$new_step = new MigrateUIStep($name, $form_method, $context);
// There were no steps, so this is the only one.
if (is_null($this->firstStep)) {
$this->firstStep = $this->lastStep = $this->currentStep = $new_step;
}
else {
// If no insertion point is specified, append to the end.
if (is_null($after)) {
$after = $this->lastStep;
}
// Do the insert, rewriting the links appropriately.
$new_step->nextStep = $after->nextStep;
if (is_null($new_step->nextStep)) {
$this->lastStep = $new_step;
}
else {
$new_step->nextStep->previousStep = $new_step;
}
$new_step->previousStep = $after;
$after->nextStep = $new_step;
}
return $new_step;
}
/**
* Remove the named step from the wizard.
*
* @param $name
*/
protected function removeStep($name) {
for ($current_step = $this->firstStep; !is_null($current_step); $current_step = $current_step->nextStep) {
if ($current_step->getName() == $name) {
if (is_null($current_step->previousStep)) {
$this->firstStep = $current_step->nextStep;
}
else {
$current_step->previousStep->nextStep = $current_step->nextStep;
}
if (is_null($current_step->nextStep)) {
$this->lastStep = $current_step->previousStep;
}
else {
$current_step->nextStep->previousStep = $current_step->previousStep;
}
break;
}
}
}
/**
* Move the wizard to the next step in line (if any), first squirreling away
* the current step's form values.
*/
public function gotoNextStep(&$form_state) {
if ($this->currentStep && $this->currentStep->nextStep) {
$this->currentStep->setFormValues($form_state['values']);
$form_state['rebuild'] = TRUE;
$this->currentStep = $this->currentStep->nextStep;
$this->stepNumber++;
// Ensure a page reload remains on the current step.
$current_step_form_values = $this->currentStep->getFormValues();
if (!empty($current_step_form_values)) {
$form_state['values'] = $current_step_form_values;
}
else {
$form_state['values'] = array();
}
}
}
/**
* Move the wizard to the previous step in line (if any), restoring its
* form values.
*/
public function gotoPreviousStep(&$form_state) {
if ($this->currentStep && $this->currentStep->previousStep) {
$this->currentStep = $this->currentStep->previousStep;
$this->stepNumber--;
$form_state['values'] = $this->currentStep->getFormValues();
$form_state['rebuild'] = TRUE;
}
}
/**
* Build the form for the current step.
*
* @return array
*/
public function form(&$form_state) {
drupal_set_title(t('Import from @source_title',
array('@source_title' => $this->getSourceName())));
$form_method = $this->currentStep->getFormMethod();
$form['title'] = array(
'#prefix' => '<h2>',
'#markup' => t('Step @step: @step_name',
array(
'@step' => $this->stepNumber,
'@step_name' => $this->currentStep->getName())),
'#suffix' => '</h2>',
);
$form += call_user_func($form_method, $form_state);
$form['actions'] = array('#type' => 'actions');
// Show the 'previous' button if appropriate. Note that #submit is set to
// a special submit handler, and that we use #limit_validation_errors to
// skip all complaints about validation when using the back button. The
// values entered will be discarded, but they will not be validated, which
// would be annoying in a "back" button.
if ($this->currentStep != $this->firstStep) {
$form['actions']['prev'] = array(
'#type' => 'submit',
'#value' => t('Previous'),
'#name' => 'prev',
'#submit' => array('migrate_ui_wizard_previous_submit'),
'#limit_validation_errors' => array(),
);
}
// Show the Next button only if there are more steps defined.
if ($this->currentStep == $this->lastStep) {
$form['actions']['finish'] = array(
'#type' => 'submit',
'#value' => t('Save import settings'),
);
$form['actions']['migrate'] = array(
'#type' => 'submit',
'#value' => t('Save import settings and run import'),
'#submit' => array('migrate_ui_wizard_migrate_submit'),
);
}
else {
$form['actions']['next'] = array(
'#type' => 'submit',
'#value' => t('Next'),
'#name' => 'next',
'#submit' => array('migrate_ui_wizard_next_submit'),
'#validate' => array('migrate_ui_wizard_next_validate'),
);
}
return $form;
}
/**
* Call the validation function for the current form (which has the same
* name of the form function with 'Validate' appended).
*
* @param array $form_state
*/
public function formValidate(&$form_state) {
$validate_method = $this->currentStep->getFormMethod();
// This is an array for a method, or a function name.
if (is_array($validate_method)) {
$validate_method[1] .= 'Validate';
}
else {
$validate_method .= 'Validate';
}
if (is_callable($validate_method)) {
call_user_func($validate_method, $form_state);
}
}
/**
* Take the information we've accumulated throughout the wizard, and create
* the Migrations to perform the import.
*/
public function formSaveSettings() {
MigrateGroup::register($this->groupName, $this->groupTitle, $this->groupArguments);
$info['arguments']['group_name'] = $this->groupName;
foreach ($this->migrations as $machine_name => $info) {
// Call the right registerMigration implementation. Note that this means
// that classes that override registerMigration() must handle registration
// themselves, they cannot leave it to us and expect their extension to be
// called.
if (is_subclass_of($info['class_name'], 'Migration')) {
Migration::registerMigration($info['class_name'], $machine_name,
$info['arguments']);
}
else {
MigrationBase::registerMigration($info['class_name'], $machine_name,
$info['arguments']);
}
};
menu_rebuild();
}
/**
* Run the import process for the migration group we've defined.
*/
public function formPerformImport() {
$migrations = migrate_migrations();
$operations = array();
/** @var Migration $migration */
foreach ($migrations as $migration) {
$group_name = $migration->getGroup()->getName();
if ($group_name == $this->groupName) {
$operations[] = array('migrate_ui_batch', array('import', $migration->getMachineName(), NULL, 0));
}
}
if (count($operations) > 0) {
$batch = array(
'operations' => $operations,
'title' => t('Import processing'),
'file' => drupal_get_path('module', 'migrate_ui') . '/migrate_ui.pages.inc',
'init_message' => t('Starting import process'),
'progress_message' => t(''),
'error_message' => t('An error occurred. Some or all of the import processing has failed.'),
'finished' => 'migrate_ui_batch_finish',
);
batch_set($batch);
}
}
/**
* Record all the information necessary to register a migration when this is
* all over.
*
* @param string $machine_name
* Machine name for the migration class.
* @param string $class_name
* Name of the Migration class to instantiate.
* @param array $arguments
* Further information configuring the migration.
*/
public function addMigration($machine_name, $class_name, $arguments) {
// Give extenders an opportunity to modify or reject this migration.
foreach ($this->extenders as $extender) {
if (!$extender->addMigrationAlter($machine_name, $class_name, $arguments, $this)) {
return FALSE;
}
}
$machine_name = $this->groupName . $machine_name;
if (isset($arguments['dependencies'])) {
foreach ($arguments['dependencies'] as $index => $dependency) {
$arguments['dependencies'][$index] = $this->groupName . $dependency;
}
}
if (isset($arguments['soft_dependencies'])) {
foreach ($arguments['soft_dependencies'] as $index => $dependency) {
$arguments['soft_dependencies'][$index] = $this->groupName . $dependency;
}
}
$arguments += array(
'group_name' => $this->groupName,
'machine_name' => $machine_name,
);
$this->migrations[$machine_name] = array(
'class_name' => $class_name,
'arguments' => $arguments,
);
return TRUE;
}
}
/**
* Class representing one step of a wizard.
*/
class MigrateUIStep {
/**
* A translatable string briefly describing this step, to be used in the page
* title for the step form.
*
* @var string
*/
protected $name;
public function getName() {
return $this->name;
}
/**
* Callable that returns the form array for this step.
*
* @var string
*/
protected $formMethod;
public function getFormMethod() {
return $this->formMethod;
}
/**
* The form values ($form_state['values']) submitted for this step, saved in
* case we need to restore them on a Previous action.
*
* @var array
*/
protected $formValues;
public function getFormValues() {
return $this->formValues;
}
public function setFormValues($form_values) {
$this->formValues = $form_values;
}
/**
* Any contextual data needed by the form for this step. For example, a
* field mapping form would need to know the source and destination content
* types so it can determine what fields to expose.
*
* @var mixed
*/
protected $context;
public function getContext() {
return $this->context;
}
/**
* The step object is a node in a doubly-linked list - it links to its
* predecessor and successor steps.
*
* @var MigrateUIStep
*/
public $nextStep;
/**
* @var MigrateUIStep
*/
public $previousStep;
/**
* Class constructor.
*
* @param $name
* The machine name of the wizard step.
* @param $form_method
* A callable for the form array for this step. The validation method is
* formed from the method name with the suffix 'Validate' added, regardless
* of which object it is on.
* @param $context = NULL
* Contextual data needed by the form for this step.
*/
public function __construct($name, $form_method, $context = NULL) {
$this->name = $name;
$this->formMethod = $form_method;
$this->context = $context;
}
}
/**
*
*/
abstract class MigrateUIWizardExtender {
/**
* Reference to the wizard object that this extender applies to.
*/
protected $wizard;
/**
* Class constructor.
*
* Wizard extenders should override this to add their steps to the wizard.
*/
public function __construct(MigrateUIWizard $wizard, array $wizard_steps) {
$this->wizard = $wizard;
}
/**
* Alter the arguments to a migration before it is registered, or potentially
* reject it.
*
* @param string $machine_name
* Machine name for the migration class.
* @param string $class_name
* Name of the Migration class to instantiate.
* @param array $arguments
* Further information configuring the migration.
* @param MigrateUIWizard $wizard
* The wizard class performing the registration.
*
* @return bool
* Return FALSE to prevent registration of this migration.
*/
public function addMigrationAlter($machine_name, $class_name, &$arguments, $wizard) {
return TRUE;
}
}

237
sites/all/modules/contrib/migrate/migrate/plugins/destinations/block_custom.inc Normal file
View File

@@ -0,0 +1,237 @@
<?php
/**
* @file
* Support for custom block destinations.
*/
/**
* Destination class implementing migration into {block_custom}.
*/
class MigrateDestinationCustomBlock extends MigrateDestination {
static public function getKeySchema() {
return array(
'bid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'ID of destination custom block',
),
);
}
public function __construct() {
parent::__construct();
}
public function __toString() {
$output = t('Custom blocks');
return $output;
}
/**
* Returns a list of fields available to be mapped for custom blocks.
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array(
'bid' => t('The custom block ID (bid).'),
'body' => t('Block contents.'),
'info' => t('Block description.'),
'format' => t('The {filter_format}.format of the block body.'),
);
return $fields;
}
/**
* Import a single row.
*
* @param $block
* Custom block object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $block, stdClass $row) {
// Updating previously-migrated content
if (isset($row->migrate_map_destid1)) {
$block->bid = $row->migrate_map_destid1;
}
// Load old values if necessary.
$migration = Migration::currentMigration();
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($block->bid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination bid provided'));
}
if (!$old_block = $this->loadCustomBlock($block->bid)) {
throw new MigrateException(t('System-of-record is DESTINATION, and the provided bid could not be found'));
}
$block_to_update = (object) $old_block;
foreach ($old_block as $key => $value) {
if (!isset($block->$key)) {
$block->$key = $old_block[$key];
}
}
}
// Invoke migration prepare handlers
$this->prepare($block, $row);
// Custom blocks are handled as arrays, so clone the object to an array.
$item = clone $block;
$item = (array) $item;
migrate_instrument_start('block_custom_save');
// Check to see if this is a new custom block.
$update = FALSE;
if (isset($item['bid'])) {
$update = TRUE;
$bid = $this->saveCustomBlock($item);
}
else {
$bid = $this->saveCustomBlock($item);
}
migrate_instrument_stop('block_custom_save');
// Return the new id or FALSE on failure.
if (!empty($bid)) {
// Increment the count if the save succeeded.
if ($update) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
// Return the primary key to the mapping table.
$return = array($bid);
}
else {
$return = FALSE;
}
// Invoke migration complete handlers.
$block = (object) $this->loadCustomBlock($bid);
$this->complete($block, $row);
return $return;
}
/**
* Implementation of MigrateDestination::prepare().
*/
public function prepare($block, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$block->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('block_custom', 'prepare', $block, $row);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($block, $row);
}
}
/**
* Implementation of MigrateDestination::complete().
*/
public function complete($block, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$block->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('block_custom', 'complete', $block, $row);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'complete')) {
$migration->complete($block, $row);
}
}
/**
* Delete a batch of custom blocks at once.
*
* @param $bids
* Array of custom block IDs to be deleted.
*/
public function bulkRollback(array $bids) {
migrate_instrument_start('block_custom_delete_multiple');
$this->prepareRollback($bids);
$this->deleteMultipleCustomBlocks($bids);
$this->completeRollback($bids);
migrate_instrument_stop('block_custom_delete_multiple');
}
/**
* Give handlers a shot at cleaning up before a block has been rolled back.
*
* @param $bid
* ID of the custom block about to be deleted.
*/
public function prepareRollback($bid) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('block_custom', 'prepareRollback', $bid);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($bid);
}
}
/**
* Give handlers a shot at cleaning up after a block has been rolled back.
*
* @param $bid
* ID of the custom block which has been deleted.
*/
public function completeRollback($bid) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('block_custom', 'completeRollback', $bid);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($bid);
}
}
public function loadCustomBlock($bid) {
return block_custom_block_get($bid);
}
public function saveCustomBlock($block) {
drupal_alter('block_custom', $block);
if (!empty($block['bid'])) {
drupal_write_record('block_custom', $block, array('bid'));
module_invoke_all('block_custom_update', $block);
}
else {
drupal_write_record('block_custom', $block);
module_invoke_all('block_custom_insert', $block);
}
return $block['bid'];
}
public function deleteCustomBlock($bid) {
$this->deleteMultipleCustomBlocks(array($bid));
}
public function deleteMultipleCustomBlocks(array $bids) {
db_delete('block_custom')->condition('bid', $bids, 'IN')->execute();
db_delete('block')->condition('module', 'block')->condition('delta', $bids, 'IN')->execute();
db_delete('block_role')->condition('module', 'block')->condition('delta', $bids, 'IN')->execute();
db_delete('block_node_type')->condition('module', 'block')->condition('delta', $bids, 'IN')->execute();
}
}

93
sites/all/modules/contrib/migrate/migrate/plugins/destinations/comment.inc
View File

@@ -65,33 +65,33 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
public function fields($migration = NULL) {
$fields = array();
// First the core (comment table) properties
$fields['cid'] = t('Comment: <a href="@doc">Existing comment ID</a>',
$fields['cid'] = t('<a href="@doc">Existing comment ID</a>',
array('@doc' => 'http://drupal.org/node/1349714#cid'));
$fields['nid'] = t('Comment: <a href="@doc">Node (by Drupal ID)</a>',
$fields['nid'] = t('<a href="@doc">Node (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#nid'));
$fields['uid'] = t('Comment: <a href="@doc">User (by Drupal ID)</a>',
$fields['uid'] = t('<a href="@doc">User (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#uid'));
$fields['pid'] = t('Comment: <a href="@doc">Parent (by Drupal ID)</a>',
$fields['pid'] = t('<a href="@doc">Parent (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#pid'));
$fields['subject'] = t('Comment: <a href="@doc">Subject</a>',
$fields['subject'] = t('<a href="@doc">Subject</a>',
array('@doc' => 'http://drupal.org/node/1349714#subject'));
$fields['created'] = t('Comment: <a href="@doc">Created timestamp</a>',
$fields['created'] = t('<a href="@doc">Created timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349714#created'));
$fields['changed'] = t('Comment: <a href="@doc">Modified timestamp</a>',
$fields['changed'] = t('<a href="@doc">Modified timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349714#changed'));
$fields['status'] = t('Comment: <a href="@doc">Status</a>',
$fields['status'] = t('<a href="@doc">Status</a>',
array('@doc' => 'http://drupal.org/node/1349714#status'));
$fields['hostname'] = t('Comment: <a href="@doc">Hostname/IP address</a>',
$fields['hostname'] = t('<a href="@doc">Hostname/IP address</a>',
array('@doc' => 'http://drupal.org/node/1349714#hostname'));
$fields['name'] = t('Comment: <a href="@doc">User name (not username)</a>',
$fields['name'] = t('<a href="@doc">User name (not username)</a>',
array('@doc' => 'http://drupal.org/node/1349714#name'));
$fields['mail'] = t('Comment: <a href="@doc">Email address</a>',
$fields['mail'] = t('<a href="@doc">Email address</a>',
array('@doc' => 'http://drupal.org/node/1349714#mail'));
$fields['homepage'] = t('Comment: <a href="@doc">Homepage</a>',
$fields['homepage'] = t('<a href="@doc">Homepage</a>',
array('@doc' => 'http://drupal.org/node/1349714#homepage'));
$fields['language'] = t('Comment: <a href="@doc">Language</a>',
$fields['language'] = t('<a href="@doc">Language</a>',
array('@doc' => 'http://drupal.org/node/1349714#language'));
$fields['thread'] = t('Comment: <a href="@doc">Thread</a>',
$fields['thread'] = t('<a href="@doc">Thread</a>',
array('@doc' => 'http://drupal.org/node/1349714#thread'));
// Then add in anything provided by handlers
@@ -149,6 +149,10 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
$comment->changed = MigrationBase::timestamp($comment->changed);
}
if (!isset($comment->node_type)) {
$comment->node_type = $this->bundle;
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($comment->cid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination cid provided'));
@@ -210,6 +214,10 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
else {
$updating = FALSE;
}
// Validate field data prior to saving.
field_attach_validate('comment', $comment);
migrate_instrument_start('comment_save');
comment_save($comment);
migrate_instrument_stop('comment_save');
@@ -291,16 +299,48 @@ class MigrateDestinationComment extends MigrateDestinationEntity {
// Empty table
db_truncate('node_comment_statistics')->execute();
// TODO: DBTNG. Ignore keyword is Mysql only? Is only used in the rare case when
// two comments on the same node share same timestamp.
$sql = "
INSERT IGNORE INTO {node_comment_statistics} (nid, cid, last_comment_timestamp, last_comment_name, last_comment_uid, comment_count) (
SELECT c.nid, c.cid, c.created, c.name, c.uid, c2.comment_count FROM {comment} c
JOIN (
SELECT c.nid, MAX(c.created) AS created, COUNT(*) AS comment_count FROM {comment} c WHERE status=:published GROUP BY c.nid
) AS c2 ON c.nid = c2.nid AND c.created=c2.created
)";
db_query($sql, array(':published' => COMMENT_PUBLISHED));
// DBTNG. IGNORE keyword is not compatible with Postgres. SQLite?
switch (db_driver()) {
case 'pgsql':
// We still want to run this under Postgres. On the very rare occasion
// when we have 2 comments on the same node with the same timestamp
// we will lose data.
$sql = "
INSERT INTO {node_comment_statistics} (nid, cid, last_comment_timestamp, last_comment_name, last_comment_uid, comment_count) (
SELECT c.nid, c.cid, c.created, c.name, c.uid, c2.comment_count
FROM {comment} c
JOIN (
SELECT c.nid, MAX(c.created) AS created, COUNT(*) AS comment_count
FROM {comment} c
WHERE status=:published
GROUP BY c.nid
) AS c2 ON c.nid = c2.nid AND c.created=c2.created
)";
break;
default:
$sql = "
INSERT IGNORE INTO {node_comment_statistics} (nid, cid, last_comment_timestamp, last_comment_name, last_comment_uid, comment_count) (
SELECT c.nid, c.cid, c.created, c.name, c.uid, c2.comment_count
FROM {comment} c
JOIN (
SELECT c.nid, MAX(c.created) AS created, COUNT(*) AS comment_count
FROM {comment} c
WHERE status=:published
GROUP BY c.nid
) AS c2 ON c.nid = c2.nid AND c.created=c2.created
)";
}
try {
db_query($sql, array(':published' => COMMENT_PUBLISHED));
}
catch (Exception $e) {
// Our edge case has been hit. A Postgres migration has likely just
// lost data. Let the user know.
Migration::displayMessage(t('Failed to update node comment statistics: !message',
array('!message' => $e->getMessage())
));
}
// Insert records into the node_comment_statistics for nodes that are missing.
$query = db_select('node', 'n');
@@ -324,7 +364,10 @@ class MigrateCommentNodeHandler extends MigrateDestinationHandler {
$this->registerTypes(array('node'));
}
public function fields($entity_type, $bundle) {
/**
* Implementation of MigrateDestinationHandler::fields().
*/
public function fields($entity_type, $bundle, $migration = NULL) {
$fields = array();
$fields['comment'] = t('Whether comments may be posted to the node');
return $fields;

5
sites/all/modules/contrib/migrate/migrate/plugins/destinations/entity.inc
View File

@@ -138,6 +138,11 @@ abstract class MigrateDestinationEntity extends MigrateDestination {
migrate_handler_invoke_all('Entity', 'prepare', $entity, $source_row);
// Then call any entity-specific handlers
migrate_handler_invoke_all($this->entityType, 'prepare', $entity, $source_row);
// Apply defaults, removing empty fields from the entity object.
$form = $form_state = array();
_field_invoke_default('submit', $this->entityType, $entity, $form, $form_state);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($entity, $source_row);

181
sites/all/modules/contrib/migrate/migrate/plugins/destinations/fields.inc
View File

@@ -12,15 +12,6 @@ class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
/**
* Implementation of MigrateDestinationHandler::fields().
*
* @param $entity_type
* The entity type (node, user, etc.) for which to list fields.
* @param $bundle
* The bundle (article, blog, etc.), if any, for which to list fields.
* @param Migration $migration
* Optionally, the migration providing the context.
* @return array
* An array keyed by field name, with field descriptions as values.
*/
public function fields($entity_type, $bundle, $migration = NULL) {
$fields = array();
@@ -29,12 +20,17 @@ class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
$field_info = field_info_field($machine_name);
$type = $field_info['type'];
$fields[$machine_name] = t('Field:') . ' ' . $instance['label'] .
' (' . $field_info['type'] . ')';
if (user_access(MIGRATE_ACCESS_ADVANCED)) {
$fields[$machine_name] = $instance['label'] . ' (' . $field_info['type'] . ')';
}
else {
$fields[$machine_name] = $instance['label'];
}
// Look for subfields
$class_list = _migrate_class_list('MigrateFieldHandler');
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
$fields_found = FALSE;
foreach ($class_list as $class_name => $handler) {
if (!in_array($class_name, $disabled) && $handler->handlesType($type)
&& method_exists($handler, 'fields')) {
@@ -45,6 +41,18 @@ class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
foreach ($subfields as $subfield_name => $subfield_label) {
$fields[$machine_name . ':' . $subfield_name] = $subfield_label;
}
$fields_found = TRUE;
}
}
if (!$fields_found) {
// Check the default field handler last.
migrate_instrument_start('MigrateDefaultFieldHandler->fields');
$subfields = call_user_func(
array(new MigrateDefaultFieldHandler, 'fields'), $type, $instance,
$migration);
migrate_instrument_stop('MigrateDefaultFieldHandler->fields');
foreach ($subfields as $subfield_name => $subfield_label) {
$fields[$machine_name . ':' . $subfield_name] = $subfield_label;
}
}
}
@@ -128,6 +136,95 @@ abstract class MigrateFieldHandler extends MigrateHandler {
}
}
/**
* A fallback field handler to do basic copying of field data.
*/
class MigrateDefaultFieldHandler extends MigrateFieldHandler {
public function __construct() {}
/**
* Implements MigrateFieldHandler::fields().
*
* @param $field_type
* @param $field_instance
*
* @return array
*/
public function fields($field_type, $field_instance) {
$field_info = field_info_field($field_instance['field_name']);
$fields = array();
$first = TRUE;
foreach ($field_info['columns'] as $column_name => $column_info) {
// The first column is the primary value, which is mapped directly to
// the field name - so, don't include it here among the subfields.
if ($first) {
$first = FALSE;
}
else {
$fields[$column_name] = empty($column_info['description']) ?
$column_name : $column_info['description'];
}
}
return $fields;
}
/**
* Implements MigrateFieldHandler::prepare().
*
* @param $entity
* @param array $field_info
* @param array $instance
* @param array $values
*
* @return null
*/
public function prepare($entity, array $field_info, array $instance,
array $values) {
$arguments = array();
if (isset($values['arguments'])) {
$arguments = array_filter($values['arguments']);
unset($values['arguments']);
}
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
// Get the name of the primary (first) column, which is mapped separately.
reset($field_info['columns']);
$primary_column = key($field_info['columns']);
// Setup the standard Field API array for saving.
$delta = 0;
foreach ($values as $value) {
// Handle multivalue arguments (especially for subfields).
$delta_arguments = array();
foreach ($arguments as $name => $argument) {
if (is_array($argument) && array_key_exists($delta, $argument)) {
$delta_arguments[$name] = $argument[$delta];
}
else {
$delta_arguments[$name] = $argument;
}
}
$return[$language][$delta] = array($primary_column => $value) +
array_intersect_key($delta_arguments, $field_info['columns']);
$delta++;
}
return isset($return) ? $return : NULL;
}
/**
* Overrides MigrateHandler::handlesType().
*
* @param string $type
*
* @return bool
*/
public function handlesType($type) {
// We claim to handle any type.
return TRUE;
}
}
/**
* Base class for creating field handlers for fields with a single value.
*
@@ -238,15 +335,19 @@ class MigrateTextFieldHandler extends MigrateFieldHandler {
public function fields($type, $instance, $migration = NULL) {
$fields = array();
if ($type == 'text_with_summary') {
$fields['summary'] = t('Subfield: Summary of field contents');
$fields['summary'] = t('Subfield: <a href="@doc">Summary of field contents</a>',
array('@doc' => 'http://drupal.org/node/1224042#summary'));
}
if ($instance['settings']['text_processing']) {
$fields['format'] = t('Subfield: Text format for the field');
$fields['format'] = t('Subfield: <a href="@doc">Text format for the field</a>',
array('@doc' => 'http://drupal.org/node/1224042#format'));
}
$fields['language'] = t('Subfield: Language for the field');
$fields['language'] = t('Subfield: <a href="@doc">Language for the field</a>',
array('@doc' => 'http://drupal.org/node/1224042#language'));
return $fields;
}
public function prepare($entity, array $field_info, array $instance, array $values) {
if (isset($values['arguments'])) {
$arguments = $values['arguments'];
@@ -351,9 +452,12 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
*/
public function fields($type, $instance, $migration = NULL) {
return array(
'source_type' => t('Option: Set to \'tid\' when the value is a source ID'),
'create_term' => t('Option: Set to TRUE to create referenced terms when necessary'),
'ignore_case' => t('Option: Set to TRUE to ignore case differences between source data and existing term names'),
'source_type' => t('Option: <a href="@doc">Set to \'tid\' when the value is a source ID</a>',
array('@doc' => 'http://drupal.org/node/1224042#source_type')),
'create_term' => t('Option: <a href="@doc">Set to TRUE to create referenced terms when necessary</a>',
array('@doc' => 'http://drupal.org/node/1224042#create_term')),
'ignore_case' => t('Option: <a href="@doc">Set to TRUE to ignore case differences between source data and existing term names</a>',
array('@doc' => 'http://drupal.org/node/1224042#ignore_case')),
);
}
@@ -375,6 +479,7 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
$tids = $values;
}
elseif ($values) {
$vocab_name = $field_info['settings']['allowed_values'][0]['vocabulary'];
$names = taxonomy_vocabulary_get_names();
// Get the vocabulary for this term
@@ -382,10 +487,12 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
$vid = $field_info['settings']['allowed_values'][0]['vid'];
}
else {
$vocab_name = $field_info['settings']['allowed_values'][0]['vocabulary'];
$vid = $names[$vocab_name]->vid;
}
// Remove leading and trailing spaces in term names
$values = array_map('trim', $values);
// Cannot use taxonomy_term_load_multiple() since we have an array of names.
// It wants a singular value. This query may return case-insensitive
// matches.
@@ -398,17 +505,33 @@ class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
// If we're ignoring case, change both the matched term name keys and the
// source values to lowercase.
if (isset($arguments['ignore_case']) && $arguments['ignore_case']) {
$ignore_case = TRUE;
$existing_terms = array_change_key_case($existing_terms);
$values = array_map('strtolower', $values);
foreach ($values as $value) {
$lower_values[$value] = strtolower($value);
}
}
else {
$ignore_case = FALSE;
}
foreach ($values as $value) {
if (isset($existing_terms[$value])) {
$tids[] = $existing_terms[$value];
}
elseif ($ignore_case && isset($existing_terms[$lower_values[$value]])) {
$tids[] = $existing_terms[$lower_values[$value]];
}
elseif (!empty($arguments['create_term'])) {
$new_term = new stdClass();
$new_term->vid = $vid;
$new_term->name = $value;
$new_term->vocabulary_machine_name = $vocab_name;
// This term is being created with no fields, but we should still call
// field_attach_validate() before saving, as that invokes
// hook_field_attach_validate().
field_attach_validate('taxonomy_term', $new_term);
taxonomy_term_save($new_term);
$tids[] = $new_term->tid;
// Add newly created term to existing array.
@@ -478,7 +601,7 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
$file_class = $mapping->getDefaultValue();
}
}
if (!isset($file_class)) {
if (empty($file_class)) {
$file_class = 'MigrateFileUri';
}
$fields += call_user_func(array($file_class, 'fields'));
@@ -502,7 +625,7 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
$arguments = array();
}
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
$default_language = $this->getFieldLanguage($entity, $field_info, $arguments);
$migration = Migration::currentMigration();
// One can override the source class via CLI or drushrc.php (the
@@ -561,6 +684,10 @@ abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
// MigrateFile class has saved a message indicating why.
if ($file) {
$field_array = array('fid' => $file->fid);
$language = isset($instance_arguments['language']) ? $instance_arguments['language'] : $default_language;
if (is_array($language)) {
$language = $language[$delta];
}
$return[$language][] = $this->buildFieldArray($field_array, $instance_arguments, $delta);
}
}
@@ -629,8 +756,10 @@ class MigrateFileFieldHandler extends MigrateFileFieldBaseHandler {
public function fields($type, $instance, $migration = NULL) {
$fields = parent::fields($type, $instance, $migration);
$fields += array(
'description' => t('Subfield: String to be used as the description value'),
'display' => t('Subfield: String to be used as the display value'),
'description' => t('Subfield: <a href="@doc">String to be used as the description value</a>',
array('@doc' => 'http://drupal.org/node/1224042#description')),
'display' => t('Subfield: <a href="@doc">String to be used as the display value</a>',
array('@doc' => 'http://drupal.org/node/1224042#display')),
);
return $fields;
}
@@ -690,8 +819,10 @@ class MigrateImageFieldHandler extends MigrateFileFieldBaseHandler {
public function fields($type, $instance, $migration = NULL) {
$fields = parent::fields($type, $instance, $migration);
$fields += array(
'alt' => t('Subfield: String to be used as the alt value'),
'title' => t('Subfield: String to be used as the title value'),
'alt' => t('Subfield: <a href="@doc">String to be used as the alt value</a>',
array('@doc' => 'http://drupal.org/node/1224042#alt')),
'title' => t('Subfield: <a href="@doc">String to be used as the title value</a>',
array('@doc' => 'http://drupal.org/node/1224042#title')),
);
return $fields;
}

258
sites/all/modules/contrib/migrate/migrate/plugins/destinations/file.inc
View File

@@ -35,13 +35,110 @@ interface MigrateFileInterface {
}
abstract class MigrateFileBase implements MigrateFileInterface {
/**
* Extension of the core FILE_EXISTS_* constants, offering an alternative to
* reuse the existing file if present as-is (core only offers the options of
* replacing it or renaming to avoid collision).
*/
const FILE_EXISTS_REUSE = -1;
/**
* An optional file object to use as a default starting point for building the
* file entity.
*
* @var stdClass
*/
protected $defaultFile;
/**
* How to handle destination filename collisions.
*
* @var int
*/
protected $fileReplace = FILE_EXISTS_RENAME;
/**
* Set to TRUE to prevent file deletion on rollback.
*
* @var bool
*/
protected $preserveFiles = FALSE;
public function __construct($arguments = array(), $default_file = NULL) {
if (isset($arguments['preserve_files'])) {
$this->preserveFiles = $arguments['preserve_files'];
}
if (isset($arguments['file_replace'])) {
$this->fileReplace = $arguments['file_replace'];
}
if ($default_file) {
$this->defaultFile = $default_file;
}
else {
$this->defaultFile = new stdClass;
}
}
/**
* Default implementation of MigrateFileInterface::fields().
*
* @return array
*/
static public function fields() {
return array();
return array(
'preserve_files' => t('Option: <a href="@doc">Boolean indicating whether files should be preserved or deleted on rollback</a>',
array('@doc' => 'http://drupal.org/node/1540106#preserve_files')),
);
}
/**
* Setup a file entity object suitable for saving.
*
* @param $destination
* Path to the Drupal copy of the file.
* @param $owner
* Uid of the file owner.
* @return stdClass
* A file object ready to be saved.
*/
protected function createFileEntity($destination, $owner) {
$file = clone $this->defaultFile;
$file->uri = $destination;
$file->uid = $owner;
if (!isset($file->filename)) {
$file->filename = drupal_basename($destination);
}
if (!isset($file->filemime)) {
$file->filemime = file_get_mimetype(urldecode($destination));
}
if (!isset($file->status)) {
$file->status = FILE_STATUS_PERMANENT;
}
if (empty($file->type) || $file->type == 'file') {
// Try to determine the file type.
if (module_exists('file_entity')) {
$type = file_get_type($file);
}
elseif ($slash_pos = strpos($file->filemime, '/')) {
$type = substr($file->filemime, 0, $slash_pos);
}
$file->type = isset($type) ? $type : 'file';
}
// If we are replacing or reusing an existing filesystem entry,
// also re-use its database record.
if ($this->fileReplace == FILE_EXISTS_REPLACE ||
$this->fileReplace == self::FILE_EXISTS_REUSE) {
$existing_files = file_load_multiple(array(), array('uri' => $destination));
if (count($existing_files)) {
$existing = reset($existing_files);
$file->fid = $existing->fid;
$file->filename = $existing->filename;
}
}
return $file;
}
/**
@@ -68,6 +165,18 @@ abstract class MigrateFileBase implements MigrateFileInterface {
}
}
/**
* The simplest possible file class - where the value is a remote URI which
* simply needs to be saved as the URI on the destination side, with no attempt
* to copy or otherwise use it.
*/
class MigrateFileUriAsIs extends MigrateFileBase {
public function processFile($value, $owner) {
$file = file_save($this->createFileEntity($value, $owner));
return $file;
}
}
/**
* Handle the degenerate case where we already have a file ID.
*/
@@ -92,13 +201,6 @@ class MigrateFileFid extends MigrateFileBase {
* Base class for creating core file entities.
*/
abstract class MigrateFile extends MigrateFileBase {
/**
* Extension of the core FILE_EXISTS_* constants, offering an alternative to
* reuse the existing file if present as-is (core only offers the options of
* replacing it or renaming to avoid collision).
*/
const FILE_EXISTS_REUSE = -1;
/**
* The destination directory within Drupal.
*
@@ -113,47 +215,14 @@ abstract class MigrateFile extends MigrateFileBase {
*/
protected $destinationFile = '';
/**
* How to handle destination filename collisions.
*
* @var int
*/
protected $fileReplace = FILE_EXISTS_RENAME;
/**
* Set to TRUE to prevent file deletion on rollback.
*
* @var bool
*/
protected $preserveFiles = FALSE;
/**
* An optional file object to use as a default starting point for building the
* file entity.
*
* @var stdClass
*/
protected $defaultFile;
public function __construct($arguments = array(), $default_file = NULL) {
parent::__construct($arguments, $default_file);
if (isset($arguments['destination_dir'])) {
$this->destinationDir = $arguments['destination_dir'];
}
if (isset($arguments['destination_file'])) {
$this->destinationFile = $arguments['destination_file'];
}
if (isset($arguments['file_replace'])) {
$this->fileReplace = $arguments['file_replace'];
}
if (isset($arguments['preserve_files'])) {
$this->preserveFiles = $arguments['preserve_files'];
}
if ($default_file) {
$this->defaultFile = $default_file;
}
else {
$this->defaultFile = new stdClass;
}
}
/**
@@ -162,55 +231,16 @@ abstract class MigrateFile extends MigrateFileBase {
* @return array
*/
static public function fields() {
return array(
return parent::fields() + array(
'destination_dir' => t('Subfield: <a href="@doc">Path within Drupal files directory to store file</a>',
array('@doc' => 'http://drupal.org/node/1540106#destination_dir')),
'destination_file' => t('Subfield: <a href="@doc">Path within destination_dir to store the file.</a>',
array('@doc' => 'http://drupal.org/node/1540106#destination_file')),
'file_replace' => t('Option: <a href="@doc">Value of $replace in that file function. Does not apply to file_fast(). Defaults to FILE_EXISTS_RENAME.</a>',
'file_replace' => t('Option: <a href="@doc">Value of $replace in that file function. Defaults to FILE_EXISTS_RENAME.</a>',
array('@doc' => 'http://drupal.org/node/1540106#file_replace')),
'preserve_files' => t('Option: <a href="@doc">Boolean indicating whether files should be preserved or deleted on rollback</a>',
array('@doc' => 'http://drupal.org/node/1540106#preserve_files')),
);
}
/**
* Setup a file entity object suitable for saving.
*
* @param $destination
* Path to the Drupal copy of the file.
* @param $owner
* Uid of the file owner.
* @return stdClass
* A file object ready to be saved.
*/
protected function createFileEntity($destination, $owner) {
$file = clone $this->defaultFile;
$file->uri = $destination;
$file->uid = $owner;
if (!isset($file->filename)) {
$file->filename = drupal_basename($destination);
}
if (!isset($file->filemime)) {
$file->filemime = file_get_mimetype($destination);
}
if (!isset($file->status)) {
$file->status = FILE_STATUS_PERMANENT;
}
// If we are replacing or reusing an existing filesystem entry,
// also re-use its database record.
if ($this->fileReplace == FILE_EXISTS_REPLACE ||
$this->fileReplace == self::FILE_EXISTS_REUSE) {
$existing_files = file_load_multiple(array(), array('uri' => $destination));
if (count($existing_files)) {
$existing = reset($existing_files);
$file->fid = $existing->fid;
$file->filename = $existing->filename;
}
}
return $file;
}
/**
* By whatever appropriate means, put the file in the right place.
*
@@ -242,13 +272,10 @@ abstract class MigrateFile extends MigrateFileBase {
// Our own file_replace behavior - if the file exists, use it without
// replacing it
if ($this->fileReplace == self::FILE_EXISTS_REUSE) {
// See if we this file already (we'll reuse a file entity if it exists).
// See if we this file already (we'll reuse and resave a file entity if it exists).
if (file_exists($destination)) {
$file = $this->createFileEntity($destination, $owner);
// File entity didn't already exist, create it
if (empty($file->fid)) {
$file = file_save($file);
}
$file = file_save($file);
$this->markForPreservation($file->fid);
return $file;
}
@@ -257,7 +284,8 @@ abstract class MigrateFile extends MigrateFileBase {
}
// Prepare the destination directory.
if (!file_prepare_directory(drupal_dirname($destination),
$destdir = drupal_dirname($destination);
if (!file_prepare_directory($destdir,
FILE_CREATE_DIRECTORY | FILE_MODIFY_PERMISSIONS)) {
$migration->saveMessage(t('Could not create destination directory for !dest',
array('!dest' => $destination)));
@@ -267,8 +295,7 @@ abstract class MigrateFile extends MigrateFileBase {
// Determine whether we can perform this operation based on overwrite rules.
$destination = file_destination($destination, $this->fileReplace);
if ($destination === FALSE) {
$migration->saveMessage(t('The file could not be copied because ' .
'file %dest already exists in the destination directory.',
$migration->saveMessage(t('The file could not be copied because file %dest already exists in the destination directory.',
array('%dest' => $destination)));
return FALSE;
}
@@ -318,11 +345,19 @@ class MigrateFileUri extends MigrateFile {
*/
protected $sourcePath = '';
/**
* Whether to apply rawurlencode to the components of an incoming file path.
*/
protected $urlEncode = TRUE;
public function __construct($arguments = array(), $default_file = NULL) {
parent::__construct($arguments, $default_file);
if (isset($arguments['source_dir'])) {
$this->sourceDir = rtrim($arguments['source_dir'], "/\\");
}
if (isset($arguments['urlencode'])) {
$this->urlEncode = $arguments['urlencode'];
}
}
/**
@@ -335,6 +370,8 @@ class MigrateFileUri extends MigrateFile {
array(
'source_dir' => t('Subfield: <a href="@doc">Path to source file.</a>',
array('@doc' => 'http://drupal.org/node/1540106#source_dir')),
'urlencode' => t('Option: <a href="@doc">Encode all segments of the incoming path (defaults to TRUE).</a>',
array('@doc' => 'http://drupal.org/node/1540106#urlencode')),
);
}
@@ -348,12 +385,13 @@ class MigrateFileUri extends MigrateFile {
* TRUE if the copy succeeded, FALSE otherwise.
*/
protected function copyFile($destination) {
// Perform the copy operation, with a cleaned-up path.
$this->sourcePath = self::urlencode($this->sourcePath);
if ($this->urlEncode) {
// Perform the copy operation, with a cleaned-up path.
$this->sourcePath = self::urlencode($this->sourcePath);
}
if (!@copy($this->sourcePath, $destination)) {
$migration = Migration::currentMigration();
$migration->saveMessage(t('The specified file %file could not be copied to ' .
'%destination.',
$migration->saveMessage(t('The specified file %file could not be copied to %destination.',
array('%file' => $this->sourcePath, '%destination' => $destination)));
return FALSE;
}
@@ -377,8 +415,10 @@ class MigrateFileUri extends MigrateFile {
$components[$key] = rawurlencode($component);
}
$filename = implode('/', $components);
// Actually, we don't want colons encoded
// Actually, we don't want certain characters encoded
$filename = str_replace('%3A', ':', $filename);
$filename = str_replace('%3F', '?', $filename);
$filename = str_replace('%26', '&', $filename);
}
return $filename;
}
@@ -473,6 +513,9 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
* @var string
*/
protected $fileClass;
public function setFileClass($file_class) {
$this->fileClass = $file_class;
}
/**
* Boolean indicating whether we should avoid deleting the actual file on
@@ -521,10 +564,10 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
public function fields($migration = NULL) {
$fields = array();
// First the core properties
$fields['fid'] = t('File: Existing file ID');
$fields['uid'] = t('File: Uid of user associated with file');
$fields['value'] = t('File: Representation of the source file (usually a URI)');
$fields['timestamp'] = t('File: UNIX timestamp for the date the file was added');
$fields['fid'] = t('Existing file ID');
$fields['uid'] = t('Uid of user associated with file');
$fields['value'] = t('Representation of the source file (usually a URI)');
$fields['timestamp'] = t('UNIX timestamp for the date the file was added');
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
@@ -559,12 +602,14 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
else {
$preserve_files = FALSE;
}
$this->prepareRollback($fid);
if ($preserve_files) {
$this->fileDelete($file);
}
else {
file_delete($file, TRUE);
}
$this->completeRollback($fid);
migrate_instrument_stop('file_delete');
}
}
@@ -616,6 +661,19 @@ class MigrateDestinationFile extends MigrateDestinationEntity {
$old_file = file_load($file->fid);
}
// 'type' is the bundle property on file entities. It must be set here for
// the sake of the prepare handlers, although it may be overridden later
// based on the detected mime type.
if (empty($file->type)) {
// If a bundle was specified in the constructor we use it for filetype.
if ($this->bundle != 'file') {
$file->type = $this->bundle;
}
else {
$file->type = 'file';
}
}
// Invoke migration prepare handlers
$this->prepare($file, $row);

180
sites/all/modules/contrib/migrate/migrate/plugins/destinations/node.inc
View File

@@ -12,6 +12,8 @@
* Destination class implementing migration into nodes.
*/
class MigrateDestinationNode extends MigrateDestinationEntity {
protected $bypassDestIdCheck = FALSE;
static public function getKeySchema() {
return array(
'nid' => array(
@@ -66,29 +68,29 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
array('@doc' => 'http://drupal.org/node/1349696#title'))
. $node_type->title_label . '</a>';
}
$fields['uid'] = t('Node: <a href="@doc">Authored by (uid)</a>',
$fields['uid'] = t('<a href="@doc">Authored by (uid)</a>',
array('@doc' => 'http://drupal.org/node/1349696#uid'));
$fields['created'] = t('Node: <a href="@doc">Created timestamp</a>',
$fields['created'] = t('<a href="@doc">Created timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349696#created'));
$fields['changed'] = t('Node: <a href="@doc">Modified timestamp</a>',
$fields['changed'] = t('<a href="@doc">Modified timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349696#changed'));
$fields['status'] = t('Node: <a href="@doc">Published</a>',
$fields['status'] = t('<a href="@doc">Published</a>',
array('@doc' => 'http://drupal.org/node/1349696#status'));
$fields['promote'] = t('Node: <a href="@doc">Promoted to front page</a>',
$fields['promote'] = t('<a href="@doc">Promoted to front page</a>',
array('@doc' => 'http://drupal.org/node/1349696#promote'));
$fields['sticky'] = t('Node: <a href="@doc">Sticky at top of lists</a>',
$fields['sticky'] = t('<a href="@doc">Sticky at top of lists</a>',
array('@doc' => 'http://drupal.org/node/1349696#sticky'));
$fields['revision'] = t('Node: <a href="@doc">Create new revision</a>',
$fields['revision'] = t('<a href="@doc">Create new revision</a>',
array('@doc' => 'http://drupal.org/node/1349696#revision'));
$fields['log'] = t('Node: <a href="@doc">Revision Log message</a>',
$fields['log'] = t('<a href="@doc">Revision Log message</a>',
array('@doc' => 'http://drupal.org/node/1349696#log'));
$fields['language'] = t('Node: <a href="@doc">Language (fr, en, ...)</a>',
$fields['language'] = t('<a href="@doc">Language (fr, en, ...)</a>',
array('@doc' => 'http://drupal.org/node/1349696#language'));
$fields['tnid'] = t('Node: <a href="@doc">The translation set id for this node</a>',
$fields['tnid'] = t('<a href="@doc">The translation set id for this node</a>',
array('@doc' => 'http://drupal.org/node/1349696#tnid'));
$fields['translate'] = t('Node: <a href="@doc">A boolean indicating whether this translation page needs to be updated</a>',
$fields['translate'] = t('<a href="@doc">A boolean indicating whether this translation page needs to be updated</a>',
array('@doc' => 'http://drupal.org/node/1349696#translate'));
$fields['revision_uid'] = t('Node: <a href="@doc">Modified (uid)</a>',
$fields['revision_uid'] = t('<a href="@doc">Modified (uid)</a>',
array('@doc' => 'http://drupal.org/node/1349696#revision_uid'));
$fields['is_new'] = t('Option: <a href="@doc">Indicates a new node with the specified nid should be created</a>',
array('@doc' => 'http://drupal.org/node/1349696#is_new'));
@@ -128,7 +130,7 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
public function import(stdClass $node, stdClass $row) {
// Updating previously-migrated content?
$migration = Migration::currentMigration();
if (isset($row->migrate_map_destid1)) {
if (isset($row->migrate_map_destid1) && !$this->bypassDestIdCheck) {
// Make sure is_new is off
$node->is_new = FALSE;
if (isset($node->nid)) {
@@ -177,7 +179,8 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
$node->uid = $old_node->uid;
}
}
elseif (!isset($node->type)) {
if (!isset($node->type)) {
// Default the type to our designated destination bundle (by doing this
// conditionally, we permit some flexibility in terms of implementing
// migrations which can affect more than one type).
@@ -207,7 +210,12 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
if (isset($node->uid)) {
$uid = $node->uid;
}
if (isset($node->revision)) {
$revision = $node->revision;
}
node_object_prepare($node);
if (isset($created)) {
$node->created = $created;
}
@@ -215,6 +223,9 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
if (isset($uid)) {
$node->uid = $uid;
}
if (isset($revision)) {
$node->revision = $revision;
}
}
// Invoke migration prepare handlers
@@ -246,6 +257,14 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
$updating = FALSE;
}
// Make sure that if is_new is not TRUE, it is not present.
if (isset($node->is_new) && empty($node->is_new)) {
unset($node->is_new);
}
// Validate field data prior to saving.
field_attach_validate('node', $node);
migrate_instrument_start('node_save');
node_save($node);
migrate_instrument_stop('node_save');
@@ -296,3 +315,136 @@ class MigrateDestinationNode extends MigrateDestinationEntity {
return $return;
}
}
/**
* Allows you to import revisions.
*
* Adapted from http://www.darrenmothersele.com/blog/2012/07/16/migrating-node-revisions-drupal-7/
*
* Class MigrateDestinationNodeRevision
*
* @author darrenmothersele
* @author cthos
*/
class MigrateDestinationNodeRevision extends MigrateDestinationNode {
/**
* Basic initialization.
*
* @see parent::__construct
*
* @param string $bundle
* A.k.a. the content type (page, article, etc.) of the node.
* @param array $options
* Options applied to nodes.
*/
public function __construct($bundle, array $options = array()) {
parent::__construct($bundle, $options);
$this->bypassDestIdCheck = TRUE;
}
/**
* Get key schema for the node revision destination.
*
* @see MigrateDestination::getKeySchema
*
* @return array
* Returns the key schema.
*/
static public function getKeySchema() {
return array(
'vid' => array(
'type' => 'int',
'unsigned' => TRUE,
'description' => 'ID of destination node revision',
),
);
}
/**
* Returns additional fields on top of node destinations.
*
* @param string $migration
* Active migration
*
* @return array
* Fields.
*/
public function fields($migration = NULL) {
$fields = parent::fields($migration);
$fields['vid'] = t('Node: <a href="@doc">Revision (vid)</a>', array('@doc' => 'http://drupal.org/node/1298724'));
return $fields;
}
/**
* Rolls back any versions that have been created.
*
* @param array $vids
* Version ids to roll back.
*/
public function bulkRollback(array $vids) {
migrate_instrument_start('revision_delete_multiple');
$this->prepareRollback($vids);
$nids = array();
foreach ($vids as $vid) {
if ($revision = node_load(NULL, $vid)) {
db_delete('node_revision')
->condition('vid', $revision->vid)
->execute();
module_invoke_all('node_revision_delete', $revision);
field_attach_delete_revision('node', $revision);
$nids[$revision->nid] = $revision->nid;
}
}
$this->completeRollback($vids);
foreach ($nids as $nid) {
$vid = db_select('node_revision', 'nr')->fields('nr', array('vid'))->condition('nid', $nid, '=')->execute()->fetchField();
if (!empty($vid)) {
db_update('node')->fields(array('vid' => $vid))->condition('nid', $nid, '=')->execute();
}
}
migrate_instrument_stop('revision_delete_multiple');
}
/**
* Overridden import method.
*
* This is done because parent::import will return the nid of the newly
* created nodes. This is bad since the migrate_map_* table will have
* nids instead of vids, which could cause a nightmare explosion on
* rollback.
*
* @param stdClass $node
* Populated entity.
*
* @param stdClass $row
* Source information in object format.
*
* @return array|bool
* Array with newly created vid, or FALSE on error.
*
* @throws MigrateException
*/
public function import(stdClass $node, stdClass $row) {
// We're importing revisions, this should be set.
$node->revision = 1;
if (empty($node->nid)) {
throw new MigrateException(t('Missing incoming nid.'));
}
$original_updated = $this->numUpdated;
parent::import($node, $row);
// Reset num updated and increment created since new revision is always an update.
$this->numUpdated = $original_updated;
$this->numCreated++;
if (empty($node->vid)) {
return FALSE;
}
return array($node->vid);
}
}

31
sites/all/modules/contrib/migrate/migrate/plugins/destinations/path.inc
View File

@@ -10,25 +10,32 @@ class MigratePathEntityHandler extends MigrateDestinationHandler {
$this->registerTypes(array('entity'));
}
public function fields($entity_type, $bundle) {
/**
* Implementation of MigrateDestinationHandler::fields().
*/
public function fields($entity_type, $bundle, $migration = NULL) {
if (module_exists('path')) {
switch ($entity_type) {
case 'node':
return array('path' => t('Node: Path alias'));
case 'user':
return array('path' => t('User: Path alias'));
case 'taxonomy_term':
return array('path' => t('Term: Path alias'));
}
return array('path' => t('Path alias'));
}
return array();
}
public function prepare($entity, stdClass $row) {
if (module_exists('path') && isset($entity->path)) {
$path = $entity->path;
$entity->path = array();
$entity->path['alias'] = $path;
// Make sure the alias doesn't already exist
$query = db_select('url_alias')
->condition('alias', $entity->path)
->condition('language', $entity->language);
$query->addExpression('1');
$query->range(0, 1);
if (!$query->execute()->fetchField()) {
$path = $entity->path;
$entity->path = array();
$entity->path['alias'] = $path;
}
else {
unset($entity->path);
}
}
}
}

19
sites/all/modules/contrib/migrate/migrate/plugins/destinations/poll.inc
View File

@@ -60,13 +60,16 @@ class MigratePollEntityHandler extends MigrateDestinationHandler {
$this->registerTypes(array('node'));
}
public function fields($entity_type, $bundle) {
/**
* Implementation of MigrateDestinationHandler::fields().
*/
public function fields($entity_type, $bundle, $migration = NULL) {
if ($bundle == 'poll') {
$fields = array(
'active' => t('Poll: Active status'),
'runtime' => t('Poll: How long the poll runs for in seconds'),
'choice' => t('Poll: Choices. Each choice is an array with chtext, chvotes, and weight keys.'),
'votes' => t('Poll: Votes. Each vote is an array with chid (or chtext), uid, hostname, and timestamp keys'),
'active' => t('Active status'),
'runtime' => t('How long the poll runs for in seconds'),
'choice' => t('Choices. Each choice is an array with chtext, chvotes, and weight keys.'),
'votes' => t('Votes. Each vote is an array with chid (or chtext), uid, hostname, and timestamp keys'),
);
}
else {
@@ -97,8 +100,7 @@ class MigratePollEntityHandler extends MigrateDestinationHandler {
// Insert actual votes.
foreach ($row->votes as $vote) {
$chid = $vote['chid'];
if (!isset($chid)) {
if (!isset($vote['chid'])) {
$result = db_select('poll_choice', 'pc')
->fields('pc', array('chid'))
->condition('pc.nid', $entity->nid)
@@ -106,6 +108,9 @@ class MigratePollEntityHandler extends MigrateDestinationHandler {
->execute();
$chid = $result->fetchField();
}
else {
$chid = $vote['chid'];
}
db_insert('poll_vote')
->fields(array(
'chid' => $chid,

20
sites/all/modules/contrib/migrate/migrate/plugins/destinations/statistics.inc
View File

@@ -10,12 +10,15 @@ class MigrateStatisticsEntityHandler extends MigrateDestinationHandler {
$this->registerTypes(array('node'));
}
public function fields() {
/**
* Implementation of MigrateDestinationHandler::fields().
*/
public function fields($entity_type, $bundle, $migration = NULL) {
if (module_exists('statistics')) {
$fields = array(
'totalcount' => t('Node: The total number of times the node has been viewed.'),
'daycount' => t('Node: The total number of times the node has been viewed today.'),
'timestamp' => t('Node: The most recent time the node has been viewed.'),
'totalcount' => t('The total number of times the node has been viewed.'),
'daycount' => t('The total number of times the node has been viewed today.'),
'timestamp' => t('The most recent time the node has been viewed.'),
);
}
else {
@@ -26,9 +29,12 @@ class MigrateStatisticsEntityHandler extends MigrateDestinationHandler {
public function complete($node, stdClass $row) {
if (module_exists('statistics') && isset($node->nid)) {
$totalcount = isset($node->totalcount) ? $node->totalcount : 0;
$daycount = isset($node->daycount) ? $node->daycount : 0;
$timestamp = isset($node->timestamp) ? $node->timestamp : 0;
$totalcount = isset($node->totalcount) && is_numeric($node->totalcount) ?
$node->totalcount : 0;
$daycount = isset($node->daycount) && is_numeric($node->daycount) ?
$node->daycount : 0;
$timestamp = isset($node->timestamp) && is_numeric($node->timestamp) ?
$node->timestamp : 0;
db_merge('node_counter')
->key(array('nid' => $node->nid))
->fields(array(

57
sites/all/modules/contrib/migrate/migrate/plugins/destinations/table.inc
View File

@@ -57,19 +57,20 @@ class MigrateDestinationTable extends MigrateDestination {
/**
* Delete a single row.
*
* @param $id
* Primary key values.
* @param array $ids
* The primary key values of the row to be deleted.
*/
public function rollback(array $id) {
public function rollback(array $ids) {
migrate_instrument_start('table rollback');
$delete = db_delete($this->tableName);
$keys = array_keys(self::getKeySchema($this->tableName));
$i = 0;
foreach ($id as $value) {
$key = $keys[$i++];
$values = array_combine($keys, $ids);
$this->prepareRollback($values);
$delete = db_delete($this->tableName);
foreach ($values as $key => $value) {
$delete->condition($key, $value);
}
$delete->execute();
$this->completeRollback($values);
migrate_instrument_stop('table rollback');
}
@@ -165,7 +166,7 @@ class MigrateDestinationTable extends MigrateDestination {
public function fields($migration = NULL) {
$fields = array();
foreach ($this->schema['fields'] as $column => $schema) {
$fields[$column] = t('Type: !type', array('!type' => $schema['type']));
$fields[$column] = t('!type', array('!type' => $schema['type']));
}
return $fields;
}
@@ -207,4 +208,44 @@ class MigrateDestinationTable extends MigrateDestination {
$migration->complete($entity, $source_row);
}
}
/**
* Give handlers a shot at cleaning up before the row has been rolled back.
*
* @param array $ids
* The primary key values of the row about to be deleted, keyed by field
* name.
*/
public function prepareRollback(array $ids) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('table', 'prepareRollback', $ids);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($ids);
}
}
/**
* Give handlers a shot at cleaning up after a row has been rolled back.
*
* @param array $ids
* The primary key values of the row which has been deleted, keyed by field
* name.
*/
public function completeRollback(array $ids) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('table', 'completeRollback', $ids);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($ids);
}
}
}

11
sites/all/modules/contrib/migrate/migrate/plugins/destinations/table_copy.inc
View File

@@ -49,13 +49,18 @@ class MigrateDestinationTableCopy extends MigrateDestination {
$migration = MigrationBase::currentMigration();
$fields = clone $row;
// Remove all map data, otherwise we'll try to write it to the destination
// table.
foreach ($fields as $field => $data) {
if (strpos($field, 'migrate_map_') === 0) {
unset($fields->$field);
}
}
$keys = array_keys($this->keySchema);
$values = array();
foreach ($keys as $key) {
$values[] = $row->$key;
}
unset($fields->migrate_map_destid1);
unset($fields->needs_update);
$query = db_merge($this->tableName)->key($keys, $values)->fields((array)$fields);
try {
$status = $query->execute();
@@ -72,7 +77,7 @@ class MigrateDestinationTableCopy extends MigrateDestination {
Migration::displayMessage($e->getMessage());
}
catch (Exception $e) {
$this->handleException($e);
$migration->handleException($e);
}
}

42
sites/all/modules/contrib/migrate/migrate/plugins/destinations/term.inc
View File

@@ -70,20 +70,20 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
public function fields($migration = NULL) {
$fields = array();
// First the core (taxonomy_term_data table) properties
$fields['tid'] = t('Term: <a href="@doc">Existing term ID</a>',
$fields['tid'] = t('<a href="@doc">Existing term ID</a>',
array('@doc' => 'http://drupal.org/node/1349702#tid'));
$fields['name'] = t('Term: <a href="@doc">Name</a>',
$fields['name'] = t('<a href="@doc">Name</a>',
array('@doc' => 'http://drupal.org/node/1349702#name'));
$fields['description'] = t('Term: <a href="@doc">Description</a>',
$fields['description'] = t('<a href="@doc">Description</a>',
array('@doc' => 'http://drupal.org/node/1349702#description'));
$fields['parent'] = t('Term: <a href="@doc">Parent (by Drupal term ID)</a>',
$fields['parent'] = t('<a href="@doc">Parent (by Drupal term ID)</a>',
array('@doc' => 'http://drupal.org/node/1349702#parent'));
// TODO: Remove parent_name, implement via arguments
$fields['parent_name'] = t('Term: <a href="@doc">Parent (by name)</a>',
$fields['parent_name'] = t('<a href="@doc">Parent (by name)</a>',
array('@doc' => 'http://drupal.org/node/1349702#parent_name'));
$fields['format'] = t('Term: <a href="@doc">Format</a>',
$fields['format'] = t('<a href="@doc">Format</a>',
array('@doc' => 'http://drupal.org/node/1349702#format'));
$fields['weight'] = t('Term: <a href="@doc">Weight</a>',
$fields['weight'] = t('<a href="@doc">Weight</a>',
array('@doc' => 'http://drupal.org/node/1349702#weight'));
// Then add in anything provided by handlers
@@ -149,6 +149,12 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
$term->tid = $row->migrate_map_destid1;
}
}
// Default to bundle if no vocabulary machine name provided
if (!isset($term->vocabulary_machine_name)) {
$term->vocabulary_machine_name = $this->bundle;
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($term->tid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination tid provided'));
@@ -166,10 +172,6 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
$term = $old_term;
}
else {
// Default to bundle if no vocabulary machine name provided
if (!isset($term->vocabulary_machine_name)) {
$term->vocabulary_machine_name = $this->bundle;
}
// vid is required
if (empty($term->vid)) {
static $vocab_map = array();
@@ -203,7 +205,13 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
if (empty($term->parent)) {
$term->parent = array(0);
}
if (is_array($term->parent) && isset($term->parent['arguments'])) {
elseif (!is_array($term->parent)) {
// Convert to an array for comparison in findMatchingTerm().
// Note: taxonomy_term_save() also normalizes to an array.
$term->parent = array($term->parent);
}
if (isset($term->parent['arguments'])) {
// Unset arguments here to avoid duplicate entries in the
// term_hierarchy table.
unset($term->parent['arguments']);
@@ -211,8 +219,15 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
if (!isset($term->format)) {
$term->format = $this->textFormat;
}
if (!isset($term->language)) {
$term->language = $this->language;
}
$this->prepare($term, $row);
if (empty($term->name)) {
throw new MigrateException(t('Taxonomy term name is required.'));
}
if (!$this->allowDuplicateTerms && $existing_term = $this->findMatchingTerm($term)) {
foreach ($existing_term as $field => $value) {
if (!isset($term->$field)) {
@@ -242,6 +257,9 @@ class MigrateDestinationTerm extends MigrateDestinationEntity {
$updating = FALSE;
}
// Validate field data prior to saving.
field_attach_validate('taxonomy_term', $term);
migrate_instrument_start('taxonomy_term_save');
$status = taxonomy_term_save($term);
migrate_instrument_stop('taxonomy_term_save');

53
sites/all/modules/contrib/migrate/migrate/plugins/destinations/user.inc
View File

@@ -75,41 +75,41 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
public function fields($migration = NULL) {
$fields = array();
// First the core (users table) properties
$fields['uid'] = t('User: <a href="@doc">Existing user ID</a>',
$fields['uid'] = t('<a href="@doc">Existing user ID</a>',
array('@doc' => 'http://drupal.org/node/1349632#uid'));
$fields['mail'] = t('User: <a href="@doc">Email address</a>',
$fields['mail'] = t('<a href="@doc">Email address</a>',
array('@doc' => 'http://drupal.org/node/1349632#mail'));
$fields['name'] = t('User: <a href="@doc">Username</a>',
$fields['name'] = t('<a href="@doc">Username</a>',
array('@doc' => 'http://drupal.org/node/1349632#name'));
$fields['pass'] = t('User: <a href="@doc">Password (plain text)</a>',
$fields['pass'] = t('<a href="@doc">Password</a>',
array('@doc' => 'http://drupal.org/node/1349632#pass'));
$fields['status'] = t('User: <a href="@doc">Status</a>',
$fields['status'] = t('<a href="@doc">Status</a>',
array('@doc' => 'http://drupal.org/node/1349632#status'));
$fields['created'] = t('User: <a href="@doc">Registered timestamp</a>',
$fields['created'] = t('<a href="@doc">Registered timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#created'));
$fields['access'] = t('User: <a href="@doc">Last access timestamp</a>',
$fields['access'] = t('<a href="@doc">Last access timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#access'));
$fields['login'] = t('User: <a href="@doc">Last login timestamp</a>',
$fields['login'] = t('<a href="@doc">Last login timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#login'));
$fields['roles'] = t('User: <a href="@doc">Role IDs</a>',
$fields['roles'] = t('<a href="@doc">Role IDs</a>',
array('@doc' => 'http://drupal.org/node/1349632#roles'));
$fields['role_names'] = t('User: <a href="@doc">Role Names</a>',
$fields['role_names'] = t('<a href="@doc">Role Names</a>',
array('@doc' => 'http://drupal.org/node/1349632#role_names'));
$fields['picture'] = t('User: <a href="@doc">Picture</a>',
$fields['picture'] = t('<a href="@doc">Picture</a>',
array('@doc' => 'http://drupal.org/node/1349632#picture'));
$fields['signature'] = t('User: <a href="@doc">Signature</a>',
$fields['signature'] = t('<a href="@doc">Signature</a>',
array('@doc' => 'http://drupal.org/node/1349632#signature'));
$fields['signature_format'] = t('User: <a href="@doc">Signature format</a>',
$fields['signature_format'] = t('<a href="@doc">Signature format</a>',
array('@doc' => 'http://drupal.org/node/1349632#signature_format'));
$fields['timezone'] = t('User: <a href="@doc">Timezone</a>',
$fields['timezone'] = t('<a href="@doc">Timezone</a>',
array('@doc' => 'http://drupal.org/node/1349632#timezone'));
$fields['language'] = t('User: <a href="@doc">Language</a>',
$fields['language'] = t('<a href="@doc">Language</a>',
array('@doc' => 'http://drupal.org/node/1349632#language'));
$fields['theme'] = t('User: <a href="@doc">Default theme</a>',
$fields['theme'] = t('<a href="@doc">Default theme</a>',
array('@doc' => 'http://drupal.org/node/1349632#theme'));
$fields['init'] = t('User: <a href="@doc">Init</a>',
$fields['init'] = t('<a href="@doc">Init</a>',
array('@doc' => 'http://drupal.org/node/1349632#init'));
$fields['data'] = t('User: <a href="@doc">Data</a>',
$fields['data'] = t('<a href="@doc">Data</a>',
array('@doc' => 'http://drupal.org/node/1349632#init'));
$fields['is_new'] = t('Option: <a href="@doc">Indicates a new user with the specified uid should be created</a>',
array('@doc' => 'http://drupal.org/node/1349632#is_new'));
@@ -230,6 +230,9 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
$account->login = MigrationBase::timestamp($account->login);
}
// Validate field data prior to saving.
field_attach_validate('user', $account);
migrate_instrument_start('user_save');
$newaccount = user_save($old_account, (array)$account);
migrate_instrument_stop('user_save');
@@ -262,6 +265,14 @@ class MigrateDestinationUser extends MigrateDestinationEntity {
}
else {
$this->numCreated++;
// user_save() doesn't update file_usage on account creation, we have
// to do it ourselves.
if (!empty($newaccount->picture)) {
$file = file_load($newaccount->picture);
if (is_object($file)) {
file_usage_add($file, 'user', 'user', $newaccount->uid);
}
}
}
$this->complete($newaccount, $row);
$return = array($newaccount->uid);
@@ -324,9 +335,9 @@ class MigrateDestinationRole extends MigrateDestinationTable {
throw new MigrateException(t("Incoming id !id and map destination id !destid don't match",
array('!id' => $entity->rid, '!destid' => $row->migrate_map_destid1)));
}
else {
$entity->rid = $row->migrate_map_destid1;
}
}
else {
$entity->rid = $row->migrate_map_destid1;
}
}

187
sites/all/modules/contrib/migrate/migrate/plugins/destinations/variable.inc Normal file
View File

@@ -0,0 +1,187 @@
<?php
/**
* @file
* Support for variable destinations.
*/
/**
* Destination class implementing migration into {variable}.
*/
class MigrateDestinationVariable extends MigrateDestination {
static public function getKeySchema() {
return array(
'name' => array(
'description' => 'The name of the variable.',
'type' => 'varchar',
'length' => 128,
'not null' => TRUE,
'default' => '',
),
);
}
public function __construct() {
parent::__construct();
}
public function __toString() {
$output = t('Variable');
return $output;
}
/**
* Returns a list of fields available to be mapped for variables.
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array(
'name' => t('The name of the variable.'),
'value' => t('The value of the variable.'),
);
return $fields;
}
/**
* Import a single row.
*
* @param $variable
* Variable object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $variable, stdClass $row) {
// Invoke migration prepare handlers
$this->prepare($variable, $row);
// Check to see if this is a new variable.
$update = FALSE;
// We cannot just check against NULL because a variable might actually be
// set to NULL. Attempt to use a unique variable default value that nothing
// else would use.
$default = 'migrate:' . REQUEST_TIME . ':' . drupal_random_key();
if (variable_get($variable->name, $default) !== $default) {
$update = TRUE;
}
// variable_set() provides no return callback, so we can't really test this
// without running a variable_get() check.
migrate_instrument_start('variable_set');
variable_set($variable->name, $variable->value);
migrate_instrument_stop('variable_set');
// Return the new id or FALSE on failure.
if (variable_get($variable->name, $default) === $variable->value) {
// Increment the count if the save succeeded.
if ($update) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
// Return the primary key to the mapping table.
$return = array($variable->name);
}
else {
$return = FALSE;
}
// Invoke migration complete handlers.
$this->complete($variable, $row);
return $return;
}
/**
* Implementation of MigrateDestination::prepare().
*/
public function prepare($variable, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$variable->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('variable', 'prepare', $variable, $row);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($variable, $row);
}
}
/**
* Implementation of MigrateDestination::complete().
*/
public function complete($variable, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$variable->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('variable', 'complete', $variable, $row);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'complete')) {
$migration->complete($variable, $row);
}
}
/**
* Delete a single variable.
*
* @param $id
* Array of fields representing the key (in this case, just variable name).
*/
public function rollback(array $id) {
$name = reset($id);
migrate_instrument_start('variable_delete');
$this->prepareRollback($name);
variable_del($name);
$this->completeRollback($name);
migrate_instrument_stop('variable_delete');
}
/**
* Give handlers a shot at cleaning up before a variable has been rolled back.
*
* @param $name
* The name of the variable about to be deleted.
*/
public function prepareRollback($name) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('variable', 'prepareRollback', $name);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($name);
}
}
/**
* Give handlers a shot at cleaning up after a variable has been rolled back.
*
* @param $name
* The name of the variable which has been deleted.
*/
public function completeRollback($name) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('variable', 'completeRollback', $name);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($name);
}
}
}

2
sites/all/modules/contrib/migrate/migrate/plugins/sources/csv.inc
View File

@@ -186,6 +186,8 @@ class MigrateSourceCSV extends MigrateSource {
public function getNextRow() {
$row = $this->getNextLine();
if ($row) {
// only use rows specified in $this->csvcolumns().
$row = array_intersect_key($row, $this->csvcolumns);
// Set meaningful keys for the columns mentioned in $this->csvcolumns().
foreach ($this->csvcolumns as $int => $values) {
list($key, $description) = $values;

163
sites/all/modules/contrib/migrate/migrate/plugins/sources/db2.inc Normal file
View File

@@ -0,0 +1,163 @@
<?php
/**
* @file
* Define a MigrateSource class for importing from IBM DB2 databases.
*/
/**
* Implementation of MigrateSource, to handle imports from remote DB2 servers.
*/
class MigrateSourceDB2 extends MigrateSource {
/**
* Array containing information for connecting to DB2:
* database - Database to connect to
* username - Username to connect as
* password - Password for authentication
*
* @var array
*/
protected $configuration;
/**
* The active DB2 connection for this source.
*
* @var resource
*/
protected $connection;
public function getConnection() {
return $this->connection;
}
/**
* The SQL query from which to obtain data. Is a string.
*/
protected $query;
/**
* The statement resource from executing the query - traversed to process the
* incoming data.
*/
protected $stmt;
/**
* Return an options array for DB2 sources.
*
* @param boolean $cache_counts
* Indicates whether to cache counts of source records.
*/
static public function options($cache_counts = FALSE) {
return compact('cache_counts');
}
/**
* Simple initialization.
*/
public function __construct(array $configuration, $query, $count_query,
array $fields, array $options = array()) {
parent::__construct($options);
$this->query = $query;
$this->countQuery = $count_query;
$this->configuration = $configuration;
$this->fields = $fields;
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
return $this->query;
}
/**
* Connect lazily to the DB server.
*/
protected function connect() {
if (!isset($this->connection)) {
// Check for the ibm_db2 extension before attempting to connect with it.
if (!extension_loaded('ibm_db2')) {
throw new Exception(t('You must configure the ibm_db2 extension in PHP.'));
}
// Connect to db2.
$this->connection = db2_connect($this->configuration['database'],
$this->configuration['username'], $this->configuration['password']);
}
if ($this->connection) {
return TRUE;
}
// If we failed to connect, throw an exception with the connection error
// message.
else {
$e = db2_conn_errormsg();
throw new Exception($e);
return FALSE;
}
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
// The fields are passed to the constructor for this plugin.
return $this->fields;
}
/**
* Return a count of all available source records.
*/
public function computeCount() {
migrate_instrument_start('MigrateSourceDB2 count');
// Make sure we're connected.
if ($this->connect()) {
// Execute the count query.
$stmt = db2_exec($this->connection, $this->countQuery);
// If something went wrong, throw an exception with the error message.
if (!$stmt) {
$e = db2_stmt_errormsg($stmt);
throw new Exception($e);
}
// Grab the first row as an array.
$count_array = db2_fetch_array($stmt);
// The first item in this array will be our count.
$count = reset($count_array);
}
else {
// Connection failed.
$count = FALSE;
}
migrate_instrument_stop('MigrateSourceDB2 count');
return $count;
}
/**
* Implementation of MigrateSource::performRewind().
*/
public function performRewind() {
migrate_instrument_start('db2_query');
// Ensure we're connected to the database.
$this->connect();
// Execute the query.
$this->stmt = db2_exec($this->connection, $this->query);
// Throw an exception with the error message if something went wrong.
if (!$this->stmt) {
$e = db2_stmt_errormsg($this->stmt);
throw new Exception($e);
}
migrate_instrument_stop('db2_query');
}
/**
* Implementation of MigrateSource::getNextRow().
*
* @return object
*/
public function getNextRow() {
return db2_fetch_object($this->stmt);
}
}

27
sites/all/modules/contrib/migrate/migrate/plugins/sources/files.inc
View File

@@ -62,6 +62,7 @@ class MigrateListFiles extends MigrateList {
protected $fileMask;
protected $directoryOptions;
protected $parser;
protected $getContents;
/**
* Constructor.
@@ -86,8 +87,12 @@ class MigrateListFiles extends MigrateList {
* @param $options
* Options that will be passed on to file_scan_directory(). See docs of that
* core Drupal function for more information.
* @param MigrateContentParser $parser
* Content parser class to use.
* @param $get_contents
* Whether to load the contents of files.
*/
public function __construct($list_dirs, $base_dir, $file_mask = NULL, $options = array(), MigrateContentParser $parser = NULL) {
public function __construct($list_dirs, $base_dir, $file_mask = NULL, $options = array(), MigrateContentParser $parser = NULL, $get_contents = TRUE) {
if (!$parser) {
$parser = new MigrateSimpleContentParser();
}
@@ -97,6 +102,7 @@ class MigrateListFiles extends MigrateList {
$this->fileMask = $file_mask;
$this->directoryOptions = $options;
$this->parser = $parser;
$this->getContents = $get_contents;
}
/**
@@ -136,11 +142,16 @@ class MigrateListFiles extends MigrateList {
protected function getIDsFromFiles(array $files) {
$ids = array();
foreach ($files as $file) {
$contents = file_get_contents($file->uri);
$this->parser->setContent($contents);
if ($this->parser->getChunkCount() > 1) {
foreach ($this->parser->getChunkIDs() as $chunk_id) {
$ids[] = str_replace($this->baseDir, '', (string) $file->uri) . MIGRATE_CHUNK_SEPARATOR . $chunk_id;
if ($this->getContents) {
$contents = file_get_contents($file->uri);
$this->parser->setContent($contents);
if ($this->parser->getChunkCount() > 1) {
foreach ($this->parser->getChunkIDs() as $chunk_id) {
$ids[] = str_replace($this->baseDir, '', (string) $file->uri) . MIGRATE_CHUNK_SEPARATOR . $chunk_id;
}
}
else {
$ids[] = str_replace($this->baseDir, '', (string) $file->uri);
}
}
else {
@@ -206,7 +217,7 @@ class MigrateItemFile extends MigrateItem {
public function getItem($id) {
$pieces = explode(MIGRATE_CHUNK_SEPARATOR ,$id);
$item_uri = $this->baseDir . $pieces[0];
$chunk = $pieces[1];
$chunk = !empty($pieces[1]) ? $pieces[1] : '';
// Get the file data at the specified URI
$data = $this->loadFile($item_uri);
@@ -242,4 +253,4 @@ class MigrateItemFile extends MigrateItem {
}
return $data;
}
}
}

4
sites/all/modules/contrib/migrate/migrate/plugins/sources/json.inc
View File

@@ -92,7 +92,7 @@ class MigrateListJSON extends MigrateList {
if ($json) {
$data = drupal_json_decode($json);
if ($data) {
$count = count($data);
$count = count($this->getIDsFromJSON($data));
}
}
return $count;
@@ -503,7 +503,7 @@ class MigrateSourceJSON extends MigrateSource {
* @return string
*/
public function activeUrl() {
if ($this->activeUrl) {
if (isset($this->activeUrl)) {
return $this->sourceUrls[$this->activeUrl];
}
}

25
sites/all/modules/contrib/migrate/migrate/plugins/sources/list.inc
View File

@@ -56,6 +56,17 @@ abstract class MigrateItem {
* @return stdClass
*/
abstract public function getItem($id);
/**
* Implementors may optionally implement a hash function, applied to the
* entire source row, if this particular item type makes it difficult to
* do on the raw row.
*
* @param $row
*
* @return mixed
*/
//abstract public function hash($row);
}
/**
@@ -192,4 +203,18 @@ class MigrateSourceList extends MigrateSource {
}
return $row;
}
/**
* Overrides MigrateSource::hash().
*/
protected function hash($row) {
// Let the item class override the default hash function.
if (method_exists($this->itemClass, 'hash')) {
$hash = $this->itemClass->hash($row);
}
else {
$hash = parent::hash($row);
}
return $hash;
}
}

189
sites/all/modules/contrib/migrate/migrate/plugins/sources/mongodb.inc Normal file
View File

@@ -0,0 +1,189 @@
<?php
/**
* @file
* Define a MigrateSource for importing from MongoDB connections
*/
/**
* Implementation of MigrateSource, to handle imports from MongoDB connections.
*/
class MigrateSourceMongoDB extends MigrateSource {
/**
* The mongodb collection object.
*
* @var MongoCollection
*/
protected $collection;
/**
* The mongodb cursor object.
*
* @var MongoCursor
*/
protected $cursor;
/**
* The mongodb query.
*
* @var array
*/
protected $query;
/**
* List of available source fields.
*
* @var array
*/
protected $fields = array();
/**
* Simple initialization.
*/
public function __construct(MongoCollection $collection, array $query,
array $fields = array(), array $sort = array('_id' => 1),
array $options = array()) {
parent::__construct($options);
$this->collection = $collection;
$this->query = $query;
$this->sort = $sort;
$this->fields = $fields;
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
// The fields are passed to the constructor for this plugin.
return $this->fields;
}
/**
* Return a count of all available source records.
*/
public function computeCount() {
return $this->cursor->count(TRUE);
}
/**
* Implementation of MigrateSource::getNextRow().
*
* @return object
*/
public function getNextRow() {
$row = $this->cursor->getNext();
if ($row) {
return (object) $row;
}
return NULL;
}
/**
* Implementation of MigrateSource::performRewind().
*
* @return void
*/
public function performRewind() {
$keys = $this->getSourceKeyNameAndType();
// If we have an existing idlist we use it.
if ($this->idList) {
foreach ($this->idList as $key => $id) {
// Try make new ObjectID.
$this->idList[$key] = $this->getMongoId($id, $keys);
}
$this->query[$keys[0]['name']]['$in'] = $this->idList;
}
migrate_instrument_start('MigrateSourceMongoDB execute');
try {
$this->cursor = $this->collection
->find($this->query)
->sort($this->sort);
$this->cursor->timeout(-1);
} catch (MongoCursorException $e) {
Migration::displayMessage($e->getMessage(), 'error');
}
migrate_instrument_stop('MigrateSourceMongoDB execute');
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
if (is_null($this->cursor)) {
$this->cursor = $this->collection
->find($this->query)
->sort($this->sort);
$this->cursor->timeout(-1);
}
$query_info = $this->cursor->info();
$query = 'query: ' . drupal_json_encode($query_info['query']['$query']);
$sort = 'order by: ' . drupal_json_encode($query_info['query']['$orderby']);
$fields = 'fields: ' . drupal_json_encode($query_info['fields']);
return $query . PHP_EOL .
$sort . PHP_EOL .
$fields . PHP_EOL;
}
/**
* Check if given document id is a mongo ObjectId and return mongo ObjectId
* or simple value.
*
* @param mixed $document_id
* Document key value.
* @param array $keys
* List of keys.
* @return type
*/
public function getMongoId($document_id, $keys) {
if ($keys[0]['name'] != '_id') {
switch ($keys[0]['type']) {
case 'int':
return (int)$document_id;
break;
default:
return $document_id;
}
}
// Trying create Mongo ObjectId
$mongoid = new MongoId($document_id);
// If (string) $mongoid == $document_id we return $mongoid object
if ((string) $mongoid == $document_id) {
return $mongoid;
}
return $document_id;
}
/**
* Get source keys array.
*/
public function getSourceKeyNameAndType() {
// Get the key name, and type.
$keys = array();
foreach ($this->activeMap->getSourceKey() as $field_name => $field_schema) {
$keys[] = array(
'name' => $field_name,
'type' => $field_schema['type'],
);
}
return $keys;
}
}

3
sites/all/modules/contrib/migrate/migrate/plugins/sources/mssql.inc
View File

@@ -125,7 +125,8 @@ class MigrateSourceMSSQL extends MigrateSource {
migrate_instrument_start('MigrateSourceMSSQL count');
if ($this->connect()) {
$result = mssql_query($this->countQuery);
$count = reset(mssql_fetch_object($result));
$result_array = mssql_fetch_array($result);
$count = reset($result_array);
}
else {
// Do something else?

27
sites/all/modules/contrib/migrate/migrate/plugins/sources/multiitems.inc
View File

@@ -47,8 +47,18 @@ abstract class MigrateItems {
* @return stdClass
*/
abstract public function getItem($id);
}
/**
* Implementors may optionally implement a hash function, applied to the
* entire source row, if this particular item type makes it difficult to
* do on the raw row.
*
* @param $row
*
* @return mixed
*/
//abstract public function hash($row);
}
/**
* Implementation of MigrateItems, for providing a list of IDs and for
@@ -182,5 +192,18 @@ class MigrateSourceMultiItems extends MigrateSource {
}
return $row;
}
}
/**
* Overrides MigrateSource::hash().
*/
protected function hash($row) {
// Let the item class override the default hash function.
if (method_exists($this->itemsClass, 'hash')) {
$hash = $this->itemsClass->hash($row);
}
else {
$hash = parent::hash($row);
}
return $hash;
}
}

238
sites/all/modules/contrib/migrate/migrate/plugins/sources/spreadsheet.inc Normal file
View File

@@ -0,0 +1,238 @@
<?php
/**
* @file
* Define a MigrateSource for importing from spreadsheet files.
*
* Requires the PHPExcel library to be installed.
* - Download PHPExcel at http://phpexcel.codeplex.com/.
* - Extract the archive to a temporary folder.
* - Ensure the hosting environment fulfills the requirements found in
* install.txt.
* - Copy the contents of the Classes folder to an appropriate location
* (sites/all/libraries/PHPExcel).
*/
/**
* Implements MigrateSource, to handle imports from XLS files.
*/
class MigrateSourceSpreadsheet extends MigrateSource {
/**
* PHPExcel object for storing the workbook data.
*
* @var PHPExcel
*/
protected $workbook;
/**
* PHPExcel object for storing the worksheet data.
*
* @var PHPExcel_Worksheet
*/
protected $worksheet;
/**
* The name of the worksheet that will be processed.
*
* @var string
*/
protected $sheetName;
/**
* Number of rows in the worksheet that is being processed.
*
* @var integer
*/
protected $rows = 0;
/**
* Number of columns in the worksheet that is being processed.
*
* @var integer
*/
protected $cols = 0;
/**
* List of available source fields.
*
* @var array
*/
protected $fields = array();
/**
* The current row number in the XLS file.
*
* @var integer+ */
protected $rowNumber;
/**
* The columns to be read from Excel
*/
protected $columns;
/**
* Simple initialization.
*
* @param string $path
* The path to the source file.
* @param string $sheet_name
* The name of the sheet to be processed.
* @param array $options
* Options applied to this source.
*/
public function __construct($path, $sheet_name, $columns = array(), array $options = array()) {
parent::__construct($options);
$this->file = $path;
$this->sheetName = $sheet_name;
$this->columns = $columns;
// Load the workbook.
if ($this->load()) {
// Get the dimensions of the worksheet.
$this->rows = $this->worksheet->getHighestDataRow();
$this->cols = PHPExcel_Cell::columnIndexFromString($this->worksheet->getHighestDataColumn());
// Map field names to their column index.
for ($col = 0; $col < $this->cols; ++$col) {
$this->fields[$col] = trim($this->worksheet->getCellByColumnAndRow($col, 1)->getValue());
}
$this->unload();
}
}
/**
* Loads the workbook.
*
* @return bool
* Returns true if the workbook was successfully loaded, otherwise false.
*/
public function load() {
// Check that the file exists.
if (!file_exists($this->file)) {
Migration::displayMessage(t('The file !filename does not exist.', array('!filename' => $this->file)));
return FALSE;
}
// Check that required modules are enabled.
if (!module_exists('libraries')) {
Migration::displayMessage(t('The Libraries API module is not enabled.'));
return FALSE;
}
if (!module_exists('phpexcel')) {
Migration::displayMessage(t('The PHPExcel module is not enabled.'));
return FALSE;
}
$library = libraries_load('PHPExcel');
if (empty($library['loaded'])) {
Migration::displayMessage(t('The PHPExcel library could not be found.'));
return FALSE;
}
// Load the workbook.
try {
// Identify the type of the input file.
$type = PHPExcel_IOFactory::identify($this->file);
// Create a new Reader of the file type.
$reader = PHPExcel_IOFactory::createReader($type);
// Advise the Reader that we only want to load cell data.
$reader->setReadDataOnly(TRUE);
// Advise the Reader of which worksheet we want to load.
$reader->setLoadSheetsOnly($this->sheetName);
// Load the source file.
$this->workbook = $reader->load($this->file);
$this->worksheet = $this->workbook->getSheet();
}
catch (Exception $e) {
Migration::displayMessage(t('Error loading file: %message', array('%message' => $e->getMessage())));
return FALSE;
}
return TRUE;
}
/**
* Unloads the workbook.
*/
public function unload() {
$this->workbook->disconnectWorksheets();
unset($this->workbook);
}
/**
* Returns a string representing the source query.
*
* @return string
*/
public function __toString() {
return $this->file;
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping).
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
$fields = array();
foreach ($this->fields as $name) {
$fields[$name] = $name;
}
return $fields;
}
/**
* Returns a count of all available source records.
*/
public function computeCount() {
// Subtract 1 for the header.
return $this->rows - 1;
}
/**
* Implements MigrateSource::performRewind().
*
* @return void
*/
public function performRewind() {
// Initialize the workbook if it isn't already.
if (!isset($this->workbook)) {
$this->load();
}
$this->rowNumber = 1;
}
/**
* Implements MigrateSource::getNextRow().
*
* @return null|object
*/
public function getNextRow() {
migrate_instrument_start('MigrateSourceSpreadsheet::next');
++$this->rowNumber;
if ($this->rowNumber <= $this->rows) {
$row_values = array();
for ($col = 0; $col < $this->cols; ++$col) {
if (in_array($this->fields[$col], $this->columns) || empty($this->columns)) {
$row_values[$this->fields[$col]] = trim($this->worksheet->getCellByColumnAndRow($col, $this->rowNumber)->getValue());
}
}
return (object) $row_values;
}
else {
// EOF, close the workbook.
$this->unload();
migrate_instrument_stop('MigrateSourceSpreadsheet::next');
return NULL;
}
}
}

148
sites/all/modules/contrib/migrate/migrate/plugins/sources/sql.inc
View File

@@ -14,7 +14,7 @@ class MigrateSourceSQL extends MigrateSource {
*
* @var SelectQueryInterface
*/
protected $originalQuery, $query, $countQuery;
protected $originalQuery, $query, $countQuery, $alteredQuery;
/**
* Return a reference to the base query, in particular so Migration classes
@@ -24,7 +24,7 @@ class MigrateSourceSQL extends MigrateSource {
* @return SelectQueryInterface
*/
public function &query() {
return $this->originalQuery();
return $this->originalQuery;
}
/**
@@ -42,6 +42,21 @@ class MigrateSourceSQL extends MigrateSource {
*/
protected $numProcessed = 0;
/**
* Current data batch.
*
* @var int
*/
protected $batch = 0;
/**
* Number of records to fetch from the database during each batch. A value
* of zero indicates no batching is to be done.
*
* @var int
*/
protected $batchSize = 0;
/**
* List of available source fields.
*
@@ -118,6 +133,18 @@ class MigrateSourceSQL extends MigrateSource {
$this->countQuery = $count_query;
}
if (isset($options['batch_size'])) {
$this->batchSize = $options['batch_size'];
// Joining to the map table is incompatible with batching, disable it.
$options['map_joinable'] = FALSE;
}
// If we're tracking changes, then we need to fetch all rows to see if
// they've changed, we can't make that determination through a direct join.
if (!empty($options['track_changes'])) {
$options['map_joinable'] = FALSE;
}
if (isset($options['map_joinable'])) {
$this->mapJoinable = $options['map_joinable'];
}
@@ -153,14 +180,15 @@ class MigrateSourceSQL extends MigrateSource {
}
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
return (string) $this->query;
$query = clone $this->query;
$query = $query->extend('MigrateConnectionQuery');
return $query->getString();
}
/**
@@ -243,6 +271,7 @@ class MigrateSourceSQL extends MigrateSource {
public function performRewind() {
$this->result = NULL;
$this->query = clone $this->originalQuery;
$this->batch = 0;
// Get the key values, for potential use in joining to the map table, or
// enforcing idlist.
@@ -259,7 +288,41 @@ class MigrateSourceSQL extends MigrateSource {
// 1. If idlist is provided, then only process items in that list (AND key
// IN (idlist)). Only applicable with single-value keys.
if ($this->idList) {
$this->query->condition($keys[0], $this->idList, 'IN');
$simple_ids = array();
$compound_ids = array();
$key_count = count($keys);
foreach ($this->idList as $id) {
// Look for multi-key separator. If there is only 1 key, ignore.
if (strpos($id, $this->multikeySeparator) === FALSE || $key_count == 1) {
$simple_ids[] = $id;
continue;
}
$compound_ids[] = explode($this->multikeySeparator, $id);
}
// Check for compunded ids. If present add them with subsequent OR statements.
if (!empty($compound_ids)) {
$condition = db_or();
if (!empty($simple_ids)) {
$condition->condition($keys[0], $simple_ids, 'IN');
}
foreach ($compound_ids as $values) {
$temp_and = db_and();
foreach ($values as $pos => $value) {
$temp_and->condition($keys[$pos], $value);
}
$condition->condition($temp_and);
}
$this->query->condition($condition);
}
else {
$this->query->condition($keys[0], $simple_ids, 'IN');
}
}
else {
// 2. If the map is joinable, join it. We will want to accept all rows
@@ -325,8 +388,16 @@ class MigrateSourceSQL extends MigrateSource {
if ($condition_added) {
$this->query->condition($conditions);
}
// 4. Download data in batches for performance.
if ($this->batchSize > 0) {
$this->query->range($this->batch * $this->batchSize, $this->batchSize);
}
}
// Save our fixed-up query so getNextBatch() matches it.
$this->alteredQuery = clone $this->query;
migrate_instrument_start('MigrateSourceSQL execute');
$this->result = $this->query->execute();
migrate_instrument_stop('MigrateSourceSQL execute');
@@ -338,6 +409,71 @@ class MigrateSourceSQL extends MigrateSource {
* @return object
*/
public function getNextRow() {
return $this->result->fetchObject();
$row = $this->result->fetchObject();
// We might be out of data entirely, or just out of data in the current batch.
// Attempt to fetch the next batch and see.
if (!is_object($row) && $this->batchSize > 0) {
$this->getNextBatch();
$row = $this->result->fetchObject();
}
if (is_object($row)) {
return $row;
}
else {
return NULL;
}
}
/**
* Downloads the next set of data from the source database.
*/
protected function getNextBatch() {
$this->batch++;
$query = clone $this->alteredQuery;
$query->range($this->batch * $this->batchSize, $this->batchSize);
$this->result = $query->execute();
}
}
/**
* Query extender for retrieving the connection used on the query.
*/
class MigrateConnectionQuery extends SelectQueryExtender {
public function __construct(SelectQueryInterface $query, DatabaseConnection $connection) {
parent::__construct($query, $connection);
// Add the connection as metadata if anything else wants to access it.
$query->addMetaData('connection', $connection);
}
/**
* Return a string representing the source query.
*
* This is copied from devel module's dpq() function.
*
* @param bool $prefix
* If the tables should be prefixed. If FALSE will return tables names in
* the query like {tablename}.
*
* @return string
* The SQL query.
*/
public function getString($prefix = TRUE) {
$query = $this;
if (method_exists($this, 'preExecute')) {
$query->preExecute();
}
$sql = (string) $this;
$quoted = array();
foreach ((array) $this->arguments() as $key => $val) {
$quoted[$key] = $this->connection->quote($val);
}
$sql = strtr($sql, $quoted);
if ($prefix) {
$sql = $this->connection->prefixTables($sql);
}
return $sql;
}
}

66
sites/all/modules/contrib/migrate/migrate/plugins/sources/sqlmap.inc
View File

@@ -69,19 +69,40 @@ class MigrateSQLMap extends MigrateMap {
*/
protected $ensured;
/**
* Constructor.
*
* @param string $machine_name
* The unique reference to the migration that we are mapping.
* @param array $source_key
* The database schema for the source key.
* @param array $destination_key
* The database schema for the destination key.
* @param string $connection_key
* Optional - The connection used to create the mapping tables. By default
* this is the destination (Drupal). If it's not possible to make joins
* between the destination database and your source database you can specify
* a different connection to create the mapping tables on.
* @param array $options
* Optional - Options applied to this source.
*/
public function __construct($machine_name, array $source_key,
array $destination_key, $connection_key = 'default', $options = array()) {
if (isset($options['track_last_imported'])) {
$this->trackLastImported = TRUE;
}
$this->connection = Database::getConnection('default', $connection_key);
// Default generated table names, limited to 63 characters
$prefixLength = strlen($this->connection->tablePrefix()) ;
$this->mapTable = 'migrate_map_' . drupal_strtolower($machine_name);
$this->mapTable = drupal_substr($this->mapTable, 0, 63);
$this->mapTable = drupal_substr($this->mapTable, 0, 63 - $prefixLength);
$this->messageTable = 'migrate_message_' . drupal_strtolower($machine_name);
$this->messageTable = drupal_substr($this->messageTable, 0, 63);
$this->messageTable = drupal_substr($this->messageTable, 0, 63 - $prefixLength);
$this->sourceKey = $source_key;
$this->destinationKey = $destination_key;
$this->connection = Database::getConnection('default', $connection_key);
// Build the source and destination key maps
$this->sourceKeyMap = array();
$count = 1;
@@ -147,6 +168,12 @@ class MigrateSQLMap extends MigrateMap {
'default' => 0,
'description' => 'UNIX timestamp of the last time this row was imported',
);
$fields['hash'] = array(
'type' => 'varchar',
'length' => '32',
'not null' => FALSE,
'description' => 'Hash of source row data, for detecting changes',
);
$schema = array(
'description' => t('Mappings from source key to destination key'),
'fields' => $fields,
@@ -182,6 +209,29 @@ class MigrateSQLMap extends MigrateMap {
);
$this->connection->schema()->createTable($this->messageTable, $schema);
}
else {
// Add any missing columns to the map table
if (!$this->connection->schema()->fieldExists($this->mapTable,
'rollback_action')) {
$this->connection->schema()->addField($this->mapTable,
'rollback_action', array(
'type' => 'int',
'size' => 'tiny',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => 0,
'description' => 'Flag indicating what to do for this item on rollback',
));
}
if (!$this->connection->schema()->fieldExists($this->mapTable, 'hash')) {
$this->connection->schema()->addField($this->mapTable, 'hash', array(
'type' => 'varchar',
'length' => '32',
'not null' => FALSE,
'description' => 'Hash of source row data, for detecting changes',
));
}
}
$this->ensured = TRUE;
}
}
@@ -300,9 +350,12 @@ class MigrateSQLMap extends MigrateMap {
* @param int $rollback_action
* How to handle the destination object on rollback. Defaults to
* ROLLBACK_DELETE.
* $param string $hash
* If hashing is enabled, the hash of the raw source row.
*/
public function saveIDMapping(stdClass $source_row, array $dest_ids,
$needs_update = MigrateMap::STATUS_IMPORTED, $rollback_action = MigrateMap::ROLLBACK_DELETE) {
$needs_update = MigrateMap::STATUS_IMPORTED,
$rollback_action = MigrateMap::ROLLBACK_DELETE, $hash = NULL) {
migrate_instrument_start('saveIDMapping');
// Construct the source key
$keys = array();
@@ -321,6 +374,7 @@ class MigrateSQLMap extends MigrateMap {
$fields = array(
'needs_update' => (int)$needs_update,
'rollback_action' => (int)$rollback_action,
'hash' => $hash,
);
$count = 1;
if (!empty($dest_ids)) {
@@ -354,8 +408,8 @@ class MigrateSQLMap extends MigrateMap {
if (is_array($source_key)) {
foreach ($source_key as $key_value) {
$fields['sourceid' . $count++] = $key_value;
// If any key value is empty, we can't save - print out and abort
if (empty($key_value)) {
// If any key value is not set, we can't save - print out and abort
if (!isset($key_value)) {
print($message);
return;
}

733
sites/all/modules/contrib/migrate/migrate/plugins/sources/xml.inc
View File

File diff suppressed because it is too large Load Diff

2
sites/all/modules/contrib/migrate/migrate/tests/import/options.test
View File

@@ -21,7 +21,7 @@ class MigrateImportOptionsTest extends DrupalWebTestCase {
parent::setUp('migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testItemLimitOption() {

2
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/comment.test
View File

@@ -21,7 +21,7 @@ class MigrateCommentUnitTest extends DrupalWebTestCase {
parent::setUp('taxonomy', 'image', 'comment', 'migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testCommentImport() {

2
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/node.test
View File

@@ -21,7 +21,7 @@ class MigrateNodeUnitTest extends DrupalWebTestCase {
parent::setUp('list', 'number', 'taxonomy', 'image', 'migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testNodeImport() {

2
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/table.test
View File

@@ -21,7 +21,7 @@ class MigrateTableUnitTest extends DrupalWebTestCase {
parent::setUp('migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testTableImport() {

2
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/term.test
View File

@@ -21,7 +21,7 @@ class MigrateTaxonomyUnitTest extends DrupalWebTestCase {
parent::setUp('taxonomy', 'migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testTermImport() {

15
sites/all/modules/contrib/migrate/migrate/tests/plugins/destinations/user.test
View File

@@ -24,7 +24,7 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
date_default_timezone_set('US/Mountain');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testUserImport() {
@@ -46,6 +46,14 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
$roles[$row->name] = $row->rid;
}
$this->assertEqual(count($roles), 2, t('Both roles imported'));
// Make sure update does not fail (regression test of
// http://drupal.org/node/1872446)
$migration->prepareUpdate();
$migration->processImport();
$num_messages = $migration->getMap()->messageCount();
$this->assertEqual($num_messages, 0, t('No messages generated'));
$migration = Migration::getInstance('WineUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
@@ -106,9 +114,10 @@ class MigrateUserUnitTest extends DrupalWebTestCase {
1, t('Female gender migrated'));
$this->assert(!isset($users['fonzie']->field_migrate_example_gender[LANGUAGE_NONE][0]['value']),
t('Missing gender left unmigrated'));
/* For some reason, this fails on d.o but not in local environments
$this->assert(is_object($users['fonzie']->picture) &&
$users['fonzie']->picture->filename == 'association-individual.png',
t('Picture migrated'));
$users['fonzie']->picture->filename == '200',
t('Picture migrated'));*/
$this->assertNotNull($users['fonzie']->roles[$roles['Taster']], t('Taster role'));
$this->assertNotNull($users['fonzie']->roles[$roles['Vintner']], t('Vintner role'));

2
sites/all/modules/contrib/migrate/migrate/tests/plugins/sources/oracle.test
View File

@@ -31,7 +31,7 @@ class MigrateOracleUnitTest extends DrupalWebTestCase {
}
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testOracleImport() {

67
sites/all/modules/contrib/migrate/migrate/tests/plugins/sources/xml.test
View File

@@ -21,7 +21,7 @@ class MigrateXMLUnitTest extends DrupalWebTestCase {
parent::setUp('taxonomy', 'migrate', 'migrate_example');
// Make sure the migrations are registered.
migrate_get_module_apis();
migrate_static_registration();
}
function testNodeImport() {
@@ -29,22 +29,51 @@ class MigrateXMLUnitTest extends DrupalWebTestCase {
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Region term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineFileCopy');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('File import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineRole');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Role import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineProducerXML');
$result = $migration->processImport();
$migration1 = Migration::getInstance('WineProducerXML');
$result = $migration1->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import returned RESULT_COMPLETED'));
t('Producer node import 1 returned RESULT_COMPLETED'));
$migration2 = Migration::getInstance('WineProducerNamespaceXML');
$result = $migration2->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 2 returned RESULT_COMPLETED'));
$migration3 = Migration::getInstance('WineProducerMultiXML');
$result = $migration3->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 3 returned RESULT_COMPLETED'));
$migration4 = Migration::getInstance('WineProducerMultiNamespaceXML');
$result = $migration4->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 4 returned RESULT_COMPLETED'));
$migration5 = Migration::getInstance('WineProducerXMLPull');
$result = $migration5->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 5 returned RESULT_COMPLETED'));
$migration6 = Migration::getInstance('WineProducerNamespaceXMLPull');
$result = $migration6->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import 6 returned RESULT_COMPLETED'));
// Gather producer nodes, and their corresponding input data
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_producer'), TRUE);
@@ -54,7 +83,7 @@ class MigrateXMLUnitTest extends DrupalWebTestCase {
$producer_nodes[$node->title] = $node;
}
$this->assertEqual(count($producer_nodes), 1,
$this->assertEqual(count($producer_nodes), 10,
t('Counts of producer nodes and input rows match'));
// Test each base node field
@@ -81,10 +110,32 @@ class MigrateXMLUnitTest extends DrupalWebTestCase {
$this->assertEqual($region[0]['tid'], $term->tid,
t('region properly migrated'));
// Test rollback
$result = $migration->processRollback();
// Rollback producer migrations
$result = $migration1->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback returned RESULT_COMPLETED'));
t('Producer node rollback 1 returned RESULT_COMPLETED'));
$result = $migration2->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 2 returned RESULT_COMPLETED'));
$result = $migration3->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 3 returned RESULT_COMPLETED'));
$result = $migration4->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 4 returned RESULT_COMPLETED'));
$result = $migration5->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 5 returned RESULT_COMPLETED'));
$result = $migration6->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback 6 returned RESULT_COMPLETED'));
// Test rollback
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_producer'), TRUE);
$this->assertEqual(count($rawnodes), 0, t('All nodes deleted'));
$count = db_select('migrate_map_wineproducerxml', 'map')