bachir/popsu-d7
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

first import

Browse Source
This commit is contained in:
Bachir Soussi Chiadmi
2015-04-08 11:40:19 +02:00
commit 1bc61b12ad
8435 changed files with 1582817 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
sites/all/modules/migrate/BACKPORT.txt Normal file
View File

@@ -0,0 +1,5 @@
Conventions to make porting changes between Drupal 6 and Drupal 7 easier:
Try to always use specific DBTNG functions such as db_select() instead of the
more general db_query(), which needs to be renamed to dbtng_query() under
Drupal 6.

390
sites/all/modules/migrate/CHANGELOG.txt Normal file
View File

@@ -0,0 +1,390 @@
Migrate 2.4
===========
IMPORTANT: The file migration support (both MigrateDestinationFile and
the file field handler) has been completely refactored since Migrate 2.3 - if you
are importing files as part of your migration, you MUST change your migrations
to properly use the new implementation. See http://drupal.org/node/1540106 for
details on how to use the new handlers and destination.
Features and enhancements
- #1606586 - Add activeUrl public method to MigrateSourceXML.
Bug fixes
- #1614318 - MigrateFileUri failed to pass file object through.
- #1601974 - Remove percent sign fix-up, leave to particular application.
- #1600912 - Base constructor was undermining UI override of displayFunction.
- #1595056 - Handle empty arguments from migrate_status table.
- #1595166 - Support multiple source keys in MigrateList.
Migrate 2.4 Release Candidate 1
===============================
Features and enhancements
- #1587842 - Remove sample CSV files from migrate project.
- #1152878 - drush migrate-analyze command, for source data analysis.
- #1587566 - Support escape option for CSV sources on PHP 5.3.
- #1189328 - UI option to ignore dependencies.
- #1016452 - Support migrating user role assignments by name.
- #653636 - Add migrate-messages command to dump message table.
Bug fixes
- #919108 - Cleaner error-handling when updating destinations that don't exist.
- #1574502 - Coder review, remove dead code.
- #1570536 - Undefined property in term.inc.
Migrate 2.4 Beta 1
==================
Features and enhancements
- #1428166 - Make message columns sortable.
- #1459888 - Removed deprecated source count(), showMessage(), setOutputFunction().
- #1528916 - Link field descriptions to drupal.org documentation.
- #722686 - Support for node statistics.
- #1299646 - Enforce dependencies on rollback.
- #1328408 - Support --group on drush migrate-status.
- #1402822 - Cleaner handling of exceptions from source plugins.
- #1240928 - File handling completely refactored - see note above.
- #1279778 - Improved method for mapping options and subfields.
- #1403044 - Added menu destination plugins.
- #1350284 - Visually group migrations by group in drush.
Bug fixes
- #1569612 - skip_empty should only skip NULL values.
- #1561448 - Proper handling of is_new for users.
- #1205278 - Don't multiply-increment usage counts when preserving files; remove
file_usages on uninstall.
- #1438282 - Handle errors thrown when getting counts.
- #1358318 - Fix dedupe() overdeduping on --update.
- #1538046 - Give highwater marks a chance at preparation before using them.
- #1408248 - Handle errors in complete() cleanly.
- #1541882 - Make sure displayFunction is set before calling it.
- #1529362 - Make proper check if previously imported.
- #1537076 - Apply languages to each value in a multi-value field.
- #1542922 - Fix XML parsing bug.
- #1538508 - Fix notice when mapping parent_name.
- #1537352 - Check limits and status before next().
- #1540120 - Zero not working as defaultValue.
- #1508654 - MigrateSourceSQL needs to respect join aliases.
- #1518076 - UI side of subfield handling.
- #1480762 - Fix cross-database joins to map table.
- #1520688 - setDisplayFunction needs to be static.
- #1518064 - Upgrade needs to check field existence.
- #1518008 - Upgrade uses obsolete db_column_exists API.
- #1308268 - Term migration should link to pre-existing terms.
Migrate 2.3
===========
Features and enhancements
- #1315910 - Added migrate-deregister drush command.
Bug fixes
- #1378114 - Handle case-sensitive term name variations properly.
- #1450950 - Convert user datetimes values to timestamps.
- #1437208 - Handle malformed CSV files quietly.
- #1413350 - Support invoking subprocesses under Drush 5.
- #1404732 - Properly call taxonomy_term handler fields() method.
Migrate 2.3 RC1
===============
Features and enhancements
- #1437076 - Make Migration class field mapping methods public.
- #1430528 - Document importance of migrating term parents first.
- #1418126 - Use node_delete_multiple on migrate-wipe.
- #1424632 - Improve user migration performance by reducing hash count.
- #1414616 - Automatically update migration arguments.
- #1364326 - Default --limit option to items.
- #1337640 - Properly optimize the SQL source query for highwater marks.
- #1366588 - Source plugin support has been refactored to put more of the common
processing in the base class. When implementing a source plugin,
implement performRewind() and getNextRow().
- #519906 - Added support for the core poll module (choices and votes).
- #1355940 - Track all source rows in the map table.
- #1358650 - Truncate text fields according to configured max_length.
- #941440 - Autodetection of source fields when unspecified.
- #1341776 - Add option to skip source counting.
- #1342936 - Pass source key in calls to Migration::createStub().
- #1335110 - Automatically lowercase MD5 passwords.
- #1331912 - Catch exception due to skipped beginProcess().
- #1337810 - Documented hooks in migrate.api.php.
- #1342322 - Added default implementation of prepareRow(), simplifying source plugins.
- #1336534 - Added MigrateSourceXML for efficient handling of large XML files.
- #1330552 - Provide more detail on bad field mappings.
- #1321078 - Display incomplete dependencies in the error message.
- #1321062 - Simple base class for field handlers.
- #1314448 - Added tnid to node destination handler.
- #1231492 - Added source handler for retrieving content from file directories.
- #1290706 - Centralize loading of XML.
- #1295040 - Support --update functionality in UI.
- #1304444 - Added static displayMessage(), deprecating showMessage().
- #1290706 - Centralize loading of XML.
- #1254398 - Prevent accidental emailing during migration.
Bug fixes
- #1447368 - Handle purely numeric PHP memory_limit.
- #1432802 - Default $options to prevent PHP 5.4 error.
- #1333656 - Remove content type/fields when uninstalling migrate_example_baseball.
- #1416012 - Cleanly handle NULL source key values in saveIDMapping().
- #1419086 - Better error handling in MigrateXMLReader::next().
- #1422114 - ID list array must be imploded for queries.
- #1372204 - Fix loss of pictures when updating users.
- #1364034 - Add cache_key option to sources, for disambiguation.
- #1354162 - Make sure messages are retrieved from the right connection.
- #1352648 - Update mode reapplied on each batch in UI.
- #1349246 - Postgres error updating migrate_log.
- #1342686 - Generalized needs-update support.
- #1341598 - filename/filemime not set in file_link/file_blob cases.
- #1342736 - drush options don't need -- prefix.
- #1340204 - MigrateDestinationFile::prepare() didn't call parent::prepare().
- #1336880 - Fix XPath handling in MigrateItemsXML::getIDSFromXML.
- #1334546 - Validate compound keys passed to handleSourceMigration().
- #1305338 - Corrected type-hint in MigrateSourceSQL constructor.
- #1305910 - MigrateDestinationTable now tracks inserted/updated records.
- #1284592 - Fixed error on empty term parents.
- #1300258 - Fixed error on unfulfilled term references.
- #1292462 - Group missing from drush options.
- #1290122 - Force destinations to implement __toString().
- #1290120 - Warnings updating users if roles not set.
- #1290118 - Warning in MigrateItemJSON::getItem()
- #1133096 - Failure to clear currentRow in MigrateSourceMultiItems.
Migrate 2.2
===========
Bug fixes
- #1265514 - Fixed error message for failed XML load in MigrateItemsXML.
Migrate 2.2 RC2
===============
Features and enhancements
- #1240928 - Generalize file handlers, to ease media module support.
- #1226768 - Improved exception handling.
- #1230294 - Get correct count of CSV with embedded newlines.
- #1185632 - Handle (potentially multiple) headers in CSV source plugin.
Bug fixes
- #1231398 - Add map data to row in Oracle source plugin.
- #1225636 - File field: fixed warnings, added docs.
- #1227130 - On import, clear messages before applyMappings().
- #1195802 - Fixed SQL Server problem handling end of batch; brought Oracle/SQL
Server implementations in line with each other.
- #1205278 - Fixed file entity rollback to preserve files when requested.
- #1223468 - Make sure getIDsFromXML always returns an array.
- #1223734 - Fixed bogus assignment of uid to file fields.
- #1223756 - Fix warnings with file_blob when file exists in file_managed.
- #1216796 - Make sure file_replace argument exists.
- #1195802 - Fixed prepareRow() support for MSSQL plugin.
Migrate 2.2 RC1
===============
Features and enhancements
- #1210152 - Improved error messages on file field migration.
- #1212818 - migrate_ui support for one-step rollback-and-import.
- #1218244 - Support for is_new for users.
- #1216796 - Use file_destination() to generate destination filespec.
- #719650 - Implemented ability to assign migrations to groups, and run by group.
- #1201762 - Implemented built-in caching for source counts, and flag to enable it.
- #1205278 - Added preserve_files argument to MigrateFileFieldHandler.
- #1205278 - Added preserve_files option to MigrateDestinationFile.
- #1078368 - Implemented create_term argument for term references, to lazy-create terms.
- #1199150 - Added file_fid function to file field migration.
- #1195802 - Added prepareRow() support for MSSQL plugin.
- #1195784 - Added verification of required extension to MSSQL plugin.
- #1180188 - Added ability to save BLOB data to file fields and file entities.
- #1181136 - Provided mechanism to handle incoming MD5-encrypted passwords.
- #1181720 - Add item/second limiting to UI.
- #1181826 - Implemented hook_hook_info(), so implementing modules can define
hook_migrate_api() in example.migrate.inc.
- #1179464 - Added capability of disabling handlers.
- #1175094 - Added support for field handler complete() methods.
- #1176930 - Renamed migrate-descriptions to migrate-mappings, enhanced to export to CSV.
- #1169724 - Implemented source plugin for Oracle.
Bug fixes
- #1210076 - Filled out parameter docs for MigrateFileFieldHandler::buildFileArray().
- #1185046 - Wine example mistakenly said GROUP_CONCAT could only be used once.
- #1202234 - Fix translatable field language handling.
- #1184538 - Fixed term field allowed values in migrate_example.
- #1181652 - Fixed MigrateDestinationTable to work on update.
- #1174934 - Support multi-column keys for table_copy destination.
- #1176790 - Prevent multiple handlers from messing up fields info.
- #1169796 - Handle example upgrades if image field is missing.
Migrate 2.1
===========
Features and enhancements
- #1025754 - Added support for multi-value source keys to handleSourceMigration().
- #1161584 - Added lookupSourceID() and lookupDestinationID() to MigrateMap.
Bug fixes
- #1161612 - Handle integer highwater marks properly.
- #1159274 - Handle spaces in URLs for MigrateDestinationFile.
- #1161812 - Added handling for NULL watchdog variables.
- #1009708 - Fixed E_STRICT notice on MigrateDestinationRole::getKeySchema().
- #1161590 - Made $migration optional for handleSourceMigration(), createStubWrapper(),
lookupDestinationID().
- #1161482 - Handle NULL source_field mappings.
- #1156972 - Do not include message table in source queries.
Migrate 2.1 Beta 1
==================
API change:
Any field handler prepare() or complete() methods you have defined must be changed
to remove stdClass for the $entity argument. I.e., prepare(stdClass $entity, stdClass $row)
should now be prepare($entity, stdClass $row).
Features and enhancements
- #1017246 - Added support for running migrations from the dashboard.
- #1004812 - Added schema-driven table destination plugin.
- #1009708 - Added role destination plugin.
- #737170 - Added support for field level callbacks.
- #1005090 - Modified filefield property import to use JSON input.
- #730980 - Added more detailed reporting on import.
- #1142384 - Extended file field support to copy from remote URLs.
- #1138096 - Added MigrateSourceMultiItems class for self-contained XML sources.
- #1101586 - Add shortcut methods for adding several field mappings at once.
- #1101592 - Replace --itemlimit with --limit, supporting time limits as well.
- #1139080 - Added example and test for importing profile pictures over http.
Bug fixes
- #1155740 - Make remote file fetch HTTP version independent
- #1037872 - Deal with updates with idlists when mapping nids directly.
- #943546 - Make sure both slashes and backslashes are trimmed where necessary.
- #1146366 - Fixed rollback of table destinations.
- #1148474 - Enforce unique IDs in MigrateItemsXML.
- #1132034 - Don't assume entity is a stdClass.
- #753284 - Fixed systemOfRecord==DESTINATION updates for nodes and users.
- #1142104 - Proper prefixing of tables when updating comment statistics.
- #1136852 - Fixed cross-threaded test in preImport().
- #1134858 - Improved validation for XML imports.
- #1134300 - Add parameter $messages_only to MigrateMap::delete().
- #1134506 - Removed obsolete views_alter() hook.
- #1133096 - CSV source always processed last row, fix to clear currentRow.
- #1124318 - Properly use source_field arguments for file field attributes.
- #1133030 - Undefined constant in drush migrate-fields-destination.
- #1128532 - Handle call to watchdog when Migration class not defined.
- #1126108 - Fix undefined variables in XML error handling.
- #1117602 - Bad counts reported on bulk rollback.
- #1118480 - Document _name arguments to file fields.
- #1108700 - Remove obsolete references to prepare().
- #1107546 - Check file presence with is_file() instead of file_exists().
- #1097136 - Replace list_number with list_integer and list_float in set of
supported simple field types.
- #1103042 - Rollback broken with multi-valued keys.
Migrate 2.0
===========
No changes since RC3.
Migrate 2.0 Release Candidate 3
===============================
Features and enhancements
Added JSON source plugin.
Bug fixes
- #867940 - Prevent overwriting of migrated files with common basenames.
- #1072170 - Allow revision setting to be overridden.
- #1073770 - Quietly skip missing files in MigrateFileFieldHandler.
- #1079416 - postRollback called outside of rollback operation.
- #1070894 - Apply tokens when saving files.
- #1067918 - Static cache of migrations messes up simpletests.
- #1053798 - Message meant for debug not marked 'debug'.
- #1063926 - prepareRow() call missing from MigrateSourceList.
- #1062200 - Validate arguments for getFieldLanguage().
- #1061284 - Appropriately translate watchdog severities to migrate severities.
Migrate 2.0 Release Candidate 2
===============================
Bug fixes
- migrate_example error when auto_nodetitle module not present.
- #1053798 - Missing 'debug' on showMessage.
Migrate 2.0 Release Candidate 1
===============================
Features and enhancements
- #919108 - Add deleteDestination() for maps; cache migration list.
- #1005090 - Support multiple values for file fields.
- #996086 - Add prepareRollback and completeRollback methods for entities.
- #1039882 - Pass client migration to stub creation. Note that $migration->values
(the destination object being built) is now $migration->destinationValues - any
migrations referencing this member will need to change.
- #946350 - Accept arrays in sourceMigration();
Let a node migration set node_revisions.uid. That's the 'last edited by' user
Bug fixes
- #1053798 - Consistent use of error codes.
- #1053526 - Machine names in drush commands now case-insensitive.
- #914440 - Enhance multilingual support.
- #1014648 - Defaulting of file field subfields.
- #1037872 - Make sure is_new is off for previously migrated content.
- #919706 - Fix drush migrate-descriptions command.
- #1027468 - Limit map/message table name lengths.
- #1045646 - Handle PHP memory_limit of -1.
- #1050348 - Check map table existence in correct DB.
- #1028824 - Errors during registry rebuild trigger fatal error.
Fix highwater handling to deal with duplicate values.
- #998604 - Fixed dedupe.
- #998586 - Improved feedback for disabled migrations.
- #972382 - Fix MigrateDestinationTableCopy.
- #962120 - Support idlist on rollback.
- #1017256 - Document case-sensitivity in beer.inc.
- #1013844 - Better error handling in getInstance().
- #1008228 - migrate_example broken by list changes in D7 RC3.
Migrate 2.0 Beta 3
==================
Features and enhancements
- #989200 - Support "dynamic" migrations. Changes to be aware of:
All modules implementing migration classes must now implement hook_migrate_api()
(see migrate_migrate_api() for an example).
Dependencies and sourceMigrations must now be expressed in terms of
machine name rather than class name.
MigrationBase::getInstance now takes a machine name rather than a class name.
Migration class names are no longer required to end in 'Migration'.
- #992898 - Pass options to source and destination constructors as arrays.
File destinations (i.e., migrating directly to the file_managed table, with
optional copying of the files themselves) are now supported.
Allow migration of comment enable/disable.
Check max_execution_time as well as memory_limit, for graceful exit when
max_execution_time is in play.
Add dedupe() method for a field mapping.
Apply MigrateBase::timestamp() to comment created/changed
Bug fixes
- #1006272 - Remove lengths on int fields in schema (breaks Postgres)
- #1005030 - Replace eval() with proper call_user_func().
- #1005360 - Skip .test files in registry scan
- #1004268 - Don't munge date separators
- prepareRow() hook skips current row only if boolean FALSE is returned.
- #989622 - Fix undefined variable in MigrateDestinationNode::import().
- #984294 - Make migrate dashbard a local task.
needs_update => needs-update to match drush conventions.
- #959400 - Better way to get vid for vocabulary.
- #984336 - Additional fields and vocabularies not reported on migration info page
- #990960 - Initialize fields array for profiles
- #993684 - hook_requirements should always return array.
- #983968 - Bogus settings of ->uid
Migrate 2.0 Beta 2
==================
Bug fixes
- #984294 - Migration info pages are blank
Migrate 2.0 Beta 1
==================
Version 2 of the Migrate module is an entirely new implementation - it is
pointless to list changes since Migrate 1.

339
sites/all/modules/migrate/LICENSE.txt Normal file
View File

@@ -0,0 +1,339 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.

40
sites/all/modules/migrate/README.txt Normal file
View File

@@ -0,0 +1,40 @@
The Migrate module provides a flexible framework for migrating content into Drupal
from other sources (e.g., when converting a web site from another CMS to Drupal).
Out-of-the-box, support for creating Drupal nodes, taxonomy terms, comments, and
users are included. Plugins permit migration of other types of content.
Usage
-----
Documentation is at http://drupal.org/node/415260. To get started, enable the
migrate_example module and browse to admin/content/migrate to see its dashboard.
The code for this migration is in migrate_example/beer.inc (advanced examples are
in wine.inc). Mimic that file in order to specify your own migrations.
The Migrate module itself has support for migration into core objects. Support
for migration involving contrib modules is in the migrate_extras module.
Known issues
------------
A user migration with systemOfRecord == DESTINATION will drop pictures from user
records due to core bug http://drupal.org/node/935592 - the simpletests report an
error reflecting this. We have not developed a work-around.
Upgrading
---------
Do not attempt to upgrade directly from Migrate 1 to Migrate 2! There is no
automated path to upgrade - your migrations (formerly known as "content sets")
must be reimplemented from scratch. It is recommended that projects using
Migrate 1 stay with Migrate 1, and that Migrate 2 be used for any new migration
projects.
Acknowledgements
----------------
Much of the Migrate module functionality was sponsored by Cyrve, for its clients GenomeWeb
(http://www.genomeweb.com), The Economist (http://www.economist.com), and Examiner.com
(http://www.examiner.com).
Authors
-------
Mike Ryan - http://drupal.org/user/4420
Moshe Weitzman - http://drupal.org/user/23

1015
sites/all/modules/migrate/includes/base.inc Normal file
View File

File diff suppressed because it is too large Load Diff

115
sites/all/modules/migrate/includes/destination.inc Normal file
View File

@@ -0,0 +1,115 @@
<?php
/**
* @file
* Defines base for migration destinations.
*/
/**
* Abstract base class for destination handling.
*
* Derived classes are expected to define __toString(), returning a string
* describing the type of destination and significant options. See
* MigrateDestinationEntity for an example.
*/
abstract class MigrateDestination {
/**
* To support MigrateSQLMap maps, derived destination classes should return
* schema field definition(s) corresponding to the primary key of the destination
* being implemented. These are used to construct the destination key fields
* of the map table for a migration using this destination.
*
* abstract static public function getKeySchema()
*/
/**
* Derived classes must implement __toString().
*
* @return string
* Description of the destination being migrated into
*/
abstract public function __toString();
/**
* Derived classes must implement fields(), returning a list of available
* destination fields.
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
abstract public function fields();
/**
* Derived classes must implement either bulkRollback or rollback() according to
* the signatures below, to rollback (usually by deletion) previously-migrated
* items.
*
* $ids is an array of single-field keys to be deleted
* abstract public function bulkRollback(array $ids);
*
* $key is an array of fields keying a single entry to delete
* abstract public function rollback(array $key);
*/
/**
* Derived classes must implement import(), to construct one new object (pre-pppulated
* using field mappings in the Migration). It is expected to call prepare and
* complete handlers, passing them $row (the raw data from the source).
*/
abstract public function import(stdClass $object, stdClass $row);
/**
* Derived classes may implement preImport() and/or postImport(), to do any
* processing they need done before or after looping over all source rows.
* Similarly, preRollback() or postRollback() may be implemented.
*
* abstract public function preImport();
* abstract public function postImport();
* abstract public function preRollback();
* abstract public function postRollback();
*/
/**
* Maintain stats on the number of destination objects created or updated.
*
* @var int
*/
protected $numCreated = 0;
public function getCreated() {
return $this->numCreated;
}
protected $numUpdated = 0;
public function getUpdated() {
return $this->numUpdated;
}
/**
* Reset numCreated and numUpdated back to 0.
*/
public function resetStats() {
$this->numCreated = 0;
$this->numUpdated = 0;
}
/**
* Null constructor
*/
public function __construct() {
}
}
/**
* All destination handlers should be derived from MigrateDestinationHandler
*/
abstract class MigrateDestinationHandler extends MigrateHandler {
// abstract function arguments(...)
/**
* Any one or more of these methods may be implemented
*/
//abstract public function fields();
//abstract public function prepare($entity, stdClass $row);
//abstract public function complete($entity, stdClass $row);
}

18
sites/all/modules/migrate/includes/exception.inc Normal file
View File

@@ -0,0 +1,18 @@
<?php
/**
* @file
* Custom exception class for the migrate module.
*/
class MigrateException extends Exception {
protected $level;
public function getLevel() {
return $this->level;
}
public function __construct($message, $level = Migration::MESSAGE_ERROR) {
$this->level = $level;
parent::__construct($message);
}
}

202
sites/all/modules/migrate/includes/field_mapping.inc Normal file
View File

@@ -0,0 +1,202 @@
<?php
/**
* @file
* The MigrateFieldMapping class - tracking mappings between source and
* destination.
*/
class MigrateFieldMapping {
/**
* Destination field name for the mapping. If empty, the mapping is just a
* stub for annotating the source field.
*
* @var string
*/
protected $destinationField;
public function getDestinationField() {
return $this->destinationField;
}
/**
* Source field name for the mapping. If empty, the defaultValue will be
* applied.
*
* @var string
*/
protected $sourceField;
public function getSourceField() {
return $this->sourceField;
}
/**
* Default value for simple mappings, when there is no source mapping or the
* source field is empty. If both this and the sourceField are omitted, the
* mapping is just a stub for annotating the destination field.
*
* @var mixed
*/
protected $defaultValue;
public function getDefaultValue() {
return $this->defaultValue;
}
/**
* Separator string. If present, the destination field will be set up as an
* array of values exploded from the corresponding source field.
*
* @var string
*/
protected $separator;
public function getSeparator() {
return $this->separator;
}
/**
* Class name of source migration for a field. If present, the value in the
* source field is considered to be a source ID in the mapping table of this
* migration, and the corresponding destination ID will be retrieved.
*
* @var mixed
* An array of source migrations, or string for a single migration.
*/
protected $sourceMigration;
public function getSourceMigration() {
return $this->sourceMigration;
}
/**
* Array of callbacks to be called on a source value.
*
* @var string
*/
protected $callbacks = array();
public function getCallbacks() {
return $this->callbacks;
}
/**
* An associative array with keys:
* - table: The table for querying for a duplicate.
* - column: The column for querying for a duplicate.
*
* @todo: Let fields declare this data and a replacement pattern. Then
* developers won't have to specify this.
*
* @var string
*/
protected $dedupe;
public function getDedupe() {
return $this->dedupe;
}
/**
* Argument overrides. If present this will be an array, keyed by
* a field API array key, with one or both of these entries:
* 'source_field' - Name of the source field in the incoming row containing the
* value to be assigned
* 'default_value' - A constant value to be assigned in the absence of source_field
*
* @var array
*/
protected $arguments;
public function getArguments() {
return $this->arguments;
}
protected $description = '';
public function getDescription() {
return $this->description;
}
protected $issueGroup;
public function getIssueGroup() {
return $this->issueGroup;
}
protected $issueNumber;
public function getIssueNumber() {
return $this->issueNumber;
}
protected $issuePriority = self::ISSUE_PRIORITY_OK;
public function getIssuePriority() {
return $this->issuePriority;
}
const ISSUE_PRIORITY_OK = 1;
const ISSUE_PRIORITY_LOW = 2;
const ISSUE_PRIORITY_MEDIUM = 3;
const ISSUE_PRIORITY_BLOCKER = 4;
public static $priorities = array();
public function __construct($destination_field, $source_field) {
// Must have one or the other
if (!$destination_field && !$source_field) {
throw new Exception('Field mappings must have a destination field or a source field');
}
$this->destinationField = $destination_field;
$this->sourceField = $source_field;
$this->issueGroup = t('Done');
if (count(self::$priorities) == 0) {
self::$priorities[self::ISSUE_PRIORITY_OK] = t('OK');
self::$priorities[self::ISSUE_PRIORITY_LOW] = t('Low');
self::$priorities[self::ISSUE_PRIORITY_MEDIUM] = t('Medium');
self::$priorities[self::ISSUE_PRIORITY_BLOCKER] = t('Blocker');
}
}
public function defaultValue($default_value) {
$this->defaultValue = $default_value;
return $this;
}
public function separator($separator) {
$this->separator = $separator;
return $this;
}
public function sourceMigration($source_migration) {
$this->sourceMigration = $source_migration;
return $this;
}
public function callbacks($callbacks) {
$this->callbacks = func_get_args();
return $this;
}
public function dedupe($table, $column) {
$this->dedupe = array('table' => $table, 'column' => $column);
return $this;
}
public function arguments($arguments) {
$this->arguments = $arguments;
return $this;
}
public function description($text) {
$this->description = $text;
return $this;
}
public function issueGroup($group) {
if (!$group) {
$group = t('Done');
}
$this->issueGroup = $group;
return $this;
}
public function issueNumber($number) {
$this->issueNumber = $number;
return $this;
}
public function issuePriority($priority) {
$this->issuePriority = $priority;
return $this;
}
}

109
sites/all/modules/migrate/includes/group.inc Normal file
View File

@@ -0,0 +1,109 @@
<?php
/**
* @file
* Definition for a migration group.
*/
class MigrateGroup {
/**
* The name of the group - used to identify it in drush commands.
*
* @var string
*/
protected $name;
public function getName() {
return $this->name;
}
/**
* List of groups this group is dependent on.
*
* @var array
*/
protected $dependencies = array();
public function getDependencies() {
return $this->dependencies;
}
/**
* The central list of all known groups, keyed by group name.
*
* @var array
*/
static protected $groupList = array();
static public function groups() {
$groups = array();
$dependent_groups = array();
$required_groups = array();
foreach (self::$groupList as $name => $group) {
$dependencies = $group->getDependencies();
if (count($dependencies) > 0) {
// Set groups with dependencies aside for reordering
$dependent_groups[$name] = $group;
$required_groups += $dependencies;
}
else {
// No dependencies, just add
$groups[$name] = $group;
}
}
$iterations = 0;
while (count($dependent_groups) > 0) {
if ($iterations++ > 20) {
$group_names = implode(',', array_keys($dependent_groups));
throw new MigrateException(t('Failure to sort migration list - most likely due ' .
'to circular dependencies involving groups !group_names',
array('!group_names' => $group_names)));
}
foreach ($dependent_groups as $name => $group) {
$ready = TRUE;
// Scan all the dependencies for this group and make sure they're all
// in the final list
foreach ($group->getDependencies() as $dependency) {
if (!isset($groups[$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this group to the final list
$groups[$name] = $group;
unset($dependent_groups[$name]);
}
}
}
return $groups;
}
/**
* Basic constructor.
*
* @param string $name
* Group name.
*
* @param array $dependencies
* List of dependent groups.
*/
public function __construct($name, $dependencies = array()) {
$this->name = $name;
$this->dependencies = $dependencies;
}
/**
* Retrieve (creating if necessary) an instance of the named group.
*
* @param string $name
* Group name.
*
* @param array $dependencies
* List of dependent groups.
*/
static public function getInstance($name, $dependencies = array()) {
if (empty(self::$groupList[$name])) {
self::$groupList[$name] = new MigrateGroup($name, $dependencies);
}
return self::$groupList[$name];
}
}

57
sites/all/modules/migrate/includes/handler.inc Normal file
View File

@@ -0,0 +1,57 @@
<?php
/**
* @file
* Defines the base class for destination handlers.
*/
/**
* Abstract base class for destination handlers. Handler objects are expected
* to implement appropriate methods (e.g., prepare, complete, or fields).
*/
abstract class MigrateHandler {
/**
* List of other handler classes which should be invoked before the current one.
*
* @var array
*/
protected $dependencies = array();
public function getDependencies() {
return $this->dependencies;
}
/**
* List of "types" handled by this handler. Depending on the kind of handler,
* these may be destination types, field types, etc.
*
* @var array
*/
protected $typesHandled = array();
public function getTypesHandled() {
return $this->typesHandled;
}
/**
* Register a list of types handled by this class
*
* @param array $types
*/
protected function registerTypes(array $types) {
// Make the type names the keys
foreach ($types as $type) {
$type = drupal_strtolower($type);
$this->typesHandled[$type] = $type;
}
}
/**
* Does this handler handle the given type?
*
* @param boolean $type
*/
public function handlesType($type) {
return isset($this->typesHandled[strtolower($type)]);
}
abstract public function __construct();
}

158
sites/all/modules/migrate/includes/map.inc Normal file
View File

@@ -0,0 +1,158 @@
<?php
/**
* @file
* Defines the framework for map and message handling.
*/
/**
* We implement the Iterator interface to support iteration over the map table
* for the purpose of rollback.
*/
abstract class MigrateMap implements Iterator {
/**
* Codes reflecting the current status of a map row.
*/
const STATUS_IMPORTED = 0;
const STATUS_NEEDS_UPDATE = 1;
const STATUS_IGNORED = 2;
const STATUS_FAILED = 3;
/**
* Arrays of key fields for the source and destination. Array keys are the
* field names - array values are specific to the concrete map class.
*
* @var array
*/
protected $sourceKey, $destinationKey;
abstract public function getSourceKey();
abstract public function getDestinationKey();
/**
* Mapping from field names to the map/message table key names (e.g.,
* from input_field to sourceid1, or from nid to destid1)
*
* @var array
*/
protected $sourceKeyMap, $destinationKeyMap;
/**
* Boolean determining whether to track last_imported times in map tables
*
* @var boolean
*/
protected $trackLastImported = FALSE;
/**
* Save a mapping from the key values in the source row to the destination
* keys.
*
* @param $source_row
* @param $dest_ids
* @param $status
*/
abstract public function saveIDMapping(stdClass $source_row, array $dest_ids,
$status = MigrateMap::STATUS_IMPORTED);
/**
* Record a message related to a source record
*
* @param array $source_key
* Source ID of the record in error
* @param string $message
* The message to record.
* @param int $level
* Optional message severity (defaults to MESSAGE_ERROR).
*/
abstract public function saveMessage($source_key, $message, $level = MigrationBase::MESSAGE_ERROR);
/**
* Prepare to run a full update - mark all previously-imported content as
* ready to be re-imported.
*/
abstract public function prepareUpdate();
/**
* Report the number of processed items in the map
*/
abstract public function processedCount();
/**
* Report the number of imported items in the map
*/
abstract public function importedCount();
/**
* Report the number of items that failed to import
*/
abstract public function errorCount();
/**
* Report the number of messages
*/
abstract public function messageCount();
/**
* Delete the map and message entries for a given source record
*
* @param array $source_key
*/
abstract public function delete(array $source_key, $messages_only = FALSE);
/**
* Delete the map and message entries for a given destination record
*
* @param array $destination_key
*/
abstract public function deleteDestination(array $destination_key);
/**
* Delete the map and message entries for a set of given source records.
*
* @param array $source_keys
*/
abstract public function deleteBulk(array $source_keys);
/**
* Clear all messages from the map.
*/
abstract public function clearMessages();
/**
* Retrieve map data for a given source or destination item
*/
abstract public function getRowBySource(array $source_id);
abstract public function getRowByDestination(array $destination_id);
/**
* Retrieve an array of map rows marked as needing update.
*/
abstract public function getRowsNeedingUpdate($count);
/**
* Given a (possibly multi-field) destination key, return the (possibly multi-field)
* source key mapped to it.
*
* @param array $destination_id
* Array of destination key values.
* @return array
* Array of source key values, or NULL on failure.
*/
abstract public function lookupSourceID(array $destination_id);
/**
* Given a (possibly multi-field) source key, return the (possibly multi-field)
* destination key it is mapped to.
*
* @param array $source_id
* Array of source key values.
* @return array
* Array of destination key values, or NULL on failure.
*/
abstract public function lookupDestinationID(array $source_id);
/**
* Remove any persistent storage used by this map (e.g., map and message tables)
*/
abstract public function destroy();
}

1288
sites/all/modules/migrate/includes/migration.inc Normal file
View File

File diff suppressed because it is too large Load Diff

368
sites/all/modules/migrate/includes/source.inc Normal file
View File

@@ -0,0 +1,368 @@
<?php
/**
* @file
* Define base for migration sources.
*/
/**
* Abstract base class for source handling.
*
* Derived classes are expected to define __toString(), returning a string
* describing the source and significant options. See
* MigrateSourceSQL for an example.
*/
abstract class MigrateSource implements Iterator {
/**
* The current row from the quey
*
* @var stdClass
*/
protected $currentRow;
/**
* The primary key of the current row
*
* @var array
*/
protected $currentKey;
public function getCurrentKey() {
return $this->currentKey;
}
/**
* The Migration class currently invoking us, during rewind() and next().
*
* @var Migration
*/
protected $activeMigration;
/**
* The MigrateMap class for the current migration.
*
* @var MigrateMap
*/
protected $activeMap;
/**
* Number of rows intentionally ignored (prepareRow() returned FALSE)
*
* @var int
*/
protected $numIgnored = 0;
public function getIgnored() {
return $this->numIgnored;
}
/**
* Number of rows we've at least looked at.
*
* @var int
*/
protected $numProcessed = 0;
public function getProcessed() {
return $this->numProcessed;
}
/**
* Reset numIgnored back to 0.
*/
public function resetStats() {
$this->numIgnored = 0;
}
/**
* Information on the highwater mark for the current migration, if any.
*
* @var array
*/
protected $highwaterField;
/**
* List of source IDs to process.
*
* @var array
*/
protected $idList = array();
/**
* Derived classes must implement fields(), returning a list of available
* source fields.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
abstract public function fields();
/**
* Whether this instance should cache the source count.
*
* @var boolean
*/
protected $cacheCounts = FALSE;
/**
* Key to use for caching counts.
*
* @var string
*/
protected $cacheKey;
/**
* Whether this instance should not attempt to count the source.
*
* @var boolean
*/
protected $skipCount = FALSE;
/**
* By default, next() will directly read the map row and add it to the data
* row. A source plugin implementation may do this itself (in particular, the
* SQL source can incorporate the map table into the query) - if so, it should
* set this TRUE so we don't duplicate the effort.
*
* @var bool
*/
protected $mapRowAdded = FALSE;
/**
* Return a count of available source records, from the cache if appropriate.
* Returns -1 if the source is not countable.
*
* @param boolean $refresh
*/
public function count($refresh = FALSE) {
if ($this->skipCount) {
return -1;
}
if (!isset($this->cacheKey)) {
$this->cacheKey = md5((string)$this);
}
// If a refresh is requested, or we're not caching counts, ask the derived
// class to get the count from the source.
if ($refresh || !$this->cacheCounts) {
$count = $this->computeCount();
cache_set($this->cacheKey, $count, 'cache');
}
else {
// Caching is in play, first try to retrieve a cached count.
$cache_object = cache_get($this->cacheKey, 'cache');
if (is_object($cache_object)) {
// Success
$count = $cache_object->data;
}
else {
// No cached count, ask the derived class to count 'em up, and cache
// the result
$count = $this->computeCount();
cache_set($this->cacheKey, $count, 'cache');
}
}
return $count;
}
/**
* Derived classes must implement computeCount(), to retrieve a fresh count of
* source records.
*/
//abstract public function computeCount();
/**
* Class constructor.
*
* @param array $options
* Optional array of options.
*/
public function __construct($options = array()) {
if (!empty($options['cache_counts'])) {
$this->cacheCounts = TRUE;
}
if (!empty($options['skip_count'])) {
$this->skipCount = TRUE;
}
if (!empty($options['cache_key'])) {
$this->cacheKey = $options['cache_key'];
}
}
/**
* Default implementations of Iterator methods - many derivations will find
* these adequate and will only need to implement rewind() and next()
*/
/**
* Implementation of Iterator::current() - called when entering a loop
* iteration, returning the current row
*/
public function current() {
return $this->currentRow;
}
/**
* Implementation of Iterator::key - called when entering a loop iteration, returning
* the key of the current row. It must be a scalar - we will serialize
* to fulfill the requirement, but using getCurrentKey() is preferable.
*/
public function key() {
return serialize($this->currentKey);
}
/**
* Implementation of Iterator::valid() - called at the top of the loop, returning
* TRUE to process the loop and FALSE to terminate it
*/
public function valid() {
return !is_null($this->currentRow);
}
/**
* Implementation of Iterator::rewind() - subclasses of MigrateSource should
* implement performRewind() to do any class-specific setup for iterating
* source records.
*/
public function rewind() {
$this->activeMigration = Migration::currentMigration();
$this->activeMap = $this->activeMigration->getMap();
$this->numProcessed = 0;
$this->numIgnored = 0;
$this->highwaterField = $this->activeMigration->getHighwaterField();
if ($this->activeMigration->getOption('idlist')) {
$this->idList = explode(',', $this->activeMigration->getOption('idlist'));
}
else {
$this->idList = array();
}
migrate_instrument_start(get_class($this) . ' performRewind');
$this->performRewind();
migrate_instrument_stop(get_class($this) . ' performRewind');
$this->next();
}
/**
* Implementation of Iterator::next() - subclasses of MigrateSource should
* implement getNextRow() to retrieve the next valid source rocord to process.
*/
public function next() {
$this->currentKey = NULL;
$this->currentRow = NULL;
migrate_instrument_start(get_class($this) . ' getNextRow');
while ($row = $this->getNextRow()) {
migrate_instrument_stop(get_class($this) . ' getNextRow');
// Populate the source key for this row
foreach ($this->activeMap->getSourceKey() as $field_name => $field_schema) {
$this->currentKey[$field_name] = $row->$field_name;
}
// Pick up the existing map row, if any, unless getNextRow() did it.
if (!$this->mapRowAdded) {
$map_row = $this->activeMap->getRowBySource($this->currentKey);
// Add map info to the row, if present
if ($map_row) {
foreach ($map_row as $field => $value) {
$field = 'migrate_map_' . $field;
$row->$field = $value;
}
}
}
// First, determine if this row should be passed to prepareRow(), or skipped
// entirely. The rules are:
// 1. If there's an explicit idlist, that's all we care about (ignore
// highwaters and map rows).
$prepared = FALSE;
if (!empty($this->idList)) {
if (in_array(reset($this->currentKey), $this->idList)) {
// In the list, fall through.
}
else {
// Not in the list, skip it
$this->currentRow = NULL;
continue;
}
}
// 2. If the row is not in the map (we have never tried to import it before),
// we always want to try it.
elseif (!isset($row->migrate_map_sourceid1)) {
// Fall through
}
// 3. If the row is marked as needing update, pass it.
elseif ($row->migrate_map_needs_update == MigrateMap::STATUS_NEEDS_UPDATE) {
// Fall through
}
// 4. At this point, we have a row which has previously been imported and
// not marked for update. If we're not using highwater marks, then we
// will not take this row.
elseif (empty($this->highwaterField)) {
// No highwater, skip
$this->currentRow = NULL;
continue;
}
// 5. So, we are using highwater marks. Take the row if its highwater field
// value is greater than the saved marked, otherwise skip it.
else {
// Call prepareRow() here, in case the highwaterField needs preparation
if ($this->prepareRow($row) !== FALSE) {
if ($row->{$this->highwaterField['name']} > $this->activeMigration->getHighwater()) {
$this->currentRow = $row;
break;
}
else {
// Skip
$this->currentRow = NULL;
continue;
}
}
else {
$this->currentRow = NULL;
}
$prepared = TRUE;
}
// Allow the Migration to prepare this row. prepareRow() can return boolean
// FALSE to ignore this row.
if (!$prepared) {
if ($this->prepareRow($row) !== FALSE) {
// Finally, we've got a keeper.
$this->currentRow = $row;
break;
}
else {
$this->currentRow = NULL;
}
}
}
migrate_instrument_stop(get_class($this) . ' getNextRow');
if (!$this->currentRow) {
$this->currentKey = NULL;
}
}
/**
* Give the calling migration a shot at manipulating, and possibly rejecting,
* the source row.
*
* @return bool
* FALSE if the row is to be skipped.
*/
protected function prepareRow($row) {
migrate_instrument_start(get_class($this->activeMigration) . ' prepareRow');
$return = $this->activeMigration->prepareRow($row);
migrate_instrument_stop(get_class($this->activeMigration) . ' prepareRow');
// We're explicitly skipping this row - keep track in the map table
if ($return === FALSE) {
$this->activeMigration->getMap()->saveIDMapping($row, array(NULL),
MigrateMap::STATUS_IGNORED);
$this->numIgnored++;
$this->currentRow = NULL;
$this->currentKey = NULL;
}
else {
$return = TRUE;
}
$this->numProcessed++;
return $return;
}
}

34
sites/all/modules/migrate/includes/team.inc Normal file
View File

@@ -0,0 +1,34 @@
<?php
/**
* @file
* Info on migration team members. Display-only at the moment, but eventually
* there should be notification features.
*/
class MigrateTeamMember {
protected $name;
public function getName() {
return $this->name;
}
protected $emailAddress;
public function getEmailAddress() {
return $this->emailAddress;
}
protected $group;
public function getGroup() {
return $this->group;
}
public function __construct($name, $email_address, $group) {
$this->name = $name;
$this->emailAddress = $email_address;
$this->group = $group;
}
public function contact($subject, $text) {
// TODO
}
}

27
sites/all/modules/migrate/migrate.api.php Normal file
View File

@@ -0,0 +1,27 @@
<?php
/**
* @file
* Documentation for hooks defined by Migrate.
*/
/**
* Registers your module as an implementor of Migrate-based classes.
*/
function hook_migrate_api() {
$api = array(
'api' => 2,
);
return $api;
}
/**
* Provides text to be displayed at the top of the dashboard page (migrate_ui).
*/
function hook_migrate_overview() {
return t('<p>Listed below are all the migration processes defined for migration
of our old site to Drupal. Open issues applying to specific migrations
can be viewed by clicking the migration name. Also, details on how each
migration will behave when incrementally migrated are provided.</p>
<p><a href="http://issuetracker.example.com/?project=migration&status=open">Open migration tickets</a></p>');
}

1393
sites/all/modules/migrate/migrate.drush.inc Normal file
View File

File diff suppressed because it is too large Load Diff

54
sites/all/modules/migrate/migrate.info Normal file
View File

@@ -0,0 +1,54 @@
name = "Migrate"
description = "Import content from external sources"
package = "Development"
core = 7.x
files[] = includes/base.inc
files[] = includes/field_mapping.inc
files[] = includes/migration.inc
files[] = includes/destination.inc
files[] = includes/exception.inc
files[] = includes/group.inc
files[] = includes/handler.inc
files[] = includes/map.inc
files[] = includes/source.inc
files[] = includes/team.inc
files[] = migrate.mail.inc
files[] = plugins/destinations/entity.inc
files[] = plugins/destinations/term.inc
files[] = plugins/destinations/user.inc
files[] = plugins/destinations/node.inc
files[] = plugins/destinations/comment.inc
files[] = plugins/destinations/file.inc
files[] = plugins/destinations/path.inc
files[] = plugins/destinations/fields.inc
files[] = plugins/destinations/poll.inc
files[] = plugins/destinations/table.inc
files[] = plugins/destinations/table_copy.inc
files[] = plugins/destinations/menu.inc
files[] = plugins/destinations/menu_links.inc
files[] = plugins/destinations/statistics.inc
files[] = plugins/sources/csv.inc
files[] = plugins/sources/files.inc
files[] = plugins/sources/json.inc
files[] = plugins/sources/list.inc
files[] = plugins/sources/multiitems.inc
files[] = plugins/sources/sql.inc
files[] = plugins/sources/sqlmap.inc
files[] = plugins/sources/mssql.inc
files[] = plugins/sources/oracle.inc
files[] = plugins/sources/xml.inc
files[] = tests/import/options.test
files[] = tests/plugins/destinations/comment.test
files[] = tests/plugins/destinations/node.test
files[] = tests/plugins/destinations/table.test
files[] = tests/plugins/destinations/term.test
files[] = tests/plugins/destinations/user.test
files[] = tests/plugins/sources/xml.test
; Information added by drupal.org packaging script on 2012-06-02
version = "7.x-2.4"
core = "7.x"
project = "migrate"
datestamp = "1338661580"

363
sites/all/modules/migrate/migrate.install Normal file
View File

@@ -0,0 +1,363 @@
<?php
/**
* @file
* Migrate module installation
*/
function migrate_schema() {
$schema = array();
$schema['migrate_status'] = migrate_schema_status();
$schema['migrate_log'] = migrate_schema_log();
return $schema;
}
function migrate_schema_status() {
return array(
'description' => 'Status information for migrations',
'fields' => array(
'machine_name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Unique machine name for migration',
),
'class_name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Name of class to instantiate for this migration',
),
'status' => array(
'type' => 'int',
'size' => 'tiny',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => 0,
'description' => 'Current status of migration',
),
'highwater' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => '',
'description' => 'Highwater mark for detecting updated content',
),
'arguments' => array(
'type' => 'blob',
'not null' => FALSE,
'size' => 'big',
'serialize' => TRUE,
'description' => 'A serialized array of arguments to the migration constructor',
),
),
'primary key' => array('machine_name'),
);
}
function migrate_schema_log() {
return array(
'description' => 'History of migration processes',
'fields' => array(
'mlid' => array(
'type' => 'serial',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Primary key for migrate_log table',
),
'machine_name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Unique machine name for migration',
),
'process_type' => array(
'type' => 'int',
'size' => 'tiny',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Type of migration process - 1 for import, 2 for rollback',
),
'starttime' => array(
'type' => 'int',
'size' => 'big',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Begin time of a migration process, times 1000',
),
'endtime' => array(
'type' => 'int',
'size' => 'big',
'unsigned' => TRUE,
'not null' => FALSE,
'description' => 'End time of a migration process, times 1000',
),
'initialhighwater' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Initial highwater mark',
),
'finalhighwater' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Final highwater mark',
),
'numprocessed' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => FALSE,
'description' => 'Number of items processed',
),
),
'primary key' => array('mlid'),
);
}
/**
* Implements hook_uninstall().
* Drop map/message tables, in case implementing classes did not.
*/
function migrate_uninstall() {
// Note: If a derived Migration class defined its own map or message
// table name not fitting this pattern, that class is solely responsible for
// cleaning up
// TODO: Prefix table names (db_find_tables does not do it)
foreach (db_find_tables('migrate_map_%') as $tablename) {
db_drop_table($tablename);
}
foreach (db_find_tables('migrate_message_%') as $tablename) {
db_drop_table($tablename);
}
// Remove any file_usage entries we've written
if (db_table_exists('file_usage')) {
db_delete('file_usage')
->condition('module', 'migrate')
->execute();
}
}
/**
* Add highwater mark
*/
function migrate_update_7001() {
$ret = array();
if (!db_field_exists('migrate_status', 'highwater')) {
db_add_field('migrate_status', 'highwater', array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => '',
'description' => 'Highwater mark for detecting updated content',
)
);
}
$ret[] = t('Added highwater column to migrate_status table');
return $ret;
}
/**
* Add last_imported field to all map tables
*/
function migrate_update_7002() {
$ret = array();
foreach (db_find_tables('migrate_map_%') as $tablename) {
if (!db_field_exists($tablename, 'last_imported')) {
db_add_field($tablename, 'last_imported', array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => 0,
'description' => 'UNIX timestamp of the last time this row was imported',
));
}
}
$ret[] = t('Added last_imported column to all map tables');
return $ret;
}
/**
* Add lastthroughput column to migrate_status
*/
function migrate_update_7003() {
$ret = array();
if (!db_field_exists('migrate_status', 'lastthroughput')) {
db_add_field('migrate_status', 'lastthroughput', array(
'type' => 'int',
'length' => 11,
'not null' => FALSE,
'description' => 'Rate of success during most recent completed import (# per minute)',
)
);
}
$ret[] = t('Added lastthroughput column to migrate_status table');
return $ret;
}
/**
* Convert lastimported datetime field to lastimportedtime int field.
*/
function migrate_update_7004() {
$ret = array();
if (!db_field_exists('migrate_status', 'lastimportedtime')) {
db_add_field('migrate_status', 'lastimportedtime', array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => FALSE,
'description' => 'Date and time of last completed import',
)
);
if (!db_field_exists('migrate_status', 'lastimported')) {
$result = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name', 'lastimported'))
->execute();
foreach ($result as $row) {
$lastimportedtime = strtotime($row->lastimported);
db_update('migrate_status')
->fields(array('lastimportedtime' => $lastimportedtime))
->condition('machine_name', $row->machine_name)
->execute();
}
db_drop_field('migrate_status', 'lastimported');
$ret[] = t('Converted lastimported datetime field to lastimportedtime int field');
}
}
return $ret;
}
/**
* Add support for history logging
*/
function migrate_update_7005() {
$ret = array();
if (!db_table_exists('migrate_log')) {
$ret[] = t('Create migrate_log table');
db_create_table('migrate_log', migrate_schema_log());
$ret[] = t('Remove historic columns from migrate_status table');
db_drop_field('migrate_status', 'lastthroughput');
db_drop_field('migrate_status', 'lastimportedtime');
}
return $ret;
}
/**
* Add and populate class_name field. Any existing migration code using
* dependencies or sourceMigration() must be changed! See CHANGELOG.txt.
*/
function migrate_update_7006() {
$ret = array();
if (!db_field_exists('migrate_status', 'class_name')) {
db_add_field('migrate_status', 'class_name', array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'default' => '',
'description' => 'Name of class to instantiate for this migration',
)
);
db_query("UPDATE {migrate_status}
SET class_name = CONCAT(machine_name, 'Migration')
");
$ret[] = t('Added class_name column to migrate_status table');
}
return $ret;
}
/**
* Add arguments field to migrate_status table.
*/
function migrate_update_7007() {
$ret = array();
if (!db_field_exists('migrate_status', 'arguments')) {
db_add_field('migrate_status', 'arguments', array(
'type' => 'blob',
'not null' => FALSE,
'size' => 'big',
'serialize' => TRUE,
'description' => 'A serialized array of arguments to the migration constructor',
)
);
$ret[] = t('Added arguments column to migrate_status table');
}
return $ret;
}
/**
* Update map tables to reflect change of needs_update to a status column.
*/
function migrate_update_7008() {
// Updates can be run when the module is disabled, which would mean the
// call to migrate_migrations() will fail. Just bail in that case...
if (!module_exists('migrate')) {
throw new DrupalUpdateException(t('This update cannot be run while the Migrate ' .
'module is disabled - you must enable Migrate to run this update.'));
}
$ret = array();
foreach (migrate_migrations() as $migration) {
if (is_a($migration, 'Migration')) {
// Since we're now tracking failed/ignored rows in the map table,
// destination keys need to be nullable
$map = $migration->getMap();
$map_connection = $map->getConnection();
$map_table = $map->getMapTable();
$destination = $migration->getDestination();
$key_schema = $destination->getKeySchema();
$index = 1;
foreach ($key_schema as $field_schema) {
$field = 'destid' . $index++;
$field_schema['not null'] = FALSE;
$map_connection->schema()->changeField($map_table, $field, $field,
$field_schema);
$ret[] = t('Changed !table.!field to be non-null',
array('!table' => $map_table, '!field' => $field));
}
// Add any existing failures to the map table
$msg_table = $map->getMessageTable();
$msg_marked = FALSE;
$result = $map_connection->select($msg_table, 'msg')
->fields('msg')
->condition('level', Migration::MESSAGE_INFORMATIONAL, '<>')
->execute();
foreach ($result as $row) {
$keys = array();
$index = 1;
foreach ($row as $field => $value) {
if (drupal_substr($field, 0, 8) == 'sourceid') {
$keys['sourceid' . $index++] = $value;
}
}
$map_connection->merge($map_table)
->key($keys)
->fields(array('needs_update' => MigrateMap::STATUS_FAILED))
->execute();
$msg_marked = TRUE;
}
if ($msg_marked) {
$ret[] = t('Marked failures in !table', array('!table' => $map_table));
}
}
}
return $ret;
}
/**
* Warn that there have been incompatible changes to file handling.
*/
function migrate_update_7201() {
return t('File field and destination handling has been completely refactored
- if you are migrating files, you will need to change your migration
implementation to reflect these changes. Please see
<a href="@doc">Handling files in Drupal 7</a> for more information',
array('@doc' => 'http://drupal.org/node/1540106'));
}

23
sites/all/modules/migrate/migrate.mail.inc Normal file
View File

@@ -0,0 +1,23 @@
<?php
/**
* @file
* Null-op mail class, to keep migration from spamming innocent users.
*/
class MigrateMailIgnore extends DefaultMailSystem {
/**
* On an email request, do nothing and say we did.
*
* @see http://php.net/manual/en/function.mail.php
* @see drupal_mail()
*
* @param $message
* A message array, as described in hook_mail_alter().
* @return
* TRUE if the mail was successfully accepted, otherwise FALSE.
*/
public function mail(array $message) {
return TRUE;
}
}

558
sites/all/modules/migrate/migrate.module Normal file
View File

@@ -0,0 +1,558 @@
<?php
/**
* @file
* API and drush commands to support migration of data from external sources
* into a Drupal installation.
*/
// TODO:
// Continue hook_schema_alter() for map & message tables?
// Views hooks for map/message tables
// xlat support?
// Documentation
// Tests
define('MIGRATE_API_VERSION', 2);
/**
* Retrieve a list of all active migrations, ordered by dependencies. To be
* recognized, a class must be non-abstract, and derived from MigrationBase.
*
* @return
* Array of migration objects, keyed by the machine name.
*/
function migrate_migrations() {
static $migrations = array();
if (!empty($migrations)) {
return $migrations;
}
$dependent_migrations = array();
$required_migrations = array();
$result = db_select('migrate_status', 'ms')
->fields('ms', array('machine_name', 'class_name', 'arguments'))
->execute();
foreach ($result as $row) {
if (class_exists($row->class_name)) {
$reflect = new ReflectionClass($row->class_name);
if (!$reflect->isAbstract() && $reflect->isSubclassOf('MigrationBase')) {
$arguments = unserialize($row->arguments);
if (!$arguments || !is_array($arguments)) {
$arguments = array();
}
$migration = MigrationBase::getInstance($row->machine_name,
$row->class_name, $arguments);
$dependencies = $migration->getDependencies();
if (count($dependencies) > 0) {
// Set classes with dependencies aside for reordering
$dependent_migrations[$row->machine_name] = $migration;
$required_migrations += $dependencies;
}
else {
// No dependencies, just add
$migrations[$row->machine_name] = $migration;
}
}
else {
MigrationBase::displayMessage(t('Class !class is no longer a valid concrete migration class',
array('!class' => $row->class_name)));
}
}
else {
MigrationBase::displayMessage(t('Class !class no longer exists',
array('!class' => $row->class_name)));
}
}
// Scan modules with dependencies - we'll take 20 passes at it before
// giving up
// TODO: Can we share code with _migrate_class_list()?
$iterations = 0;
while (count($dependent_migrations) > 0) {
if ($iterations++ > 20) {
$migration_names = implode(',', array_keys($dependent_migrations));
throw new MigrateException(t('Failure to sort migration list - most likely due ' .
'to circular dependencies involving !migration_names',
array('!migration_names' => $migration_names)));
}
foreach ($dependent_migrations as $name => $migration) {
$ready = TRUE;
// Scan all the dependencies for this class and make sure they're all
// in the final list
foreach ($migration->getDependencies() as $dependency) {
if (!isset($migrations[$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this class to the final list
$migrations[$name] = $migration;
unset($dependent_migrations[$name]);
}
}
}
// The migrations are now ordered according to their own dependencies - now order
// them by group
$groups = MigrateGroup::groups();
// Seed the final list by properly-ordered groups.
$final_migrations = array();
foreach ($groups as $name => $group) {
$final_migrations[$name] = array();
}
// Fill in the grouped list
foreach ($migrations as $machine_name => $migration) {
$final_migrations[$migration->getGroup()->getName()][$machine_name] = $migration;
}
// Then flatten the list
$migrations = array();
foreach ($final_migrations as $group_name => $group_migrations) {
foreach ($group_migrations as $machine_name => $migration) {
$migrations[$machine_name] = $migration;
}
}
return $migrations;
}
/**
* On cache clear, scan the Drupal code registry for any new migration classes
* for us to register in migrate_status.
*/
function migrate_flush_caches() {
// Get list of modules implementing Migrate API
$modules = array_keys(migrate_get_module_apis(TRUE));
// Get list of classes we already know about
$existing_classes = db_select('migrate_status', 'ms')
->fields('ms', array('class_name'))
->execute()
->fetchCol();
// Discover class names registered with Drupal by modules implementing our API
$result = db_select('registry', 'r')
->fields('r', array('name'))
->condition('type', 'class')
->condition('module', $modules, 'IN')
->condition('filename', '%.test', 'NOT LIKE')
->execute();
foreach ($result as $record) {
$class_name = $record->name;
// If we already know about this class, skip it
if (isset($existing_classes[$class_name])) {
continue;
}
// Validate it's an implemented subclass of the parent class
// Ignore errors
try {
$class = new ReflectionClass($class_name);
}
catch (Exception $e) {
continue;
}
if (!$class->isAbstract() && $class->isSubclassOf('MigrationBase')) {
// Verify that it's not a dynamic class (the implementor will be responsible
// for registering those).
$dynamic = call_user_func(array($class_name, 'isDynamic'));
if (!$dynamic) {
// OK, this is a new non-dynamic migration class, register it
MigrationBase::registerMigration($class_name);
}
}
}
}
/**
* Invoke any available handlers attached to a given destination type.
* If any handlers have dependencies defined, they will be invoked after
* the specified handlers.
*
* @param $destination
* Destination type ('Node', 'User', etc.) - generally the same string as
* the destination class name without the MigrateDestination prefix.
* @param $method
* Method name such as 'prepare' (called at the beginning of an import operation)
* or 'complete' (called at the end of an import operation).
* @param ...
* Parameters to be passed to the handler.
*/
function migrate_handler_invoke_all($destination, $method) {
$args = func_get_args();
array_shift($args);
array_shift($args);
$return = array();
$class_list = _migrate_class_list('MigrateDestinationHandler');
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
foreach ($class_list as $class_name => $handler) {
if (!in_array($class_name, $disabled) && $handler->handlesType($destination)
&& method_exists($handler, $method)) {
migrate_instrument_start($class_name . '->' . $method);
$result = call_user_func_array(array($handler, $method), $args);
migrate_instrument_stop($class_name . '->' . $method);
if (isset($result) && is_array($result)) {
$return = array_merge($return, $result);
}
elseif (isset($result)) {
$return[] = $result;
}
}
}
return $return;
}
/**
* Invoke any available handlers attached to a given field type.
* If any handlers have dependencies defined, they will be invoked after
* the specified handlers.
*
* @param $entity
* The object we are building up before calling example_save().
* @param $field_info
* Array of info on the field, from field_info_field().
* @param $instance
* Array of info in the field instance, from field_info_instances().
* @param $values
* Array of incoming values, to be transformed into the appropriate structure
* for the field type.
* @param $method
* Handler method to call (defaults to prepare()).
*/
function migrate_field_handler_invoke_all($entity, array $field_info, array $instance,
array $values, $method = 'prepare') {
$return = array();
$type = $field_info['type'];
$class_list = _migrate_class_list('MigrateFieldHandler');
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
foreach ($class_list as $class_name => $handler) {
if (!in_array($class_name, $disabled) && $handler->handlesType($type)
&& method_exists($handler, $method)) {
migrate_instrument_start($class_name . '->' . $method);
$result = call_user_func_array(array($handler, $method),
array($entity, $field_info, $instance, $values));
migrate_instrument_stop($class_name . '->' . $method);
if (isset($result) && is_array($result)) {
$return = array_merge_recursive($return, $result);
}
elseif (isset($result)) {
$return[] = $result;
}
}
}
return $return;
}
/**
* For a given parent class, identify and instantiate objects for any non-abstract
* classes derived from the parent, returning an array of the objects indexed by
* class name. The array will be ordered such that any classes with dependencies
* are listed after the classes they are dependent on.
*
* @param $parent_class
* Name of a class from which results will be derived.
* @param $existing
* Instances already known, which don't need to be instantiated.
* @return
* Array of objects, keyed by the class name.
*/
function _migrate_class_list($parent_class, array $existing = array()) {
// Get list of modules implementing Migrate API
static $modules;
if (!isset($modules)) {
$modules = array_keys(migrate_get_module_apis());
}
static $class_lists = array();
if (!isset($class_lists[$parent_class])) {
$class_lists[$parent_class] = array();
$dependent_classes = array();
$required_classes = array();
// Discover class names registered with Drupal by modules implementing our API
$result = db_select('registry', 'r')
->fields('r', array('name'))
->condition('type', 'class')
->condition('module', $modules, 'IN')
->condition('filename', '%.test', 'NOT LIKE')
->execute();
foreach ($result as $record) {
// Validate it's an implemented subclass of the parent class
// We can get funky errors here, ignore them (and the class that caused them)
try {
$class = new ReflectionClass($record->name);
}
catch (Exception $e) {
continue;
}
if (!$class->isAbstract() && $class->isSubclassOf($parent_class)) {
// If the constructor has required parameters, this may fail. We will
// silently ignore - it is up to the implementor of such a class to
// instantiate it in hook_migrations_alter().
try {
$object = new $record->name;
}
catch (Exception $e) {
unset($object);
}
if (isset($object)) {
$dependencies = $object->getDependencies();
if (count($dependencies) > 0) {
// Set classes with dependencies aside for reordering
$dependent_classes[$record->name] = $object;
$required_classes += $dependencies;
}
else {
// No dependencies, just add
$class_lists[$parent_class][$record->name] = $object;
}
}
}
}
// Validate that each depended-on class at least exists
foreach ($required_classes as $class_name) {
if ((!isset($dependent_classes[$class_name])) && !isset($class_lists[$parent_class][$class_name])) {
throw new MigrateException(t('Dependency on non-existent class !class - make sure ' .
'you have added the file defining !class to the .info file.',
array('!class' => $class_name)));
}
}
// Scan modules with dependencies - we'll take 20 passes at it before
// giving up
$iterations = 0;
while (count($dependent_classes) > 0) {
if ($iterations++ > 20) {
$class_names = implode(',', array_keys($dependent_classes));
throw new MigrateException(t('Failure to sort class list - most likely due ' .
'to circular dependencies involving !class_names.',
array('!class_names' => $class_names)));
}
foreach ($dependent_classes as $name => $object) {
$ready = TRUE;
// Scan all the dependencies for this class and make sure they're all
// in the final list
foreach ($object->getDependencies() as $dependency) {
if (!isset($class_lists[$parent_class][$dependency])) {
$ready = FALSE;
break;
}
}
if ($ready) {
// Yes they are! Move this class to the final list
$class_lists[$parent_class][$name] = $object;
unset($dependent_classes[$name]);
}
}
}
}
return $class_lists[$parent_class];
}
/**
* Implements hook_hook_info().
*/
function migrate_hook_info() {
$hooks['migrate_api'] = array(
'group' => 'migrate',
);
return $hooks;
}
/*
* Implements hook_migrate_api().
*/
function migrate_migrate_api() {
$api = array(
'api' => MIGRATE_API_VERSION,
);
return $api;
}
/**
* Get a list of modules that support the current migrate API.
*/
function migrate_get_module_apis($reset = FALSE) {
static $cache = NULL;
if ($reset) {
$cache = NULL;
}
if (!isset($cache)) {
$cache = array();
foreach (module_implements('migrate_api') as $module) {
$function = $module . '_migrate_api';
$info = $function();
if (isset($info['api']) && $info['api'] == MIGRATE_API_VERSION) {
$cache[$module] = $info;
}
else {
drupal_set_message(t('%function supports Migrate API version %modversion,
Migrate module API version is %version - migration support not loaded.',
array('%function' => $function, '%modversion' => $info['api'],
'%version' => MIGRATE_API_VERSION)));
}
}
}
return $cache;
}
/**
* Implements hook_watchdog().
* Find the migration that is currently running and notify it.
*
* @param array $log_entry
*/
function migrate_watchdog($log_entry) {
// Ensure that the Migration class exists, as different bootstrap phases may
// not have included migration.inc yet.
if (class_exists('Migration') && $migration = Migration::currentMigration()) {
switch ($log_entry['severity']) {
case WATCHDOG_EMERGENCY:
case WATCHDOG_ALERT:
case WATCHDOG_CRITICAL:
case WATCHDOG_ERROR:
$severity = MigrationBase::MESSAGE_ERROR;
break;
case WATCHDOG_WARNING:
$severity = MigrationBase::MESSAGE_WARNING;
break;
case WATCHDOG_NOTICE:
$severity = MigrationBase::MESSAGE_NOTICE;
break;
case WATCHDOG_DEBUG:
case WATCHDOG_INFO:
default:
$severity = MigrationBase::MESSAGE_INFORMATIONAL;
break;
}
$variables = is_array($log_entry['variables']) ? $log_entry['variables'] : array();
$migration->saveMessage(t($log_entry['message'], $variables), $severity);
}
}
/**
* Resource functions modeled on Drupal's timer functions
*/
/**
* Save memory usage with the specified name. If you start and stop the same
* memory name multiple times, the measured differences will be accumulated.
*
* @param name
* The name of the memory measurement.
*/
function migrate_memory_start($name) {
global $_migrate_memory;
$_migrate_memory[$name]['start'] = memory_get_usage();
$_migrate_memory[$name]['count'] =
isset($_migrate_memory[$name]['count']) ? ++$_migrate_memory[$name]['count'] : 1;
}
/**
* Read the current memory value without recording the change.
*
* @param name
* The name of the memory measurement.
* @return
* The change in bytes since the last start.
*/
function migrate_memory_read($name) {
global $_migrate_memory;
if (isset($_migrate_memory[$name]['start'])) {
$stop = memory_get_usage();
$diff = $stop - $_migrate_memory[$name]['start'];
if (isset($_migrate_memory[$name]['bytes'])) {
$diff += $_migrate_memory[$name]['bytes'];
}
return $diff;
}
return $_migrate_memory[$name]['bytes'];
}
/**
* Stop the memory counter with the specified name.
*
* @param name
* The name of the memory measurement.
* @return
* A memory array. The array contains the number of times the memory has been
* started and stopped (count) and the accumulated memory difference value in bytes.
*/
function migrate_memory_stop($name) {
global $_migrate_memory;
if (isset($_migrate_memory[$name])) {
if (isset($_migrate_memory[$name]['start'])) {
$stop = memory_get_usage();
$diff = $stop - $_migrate_memory[$name]['start'];
if (isset($_migrate_memory[$name]['bytes'])) {
$_migrate_memory[$name]['bytes'] += $diff;
}
else {
$_migrate_memory[$name]['bytes'] = $diff;
}
unset($_migrate_memory[$name]['start']);
}
return $_migrate_memory[$name];
}
}
/**
* Start measuring time and (optionally) memory consumption over a section of code.
* Note that the memory consumption measurement is generally not useful in
* lower areas of the code, where data is being generated that will be freed
* by the next call to the same area. For example, measuring the memory
* consumption of db_query is not going to be helpful.
*
* @param $name
* The name of the measurement.
* @param $include_memory
* Measure both memory and timers. Defaults to FALSE (timers only).
*/
function migrate_instrument_start($name, $include_memory = FALSE) {
global $_migrate_track_memory, $_migrate_track_timer;
if ($_migrate_track_memory && $include_memory) {
migrate_memory_start($name);
}
if ($_migrate_track_timer) {
timer_start($name);
}
}
/**
* Stop measuring both memory and time consumption over a section of code.
*
* @param $name
* The name of the measurement.
*/
function migrate_instrument_stop($name) {
global $_migrate_track_memory, $_migrate_track_timer;
if ($_migrate_track_timer) {
timer_stop($name);
}
if ($_migrate_track_memory) {
migrate_memory_stop($name);
}
}
/**
* Call hook_migrate_overview for overall documentation on implemented migrations.
*/
function migrate_overview() {
$overview = '';
$results = module_invoke_all('migrate_overview');
foreach ($results as $result) {
$overview .= $result . ' ';
}
return $overview;
}

464
sites/all/modules/migrate/migrate_example/beer.inc Normal file
View File

@@ -0,0 +1,464 @@
<?php
/**
* @file
* A basic example of using the Migrate module to import taxonomy, users, nodes,
* and comments.
*
* The basic idea is
* - The users in the source application are listed in the
* migrate_example_beer_account table and are transformed into Drupal users.
* - Drupal "beer" nodes describe beers; The information to create the nodes
* comes from the migrate_example_beer_node table.
* - Taxonomy terms for the beer nodes (ale, pilsner) come from the
* migrate_example_beer_topic table and they are applied to nodes using the
* source information in the migrate_example_beer_topic_node table.
* - Comments to be attached to the beer nodes are described in the source
* migrate_example_beer_comment table.
*
* We will use the Migrate API to import and transform this data and turn it into
* a working Drupal system.
*/
/**
* To define a migration process from a set of source data to a particular
* kind of Drupal object (for example, a specific node type), you define
* a class derived from Migration. You must define a constructor to initialize
* your migration object. By default, your class name will be the "machine name"
* of the migration, by which you refer to it. Note that the machine name is
* case-sensitive.
*
* In any serious migration project, you will find there are some options
* which are common to the individual migrations you're implementing. You can
* define an abstract intermediate class derived from Migration, then derive your
* individual migrations from that, to share settings, utility functions, etc.
*/
abstract class BasicExampleMigration extends Migration {
public function __construct() {
// Always call the parent constructor first for basic setup
parent::__construct();
// With migrate_ui enabled, migration pages will indicate people involved in
// the particular migration, with their role and contact info. We default the
// list in the shared class; it can be overridden for specific migrations.
$this->team = array(
new MigrateTeamMember('Liz Taster', 'ltaster@example.com', t('Product Owner')),
new MigrateTeamMember('Larry Brewer', 'lbrewer@example.com', t('Implementor')),
);
// Individual mappings in a migration can be linked to a ticket or issue
// in an external tracking system. Define the URL pattern here in the shared
// class with ':id:' representing the position of the issue number, then add
// ->issueNumber(1234) to a mapping.
$this->issuePattern = 'http://drupal.org/node/:id:';
}
}
/**
* There are four essential components to set up in your constructor:
* $this->source - An instance of a class derived from MigrateSource, this
* will feed data to the migration.
* $this->destination - An instance of a class derived from MigrateDestination,
* this will receive data that originated from the source and has been mapped
* by the Migration class, and create Drupal objects.
* $this->map - An instance of a class derived from MigrateMap, this will keep
* track of which source items have been imported and what destination objects
* they map to.
* Mappings - Use $this->addFieldMapping to tell the Migration class what source
* fields correspond to what destination fields, and additional information
* associated with the mappings.
*/
class BeerTermMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
// Human-friendly description of your migration process. Be as detailed as you
// like.
$this->description = t('Migrate styles from the source database to taxonomy terms');
// Create a map object for tracking the relationships between source rows
// and their resulting Drupal objects. Usually, you'll use the MigrateSQLMap
// class, which uses database tables for tracking. Pass the machine name
// (BeerTerm) of this migration to use in generating map and message tables.
// And, pass schema definitions for the primary keys of the source and
// destination - we need to be explicit for our source, but the destination
// class knows its schema already.
$this->map = new MigrateSQLMap($this->machineName,
array(
'style' => array('type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Topic ID',
)
),
MigrateDestinationTerm::getKeySchema()
);
// In this example, we're using tables that have been added to the existing
// Drupal database but which are not Drupal tables. You can examine the
// various tables (starting here with migrate_example_beer_topic) using a
// database browser like phpMyAdmin.
// First, we set up a query for this data. Note that by ordering on
// style_parent, we guarantee root terms are migrated first, so the
// parent_name mapping below will find that the parent exists.
$query = db_select('migrate_example_beer_topic', 'met')
->fields('met', array('style', 'details', 'style_parent', 'region', 'hoppiness'))
// This sort assures that parents are saved before children.
->orderBy('style_parent', 'ASC');
// Create a MigrateSource object, which manages retrieving the input data.
$this->source = new MigrateSourceSQL($query);
// Set up our destination - terms in the migrate_example_beer_styles vocabulary
$this->destination = new MigrateDestinationTerm('migrate_example_beer_styles');
// Assign mappings TO destination fields FROM source fields. To discover
// the names used in these calls, use the drush commands
// drush migrate-fields-destination BeerTerm
// drush migrate-fields-source BeerTerm
$this->addFieldMapping('name', 'style');
$this->addFieldMapping('description', 'details');
// Documenting your mappings makes it easier for the whole team to see
// exactly what the status is when developing a migration process.
$this->addFieldMapping('parent_name', 'style_parent')
->description(t('The incoming style_parent field is the name of the term parent'));
// Mappings are assigned issue groups, by which they are grouped on the
// migration info page when the migrate_ui module is enabled. The default
// is 'Done', indicating active mappings which need no attention. A
// suggested practice is to use groups of:
// Do Not Migrate (or DNM) to indicate source fields which are not being used,
// or destination fields not to be populated by migration.
// Client Issues to indicate input from the client is needed to determine
// how a given field is to be migrated.
// Implementor Issues to indicate that the client has provided all the
// necessary information, and now the implementor needs to complete the work.
$this->addFieldMapping(NULL, 'hoppiness')
->description(t('This info will not be maintained in Drupal'))
->issueGroup(t('DNM'));
// Open mapping issues can be assigned priorities (the default is
// MigrateFieldMapping::ISSUE_PRIORITY_OK). If you're using an issue
// tracking system, and have defined issuePattern (see BasicExampleMigration
// above), you can specify a ticket/issue number in the system on the
// mapping and migrate_ui will link directory to it.
$this->addFieldMapping(NULL, 'region')
->description('Will a field be added to the vocabulary for this?')
->issueGroup(t('Client Issues'))
->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_MEDIUM)
->issueNumber(770064);
// It is good practice to account for all source and destination fields
// explicitly - this makes sure that everyone understands exactly what is
// being migrated and what is not. Also, migrate_ui highlights unmapped
// fields, or mappings involving fields not in the source and destination,
// so if (for example) a new field is added to the destination field it's
// immediately visible, and you can find out if anything needs to be
// migrated into it.
$this->addFieldMapping('format')
->issueGroup(t('DNM'));
$this->addFieldMapping('weight')
->issueGroup(t('DNM'));
$this->addFieldMapping('parent')
->issueGroup(t('DNM'));
// We conditionally DNM these fields, so your field mappings will be clean
// whether or not you have path and or pathauto enabled
if (module_exists('path')) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
}
}
}
}
/**
* And that's it for the BeerTerm migration! For a simple migration, all you
* have to do is define the source, the destination, and mappings between the
* two - to import the data you simply do:
* drush migrate-import BeerTerm
*
* However, in real-world migrations not everything can be represented simply
* through static mappings - you will frequently need to do some run-time
* transformations of the data.
*/
class BeerUserMigration extends BasicExampleMigration {
public function __construct() {
// The basic setup is similar to BeerTermMigraiton
parent::__construct();
$this->description = t('Beer Drinkers of the world');
$this->map = new MigrateSQLMap($this->machineName,
array('aid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Account ID.'
)
),
MigrateDestinationUser::getKeySchema()
);
$query = db_select('migrate_example_beer_account', 'mea')
->fields('mea', array('aid', 'status', 'posted', 'name', 'nickname', 'password', 'mail', 'sex', 'beers'));
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationUser();
// One good way to organize your mappings is in three groups - mapped fields,
// unmapped source fields, and unmapped destination fields
// Mapped fields
// This is a shortcut you can use when the source and destination field
// names are identical (i.e., the email address field is named 'mail' in
// both the source table and in Drupal).
$this->addSimpleMappings(array('status', 'mail'));
// Our source table has two entries for 'alice', but we must have a unique
// username in the Drupal 'users' table. dedupe() creates new, unique
// destination values when the source field of that value already exists.
// For example, if we're importing a user with name 'test' and a user
// 'test' already exists in the target, we'll create a new user named
// 'test_1'.
// dedupe() takes the Drupal table and column for determining uniqueness.
$this->addFieldMapping('name', 'name')
->dedupe('users', 'name');
// The migrate module automatically converts date/time strings to UNIX timestamps.
$this->addFieldMapping('created', 'posted');
$this->addFieldMapping('pass', 'password');
// Instead of mapping a source field to a destination field, you can
// hardcode a default value. You can also use both together - if a default
// value is provided in addition to a source field, the default value will
// be applied to any rows where the source field is empty or NULL.
$this->addFieldMapping('roles')
->defaultValue(DRUPAL_AUTHENTICATED_RID);
$this->addFieldMapping('field_migrate_example_gender', 'sex');
// The source field has beer names separated by a pipe character ('|'). By
// adding ->separator('|'), the migration will automatically break them out,
// look up the node with each title, and assign the node reference to this
// user.
if (module_exists('node_reference')) {
$this->addFieldMapping('field_migrate_example_favbeers', 'beers')
->separator('|');
}
// Unmapped source fields
$this->addFieldMapping(NULL, 'nickname')
->issueGroup(t('DNM'));
// Unmapped destination fields
// This is a shortcut you can use to mark several destination fields as DNM
// at once
$this->addUnmigratedDestinations(array('theme', 'signature', 'access', 'login',
'timezone', 'language', 'picture', 'is_new', 'signature_format', 'role_names'));
// Oops, we made a typo - this should have been 'init'! If you have
// migrate_ui enabled, look at the BeerUser info page - you'll see that it
// displays a warning "int used as destination field in mapping but not in
// list of destination fields", and also lists "1 unmapped" under Destination,
// where it highlights "init" as unmapped.
$this->addFieldMapping('int')
->issueGroup(t('DNM'));
if (module_exists('path')) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
}
}
}
}
/**
* The BeerNodeMigration uses the migrate_example_beer_node table as source
* and creates Drupal nodes of type 'Beer' as destination.
*/
class BeerNodeMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
$this->description = t('Beers of the world');
// You may optionally declare dependencies for your migration - other migrations
// which should run first. In this case, terms assigned to our nodes and
// the authors of the nodes should be migrated before the nodes themselves.
$this->dependencies = array('BeerTerm', 'BeerUser');
$this->map = new MigrateSQLMap($this->machineName,
array(
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID.',
'alias' => 'b',
)
),
MigrateDestinationNode::getKeySchema()
);
// We have a more complicated query. The Migration class fundamentally
// depends on taking a single source row and turning it into a single
// Drupal object, so how do we deal with zero or more terms attached to
// each node? One way (demonstrated for MySQL) is to pull them into a single
// comma-separated list.
$query = db_select('migrate_example_beer_node', 'b')
->fields('b', array('bid', 'name', 'body', 'excerpt', 'aid', 'countries',
'image', 'image_alt', 'image_title', 'image_description'));
$query->leftJoin('migrate_example_beer_topic_node', 'tb', 'b.bid = tb.bid');
// Gives a single comma-separated list of related terms
$query->groupBy('tb.bid');
$query->addExpression('GROUP_CONCAT(tb.style)', 'terms');
// By default, MigrateSourceSQL derives a count query from the main query -
// but we can override it if we know a simpler way
$count_query = db_select('migrate_example_beer_node', 'b');
$count_query->addExpression('COUNT(bid)', 'cnt');
// Passing the cache_counts option means the source count (shown in
// drush migrate-status) will be cached - this can be very handy when
// dealing with a slow source database.
$this->source = new MigrateSourceSQL($query, array(), $count_query,
array('cache_counts' => TRUE));
// Set up our destination - nodes of type migrate_example_beer
$this->destination = new MigrateDestinationNode('migrate_example_beer');
// Mapped fields
$this->addFieldMapping('title', 'name')
->description(t('Mapping beer name in source to node title'));
$this->addFieldMapping('sticky')
->description(t('Should we default this to 0 or 1?'))
->issueGroup(t('Client questions'))
->issueNumber(765736)
->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_LOW);
// To maintain node identities between the old and new systems (i.e., have
// the same unique IDs), map the ID column from the old system to nid and
// set is_new to TRUE. This works only if we're importing into a system that
// has no existing nodes with the nids being imported.
$this->addFieldMapping('nid', 'bid')
->description(t('Preserve old beer ID as nid in Drupal'));
$this->addFieldMapping('is_new')
->defaultValue(TRUE);
// References to related objects (such as the author of the content) are
// most likely going to be identifiers from the source data, not Drupal
// identifiers (such as uids). You can use the mapping from the relevant
// migration to translate from the old ID to the Drupal identifier.
// Note that we also provide a default value of 1 - if the lookup fails to
// find a corresponding uid for the aid, the owner will be the administrative
// account.
$this->addFieldMapping('uid', 'aid')
->sourceMigration('BeerUser')
->defaultValue(1);
// This is a multi-value text field
$this->addFieldMapping('field_migrate_example_country', 'countries')
->separator('|');
// These are related terms, which by default will be looked up by name
$this->addFieldMapping('migrate_example_beer_styles', 'terms')
->separator(',');
// Some fields may have subfields such as text formats or summaries
// (equivalent to teasers in previous Drupal versions).
// These can be individually mapped as we see here.
$this->addFieldMapping('body', 'body');
$this->addFieldMapping('body:summary', 'excerpt');
// Copy an image file, write DB record to files table, and save in Field storage.
// We map the filename (relative to the source_dir below) to the field itself.
$this->addFieldMapping('field_migrate_example_image', 'image');
// Here we specify the directory containing the source files.
$this->addFieldMapping('field_migrate_example_image:source_dir')
->defaultValue(drupal_get_path('module', 'migrate_example'));
// And we map the alt and title values in the database to those on the image.
$this->addFieldMapping('field_migrate_example_image:alt', 'image_alt');
$this->addFieldMapping('field_migrate_example_image:title', 'image_title');
// No description for images, only alt and title
$this->addUnmigratedSources(array('image_description'));
// Unmapped destination fields
$this->addUnmigratedDestinations(array('created', 'changed', 'status',
'promote', 'revision', 'language', 'revision_uid', 'log', 'tnid',
'body:format', 'body:language', 'migrate_example_beer_styles:source_type',
'migrate_example_beer_styles:create_term', 'field_migrate_example_image:destination_dir',
'field_migrate_example_image:language', 'field_migrate_example_image:file_replace',
'field_migrate_example_image:preserve_files', 'field_migrate_example_country:format',
'field_migrate_example_country:language', 'comment',
'field_migrate_example_image:file_class', 'field_migrate_example_image:destination_file'));
if (module_exists('path')) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
if (module_exists('pathauto')) {
$this->addFieldMapping('pathauto')
->issueGroup(t('DNM'));
}
}
if (module_exists('statistics')) {
$this->addUnmigratedDestinations(array('totalcount', 'daycount', 'timestamp'));
}
}
}
/**
* Import items from the migrate_example_beer_comment table and make them into
* Drupal comment objects.
*/
class BeerCommentMigration extends BasicExampleMigration {
public function __construct() {
parent::__construct();
$this->description = 'Comments about beers';
$this->dependencies = array('BeerUser', 'BeerNode');
$this->map = new MigrateSQLMap($this->machineName,
array('cid' => array(
'type' => 'int',
'not null' => TRUE,
)
),
MigrateDestinationComment::getKeySchema()
);
$query = db_select('migrate_example_beer_comment', 'mec')
->fields('mec', array('cid', 'cid_parent', 'name', 'mail', 'aid', 'body', 'bid', 'subject'))
->orderBy('cid_parent', 'ASC');
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationComment('comment_node_migrate_example_beer');
// Mapped fields
$this->addSimpleMappings(array('name', 'subject', 'mail'));
$this->addFieldMapping('status')
->defaultValue(COMMENT_PUBLISHED);
// We preserved bid => nid ids during BeerNode import so simple mapping suffices.
$this->addFieldMapping('nid', 'bid');
$this->addFieldMapping('uid', 'aid')
->sourceMigration('BeerUser')
->defaultValue(0);
$this->addFieldMapping('pid', 'cid_parent')
->sourceMigration('BeerComment')
->description('Parent comment.');
$this->addFieldMapping('comment_body', 'body');
// No unmapped source fields
// Unmapped destination fields
$this->addUnmigratedDestinations(array('hostname', 'created', 'changed',
'thread', 'homepage', 'language', 'comment_body:format', 'comment_body:language'));
if (module_exists('path')) {
$this->addFieldMapping('path')
->issueGroup(t('DNM'));
}
}
}

678
sites/all/modules/migrate/migrate_example/beer.install.inc Normal file
View File

@@ -0,0 +1,678 @@
<?php
/**
* @file
* Set up for the beer (basic) example.
*/
function migrate_example_beer_schema() {
$schema['migrate_example_beer_account'] = migrate_example_beer_schema_account();
$schema['migrate_example_beer_node'] = migrate_example_beer_schema_node();
$schema['migrate_example_beer_comment'] = migrate_example_beer_schema_comment();
$schema['migrate_example_beer_topic'] = migrate_example_beer_schema_topic();
$schema['migrate_example_beer_topic_node'] = migrate_example_beer_schema_topic_node();
// These two tables are primarily for testing the table_copy plugin.
// They do provide some guidance for uri redirection per uri_map_redirect.php
$schema['migrate_example_beer_legacy_urls'] = migrate_example_beer_schema_legacy_urls();
$schema['migrate_example_beer_copy_urls'] = migrate_example_beer_schema_legacy_urls();
return $schema;
}
function migrate_example_beer_install() {
migrate_example_beer_content_type();
migrate_example_beer_tags();
migrate_example_beer_image();
migrate_example_beer_country();
migrate_example_beer_gender();
if (module_exists('node_reference')) {
migrate_example_beer_favs();
}
// Populate our tables.
migrate_example_beer_data_account();
migrate_example_beer_data_node();
migrate_example_beer_data_comment();
migrate_example_beer_data_topic();
migrate_example_beer_data_topic_node();
migrate_example_beer_data_urls();
}
function migrate_example_beer_uninstall() {
if ($vids = taxonomy_vocabulary_load_multiple(array(), array('machine_name' => 'migrate_example_beer_styles'))) {
// Grab key of the first returned vocabulary.
taxonomy_vocabulary_delete(key($vids));
}
migrate_example_beer_content_type_delete();
}
function migrate_example_beer_disable() {
Migration::deregisterMigration('BeerTerm');
Migration::deregisterMigration('BeerUser');
Migration::deregisterMigration('BeerNode');
Migration::deregisterMigration('BeerComment');
}
function migrate_example_beer_schema_node() {
return array(
'description' => 'Beers of the world.',
'fields' => array(
'bid' => array(
'type' => 'serial',
'not null' => TRUE,
'description' => 'Beer ID.',
),
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
),
'body' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Full description of the beer.',
),
'excerpt' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Abstract for this beer.',
),
'countries' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Countries of origin. Multiple values, delimited by pipe',
),
'aid' => array(
'type' => 'int',
'not null' => FALSE,
'description' => 'Account Id of the author.',
),
'image' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image path',
),
'image_alt' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image ALT',
),
'image_title' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image title',
),
'image_description' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image description',
),
),
'primary key' => array('bid'),
);
}
function migrate_example_beer_schema_topic() {
return array(
'description' => 'Categories',
'fields' => array(
'style' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
),
'details' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
),
'style_parent' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Parent topic, if any',
),
'region' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Region first associated with this style',
),
'hoppiness' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Relative hoppiness of the beer',
),
),
'primary key' => array('style'),
);
}
function migrate_example_beer_schema_topic_node() {
return array(
'description' => 'Beers topic pairs.',
'fields' => array(
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID.',
),
'style' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Topic name',
),
),
'primary key' => array('style', 'bid'),
);
}
function migrate_example_beer_schema_comment() {
return array(
'description' => 'Beers comments.',
'fields' => array(
'cid' => array(
'type' => 'serial',
'not null' => TRUE,
'description' => 'Comment ID.',
),
'bid' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Beer ID that is being commented upon',
),
'cid_parent' => array(
'type' => 'int',
'not null' => FALSE,
'description' => 'Parent comment ID in case of comment replies.',
),
'subject' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Comment subject',
),
'body' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Comment body',
),
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Comment name (if anon)',
),
'mail' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Comment email (if anon)',
),
'aid' => array(
'type' => 'int',
'not null' => FALSE,
'description' => 'Account ID (if any).',
),
),
'primary key' => array('cid'),
);
}
function migrate_example_beer_schema_account() {
return array(
'description' => 'Beers accounts.',
'fields' => array(
'aid' => array(
'type' => 'serial',
//'not null' => TRUE,
'description' => 'Account ID',
),
'status' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Blocked_Allowed',
),
'posted' => array(
'type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Registration date',
),
'name' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Account name (for login)',
),
'nickname' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Account name (for display)',
),
'password' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Account password (raw)',
),
'mail' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Account email',
),
'sex' => array(
'type' => 'int',
'not null' => FALSE,
'description' => 'Gender',
),
'beers' => array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Favorite Beers',
),
),
'primary key' => array('aid'),
);
}
function migrate_example_beer_schema_legacy_urls() {
return array(
'description' => 'Stores legacy paths and destination ids for redirection.',
'fields' => array(
'id' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Primary Key: ID.',
),
'migration_name' => array(
'type' => 'varchar',
'length' => 50,
'not null' => TRUE,
'default' => '',
),
'source_id' => array(
'type' => 'int',
'not null' => FALSE,
),
'source_uri' => array(
'type' => 'varchar',
'length' => 500,
'not null' => FALSE,
),
'modificationdatetime' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
),
),
'primary key' => array('ID'),
'indexes' => array(
'source_uri' => array(array('source_uri', 255)),
),
);
}
function migrate_example_beer_content_type() {
// This code based on from standard.profile.
// Insert default user-defined node types into the database.
$types = array(
array(
'type' => 'migrate_example_beer',
'name' => st('Beer'),
'base' => 'node_content',
'description' => st("Beer is what we drink."),
'custom' => 1,
'modified' => 1,
'locked' => 1,
),
);
foreach ($types as $type) {
$type = node_type_set_defaults($type);
node_type_save($type);
node_add_body_field($type);
}
}
function migrate_example_beer_tags() {
// Create a vocabulary named "Migrate Example Beer Styles", enabled for the 'migrate_example_beer' content type.
$description = st('Use tags to group beers on similar topics into categories.');
$help = st('Enter a comma-separated list of words to describe your content.');
$vocabulary = (object) array(
'name' => 'Migrate Example Beer Styles',
'description' => $description,
'machine_name' => 'migrate_example_beer_styles',
'help' => $help,
);
taxonomy_vocabulary_save($vocabulary);
if (!field_info_field('migrate_example_beer_styles')) {
$field = array(
'field_name' => $vocabulary->machine_name,
'type' => 'taxonomy_term_reference',
// Set cardinality to unlimited for tagging.
'cardinality' => FIELD_CARDINALITY_UNLIMITED,
'settings' => array(
'allowed_values' => array(
array(
'vocabulary' => $vocabulary->machine_name,
'vid' => $vocabulary->vid,
'parent' => 0,
),
),
),
);
field_create_field($field);
}
if (!field_info_instance('node', 'migrate_example_beer_styles', 'migrate_example_beer')) {
$instance = array(
'field_name' => $vocabulary->machine_name,
'entity_type' => 'node',
'label' => $vocabulary->name,
'bundle' => 'migrate_example_beer',
'description' => $vocabulary->help,
'widget' => array(
'type' => 'taxonomy_autocomplete',
),
);
field_create_instance($instance);
}
}
// Create an image field named "Migrate Example Image", enabled for the 'Beer' content type.
function migrate_example_beer_image() {
if (!field_info_field('field_migrate_example_image')) {
$field = array(
'field_name' => 'field_migrate_example_image',
'type' => 'image',
'cardinality' => 1,
'translatable' => TRUE,
'indexes' => array('fid' => array('fid')),
'settings' => array(
'uri_scheme' => 'public',
'default_image' => FALSE,
),
);
field_create_field($field);
}
if (!field_info_instance('node', 'field_migrate_example_image', 'migrate_example_beer')) {
$instance = array(
'field_name' => 'field_migrate_example_image',
'entity_type' => 'node',
'label' => 'Image',
'bundle' => 'migrate_example_beer',
'description' => 'Upload an image to go with this beer.',
'settings' => array(
'file_directory' => 'field/migrate_example/image',
'file_extensions' => 'png gif jpg jpeg',
'max_filesize' => '',
'max_resolution' => '',
'min_resolution' => '',
'alt_field' => TRUE,
'title_field' => '',
),
'widget' => array(
'type' => 'image_image',
'settings' => array(
'progress_indicator' => 'throbber',
'preview_image_style' => 'thumbnail',
),
'weight' => -1,
),
'display' => array(
'full' => array(
'label' => 'hidden',
'type' => 'image__large',
'settings' => array(),
'weight' => -1,
),
'teaser' => array(
'label' => 'hidden',
'type' => 'image_link_content__medium',
'settings' => array(),
'weight' => -1,
),
'rss' => array(
'label' => 'hidden',
'type' => 'image__large',
'settings' => array(),
'weight' => -1,
),
'search_index' => array(
'label' => 'hidden',
'type' => 'image__large',
'settings' => array(),
'weight' => -1,
),
'search_results' => array(
'label' => 'hidden',
'type' => 'image__large',
'settings' => array(),
'weight' => -1,
),
),
);
field_create_instance($instance);
}
}
function migrate_example_beer_favs() {
if (!field_info_field('field_migrate_example_favbeers')) {
$field = array(
'field_name' => 'field_migrate_example_favbeers',
'type' => 'node_reference',
'cardinality' => -1,
'settings' => array(
'referenceable_types' => array('migrate_example_beer'),
),
);
field_create_field($field);
}
if (!field_info_instance('user', 'field_migrate_example_favbeers', 'user')) {
$instance = array(
'field_name' => 'field_migrate_example_favbeers',
'entity_type' => 'user',
'label' => 'Favorite Beers',
'bundle' => 'user',
'widget' => array(
'type' => 'node_reference_autocomplete',
),
);
field_create_instance($instance);
}
}
// Create Gender list field on User entity.
function migrate_example_beer_gender() {
if (!field_info_field('field_migrate_example_gender')) {
$field = array(
'field_name' => 'field_migrate_example_gender',
'type' => 'list_integer',
'settings' => array(
'allowed_values' =>
"0|Male\n" .
"1|Female\n",
),
);
field_create_field($field);
}
if (!field_info_instance('user', 'field_migrate_example_gender', 'user')) {
$instance = array(
'field_name' => 'field_migrate_example_gender',
'entity_type' => 'user',
'label' => 'Gender',
'bundle' => 'user',
'widget' => array(
'type' => 'options_select',
),
);
field_create_instance($instance);
}
}
// Create a text field named "Countries", enabled for the 'Beer' content type.
function migrate_example_beer_country() {
if (!field_info_field('field_migrate_example_country')) {
$field = array(
'field_name' => 'field_migrate_example_country',
'type' => 'text',
'cardinality' => -1,
);
field_create_field($field);
}
if (!field_info_instance('node', 'field_migrate_example_country', 'migrate_example_beer')) {
$instance = array(
'field_name' => 'field_migrate_example_country',
'entity_type' => 'node',
'label' => 'Countries',
'bundle' => 'migrate_example_beer',
'description' => 'Beer country.',
'widget' => array(
'type' => 'text_textfield',
),
);
field_create_instance($instance);
}
}
function migrate_example_beer_content_type_delete() {
$bundle = 'migrate_example_beer';
$field_names = array('migrate_example_beer_styles', 'field_migrate_example_image', 'field_migrate_example_country');
foreach ($field_names as $field_name) {
$instance = field_info_instance('node', $field_name, $bundle);
field_delete_instance($instance);
field_delete_field($field_name);
}
node_type_delete($bundle);
$bundle = 'user';
$field_names = array('field_migrate_example_gender');
if (module_exists('node_reference')) {
$field_names[] = 'field_migrate_example_favbeers';
}
foreach ($field_names as $field_name) {
$instance = field_info_instance('user', $field_name, $bundle);
field_delete_instance($instance);
field_delete_field($field_name);
}
}
function migrate_example_beer_data_node() {
$fields = array('bid', 'name', 'body', 'excerpt', 'countries', 'aid', 'image',
'image_alt', 'image_title', 'image_description');
$query = db_insert('migrate_example_beer_node')
->fields($fields);
// Use high bid numbers to avoid overwriting an existing node id.
$data = array(
array(99999999, 'Heineken', 'Blab Blah Blah Green', 'Green', 'Netherlands|Belgium', 0, 'heineken.jpg', 'Heinekin alt', 'Heinekin title', 'Heinekin description'), // comes with migrate_example project.
array(99999998, 'Miller Lite', 'We love Miller Brewing', 'Tasteless', 'USA|Canada', 1, NULL, NULL, NULL, NULL),
array(99999997, 'Boddington', 'English occassionally get something right', 'A treat', 'United Kingdom', 1, NULL, NULL, NULL, NULL),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
}
$query->execute();
}
// Note that alice has duplicate username. Exercies dedupe() method.
// @TODO duplicate email also.
function migrate_example_beer_data_account() {
$fields = array('status', 'posted', 'name', 'nickname', 'password', 'mail', 'sex', 'beers');
$query = db_insert('migrate_example_beer_account')
->fields($fields);
$data = array(
array(1, '2010-03-30 10:31:05', 'alice', 'alice hot pants', 'alicepass', 'alice@example.com', '1', '99999999|99999998|99999997'),
array(1, '2010-04-04 10:31:05', 'alice', 'alice dupe pants', 'alicepass', 'alice2@example.com', '1', '99999999|99999998|99999997'),
array(0, '2007-03-15 10:31:05', 'bob', 'rebob', 'bobpass', 'bob@example.com', '1', '99999999|99999997'),
array(1, '2004-02-29 10:31:05', 'charlie', 'charlie chocolate', 'mykids', 'charlie@example.com', '0', '99999999|99999998'),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
}
$query->execute();
}
function migrate_example_beer_data_comment() {
$fields = array('bid', 'cid_parent', 'subject', 'body', 'name', 'mail', 'aid');
$query = db_insert('migrate_example_beer_comment')
->fields($fields);
$data = array(
array(99999998, NULL, 'im first', 'hot body', 'alice', 'alice@example.com', 0),
array(99999998, NULL, 'im second', 'hot body', 'alice', 'alice@example.com', 0),
array(99999999, NULL, 'im parent', 'hot body', 'alice', 'alice@example.com', 0),
array(99999999, 1, 'im child', 'cold body', 'bob', NULL, 1),
array(99999999, 2, 'im grandchild', 'bitter body', 'charlie@example.com', NULL, 1),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
}
$query->execute();
}
function migrate_example_beer_data_topic() {
$fields = array('style', 'details', 'style_parent', 'region', 'hoppiness');
$query = db_insert('migrate_example_beer_topic')
->fields($fields);
$data = array(
array('ale', 'traditional', NULL, 'Medieval British Isles', 'Medium'),
array('red ale', 'colorful', 'ale', NULL, NULL),
array('pilsner', 'refreshing', NULL, 'Pilsen, Bohemia (now Czech Republic)', 'Low'),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
}
$query->execute();
}
function migrate_example_beer_data_topic_node() {
$fields = array('bid', 'style');
$query = db_insert('migrate_example_beer_topic_node')
->fields($fields);
$data = array(
array(99999999, 'pilsner'),
array(99999999, 'red ale'),
array(99999998, 'red ale'),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
}
$query->execute();
}
function migrate_example_beer_data_urls() {
$fields = array('id', 'migration_name', 'source_id', 'source_uri', 'modificationdatetime');
$query = db_insert('migrate_example_beer_legacy_urls')
->fields($fields);
$data = array(
array(1, 'BeerNode', 99999997, 'the_boddington/main', strtotime('2010-04-12 08:32:06')),
array(2, 'BeerNode', 99999998, 'Miller Lite taste', strtotime('2010-04-12 08:32:05')),
array(3, 'BeerNode', 99999999, 'green wonder', strtotime('2010-04-12 08:32:03')),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
}
$query->execute();
}

36
sites/all/modules/migrate/migrate_example/example.table_copy.inc Normal file
View File

@@ -0,0 +1,36 @@
<?php
/**
* @file
* Make a copy of the role table. To use this you must create a table named
* role_copy with the same structure as role.
*/
class RoleTableMigration extends Migration {
public function __construct() {
parent::__construct();
$this->dependencies = array();
$this->description = 'Copy the role table as an example of table_copy plugin.';
$destination_key = array(
'rid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
),
);
$query = db_select('role', 'r')->fields('r');
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationTableCopy('role_copy', $destination_key);
$this->map = new MigrateSQLMap($this->machineName,
array('rid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'alias' => 'r',
)
),
$destination_key
);
}
}

BIN
sites/all/modules/migrate/migrate_example/heineken.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 119 KiB

27
sites/all/modules/migrate/migrate_example/migrate_example.info Normal file
View File

@@ -0,0 +1,27 @@
name = "Migrate Example"
description = "Example migration data."
package = "Development"
core = 7.x
dependencies[] = taxonomy
dependencies[] = image
dependencies[] = comment
dependencies[] = migrate
dependencies[] = list
dependencies[] = number
;node_reference is useful but not required
;dependencies[] = node_reference
files[] = migrate_example.module
files[] = beer.inc
files[] = wine.inc
; For testing table_copy plugin. Since is infrequently used, we comment it out.
; files[] = example.table_copy.inc
; Information added by drupal.org packaging script on 2012-06-02
version = "7.x-2.4"
core = "7.x"
project = "migrate"
datestamp = "1338661580"

229
sites/all/modules/migrate/migrate_example/migrate_example.install Normal file
View File

@@ -0,0 +1,229 @@
<?php
/**
* @file
* Set up the migration example module.
*/
require_once DRUPAL_ROOT . '/' . drupal_get_path('module', 'migrate_example') .
'/beer.install.inc';
require_once DRUPAL_ROOT . '/' . drupal_get_path('module', 'migrate_example') .
'/wine.install.inc';
function migrate_example_schema() {
$schema = migrate_example_beer_schema();
$schema += migrate_example_wine_schema();
return $schema;
}
function migrate_example_install() {
migrate_example_beer_install();
migrate_example_wine_install();
// A simple format for testing migration of format
$example_format = array(
'format' => 'migrate_example',
'name' => 'Migrate example format',
'weight' => 20,
'filters' => array(
// Escape all HTML.
'filter_html_escape' => array(
'weight' => 0,
'status' => 1,
),
),
);
$example_format = (object) $example_format;
filter_format_save($example_format);
}
function migrate_example_uninstall() {
migrate_example_beer_uninstall();
migrate_example_wine_uninstall();
if ($format = filter_format_load('migrate_example')) {
filter_format_disable($format);
}
}
function migrate_example_disable() {
migrate_example_beer_disable();
migrate_example_wine_disable();
}
/**
* Convert modificationdate datetime field to modificationdatetime int field.
*/
function migrate_example_update_7001() {
$ret = array();
db_add_field('migrate_example_beer_legacy_urls', 'modificationdatetime', array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => FALSE,
)
);
$result = db_select('migrate_example_beer_legacy_urls', 'ms')
->fields('ms', array('machine_name', 'modificationdate'))
->execute();
foreach ($result as $row) {
$modificationdatetime = strtotime($row->modificationdate);
db_update('migrate_example_beer_legacy_urls')
->fields(array('modificationdatetime' => $modificationdatetime))
->condition('machine_name', $row->machineName)
->execute();
}
db_drop_field('migrate_example_beer_legacy_urls', 'modificationdate');
$ret[] = t('Converted modificationdate datetime field to modificationdatetime int field');
return $ret;
}
/**
* Add image alt/title/description columns.
*/
function migrate_example_update_7002() {
$ret = array();
db_add_field('migrate_example_beer_node', 'image_alt', array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image ALT',
)
);
db_add_field('migrate_example_beer_node', 'image_title', array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image title',
)
);
db_add_field('migrate_example_beer_node', 'image_description', array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Image description',
)
);
db_update('migrate_example_beer_node')
->fields(array(
'image_alt' => 'Heinekin alt',
'image_title' => 'Heinekin title',
'image_description' => 'Heinekin description',
))
->condition('bid', 99999999)
->execute();
$ret[] = t('Added image_alt, image_title, and image_description fields.');
return $ret;
}
/**
* Add data for remote file examples.
*/
function migrate_example_update_7003() {
$ret = array();
db_create_table('migrate_example_wine_files', migrate_example_wine_schema_files());
migrate_example_wine_data_files();
db_add_field('migrate_example_wine_account', 'imageid', array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => FALSE,
'description' => 'Image ID.',
)
);
db_update('migrate_example_wine_account')
->fields(array('imageid' => 1))
->condition('accountid', 9)
->execute();
$ret[] = t('Added migrate_example_wine_files table.');
$ret[] = t('Added imageid field to migrate_example_wine_account table.');
return $ret;
}
/**
* Add sample data for file fields. And, make the image field multi-value.
*/
function migrate_example_update_7004() {
$ret = array();
db_update('migrate_example_wine')
->fields(array('image' => 'http://cyrve.com/files/penguin.jpeg'))
->condition('wineid', 1)
->execute();
db_update('migrate_example_wine')
->fields(array('image' => 'http://cyrve.com/files/rioja.jpeg|http://cyrve.com/files/boutisse_0.jpeg'))
->condition('wineid', 2)
->execute();
$field = field_info_field('field_migrate_example_image');
if ($field) {
$field['cardinality'] = -1; // Unlimited
field_update_field($field);
}
else {
migrate_example_beer_image();
migrate_example_wine_fields();
}
$ret[] = t('Added sample data for file fields.');
$ret[] = t('Made field_migrate_example_image multi-value');
return $ret;
}
/**
* Expand file field example data.
*/
function migrate_example_update_7005() {
$ret = array();
// Easiest to just start over from scratch
if (db_table_exists('migrate_example_wine_files')) {
db_drop_table('migrate_example_wine_files');
}
db_create_table('migrate_example_wine_files', migrate_example_wine_schema_files());
migrate_example_wine_data_files();
// Moved this data to migrate_example_wine_files
if (db_field_exists('migrate_example_wine', 'image')) {
db_drop_field('migrate_example_wine', 'image');
}
$ret[] = t('Reconfigured sample data for file fields.');
return $ret;
}
/**
* Sample data for table destinations..
*/
function migrate_example_update_7006() {
$ret = array();
db_create_table('migrate_example_wine_table_source', migrate_example_wine_schema_table_source());
db_create_table('migrate_example_wine_table_dest', migrate_example_wine_schema_table_dest());
migrate_example_wine_data_table_source();
$ret[] = t('Added sample data for table destinations.');
return $ret;
}
/**
* Add data for testing/demonstrating roles.
*/
function migrate_example_update_7007() {
$ret = array();
db_add_field('migrate_example_wine_account', 'positions', array(
'type' => 'varchar',
'length' => 255,
'not null' => FALSE,
'description' => 'Positions held',
)
);
$query = db_update('migrate_example_wine_account')
->fields(array('positions' => '5'))
->condition('accountid', 1)
->execute();
db_update('migrate_example_wine_account')
->fields(array('positions' => '18'))
->condition('accountid', 3)
->execute();
db_update('migrate_example_wine_account')
->fields(array('positions' => '5,18'))
->condition('accountid', 9)
->execute();
$ret[] = t('Added positions field to migrate_example_wine_account table.');
return $ret;
}

12
sites/all/modules/migrate/migrate_example/migrate_example.module Normal file
View File

@@ -0,0 +1,12 @@
<?php
/*
* You must implement hook_migrate_api(), setting the API level to 2, for
* your migration classes to be recognized by the Migrate module.
*/
function migrate_example_migrate_api() {
$api = array(
'api' => 2,
);
return $api;
}

164
sites/all/modules/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.features.field.inc Normal file
View File

@@ -0,0 +1,164 @@
<?php
/**
* Implementation of hook_field_default_fields().
*/
function migrate_example_oracle_field_default_fields() {
$fields = array();
// Exported field: 'node-migrate_example_oracle-body'
$fields['node-migrate_example_oracle-body'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(
'0' => 'node',
),
'field_name' => 'body',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(),
'translatable' => '1',
'type' => 'text_with_summary',
),
'field_instance' => array(
'bundle' => 'migrate_example_oracle',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'hidden',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => 0,
),
'teaser' => array(
'label' => 'hidden',
'module' => 'text',
'settings' => array(
'trim_length' => 600,
),
'type' => 'text_summary_or_trimmed',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'body',
'label' => 'Body',
'required' => FALSE,
'settings' => array(
'display_summary' => TRUE,
'text_processing' => 1,
'user_register_form' => FALSE,
),
'widget' => array(
'module' => 'text',
'settings' => array(
'rows' => 20,
'summary_rows' => 5,
),
'type' => 'text_textarea_with_summary',
'weight' => '-4',
),
),
);
// Exported field: 'node-migrate_example_oracle-field_mainimage'
$fields['node-migrate_example_oracle-field_mainimage'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_mainimage',
'foreign keys' => array(
'fid' => array(
'columns' => array(
'fid' => 'fid',
),
'table' => 'file_managed',
),
),
'indexes' => array(
'fid' => array(
'0' => 'fid',
),
),
'module' => 'image',
'settings' => array(
'default_image' => 0,
'uri_scheme' => 'public',
),
'translatable' => '1',
'type' => 'image',
),
'field_instance' => array(
'bundle' => 'migrate_example_oracle',
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'above',
'module' => 'image',
'settings' => array(
'image_link' => '',
'image_style' => '',
),
'type' => 'image',
'weight' => 1,
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_mainimage',
'label' => 'Main image',
'required' => FALSE,
'settings' => array(
'alt_field' => 0,
'file_directory' => '',
'file_extensions' => 'png gif jpg jpeg',
'max_filesize' => '',
'max_resolution' => '',
'min_resolution' => '',
'title_field' => 0,
'user_register_form' => FALSE,
),
'widget' => array(
'module' => 'image',
'settings' => array(
'preview_image_style' => 'thumbnail',
'progress_indicator' => 'throbber',
),
'type' => 'image_image',
'weight' => '-3',
),
),
);
// Translatables
// Included for use with string extractors like potx.
t('Body');
t('Main image');
return $fields;
}

18
sites/all/modules/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.features.inc Normal file
View File

@@ -0,0 +1,18 @@
<?php
/**
* Implementation of hook_node_info().
*/
function migrate_example_oracle_node_info() {
$items = array(
'migrate_example_oracle' => array(
'name' => t('Migrate example - oracle'),
'base' => 'node_content',
'description' => t('Example and test fodder for migration directly from an Oracle database.'),
'has_title' => '1',
'title_label' => t('Title'),
'help' => '',
),
);
return $items;
}

19
sites/all/modules/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.info Normal file
View File

@@ -0,0 +1,19 @@
core = "7.x"
dependencies[] = "features"
dependencies[] = "image"
dependencies[] = "migrate"
description = "Content type supporting example of Oracle migration"
features[field][] = "node-migrate_example_oracle-body"
features[field][] = "node-migrate_example_oracle-field_mainimage"
features[node][] = "migrate_example_oracle"
files[] = "migrate_example_oracle.migrate.inc"
name = "Migrate example - Oracle"
package = "Migrate Examples"
project = "migrate_example_oracle"
; Information added by drupal.org packaging script on 2012-06-02
version = "7.x-2.4"
core = "7.x"
project = "migrate"
datestamp = "1338661580"

198
sites/all/modules/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.install Normal file
View File

@@ -0,0 +1,198 @@
<?php
/**
* @file
* Set up the Oracle migration example module.
*/
/**
* Implements hook_install().
*/
function migrate_example_oracle_install() {
global $conf;
// Should never fail - we can't get here unless hook_requirements passed, right?
$connection = @oci_connect($conf['oracle_db']['username'], $conf['oracle_db']['password'],
$conf['oracle_db']['connection_string'], 'UTF8');
if (!$connection) {
$e = oci_error();
throw new Exception($e['message']);
}
// Create a table to hold test data
$query = "CREATE TABLE ORACLE_CONTENT
(OID NUMBER NOT NULL,
TITLE VARCHAR2(255) NOT NULL,
BODY CLOB NOT NULL,
MAINIMAGE BLOB NOT NULL,
CREATED DATE NOT NULL,
UPDATED DATE NOT NULL,
CONSTRAINT ORACLE_CONTENT_PK PRIMARY KEY (OID)
)";
$result = oci_parse($connection, $query);
if (!$result) {
$e = oci_error($connection);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
$status = oci_execute($result);
if (!$status) {
$e = oci_error($result);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
// Insert a test row or three
$query = "INSERT INTO ORACLE_CONTENT
(OID, TITLE, BODY, MAINIMAGE, CREATED, UPDATED)
VALUES(:oid, :title, EMPTY_CLOB(), EMPTY_BLOB(),
TO_DATE(:created, 'yyyy/mm/dd hh24:mi:ss'),
TO_DATE(:updated, 'yyyy/mm/dd hh24:mi:ss'))
RETURNING body, mainimage INTO :body, :mainimage";
$result = oci_parse($connection, $query);
if (!$result) {
$e = oci_error($connection);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
$data = migrate_example_oracle_sample_data();
oci_bind_by_name($result, ':oid', $oid, 1, SQLT_INT);
oci_bind_by_name($result, ':title', $title, 255, SQLT_CHR);
$body_clob = oci_new_descriptor($connection, OCI_D_LOB);
$image_blob = oci_new_descriptor($connection, OCI_D_LOB);
oci_bind_by_name($result, ':body', $body_clob, -1, SQLT_CLOB);
oci_bind_by_name($result, ':mainimage', $image_blob, -1, SQLT_BLOB);
oci_bind_by_name($result, ':created', $created, 9, SQLT_CHR);
oci_bind_by_name($result, ':updated', $updated, 9, SQLT_CHR);
foreach ($data as $row) {
extract($row);
$status = oci_execute($result, OCI_DEFAULT);
if (!$status) {
$e = oci_error($result);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
$body_clob->save($body);
$image_blob->save($mainimage);
}
oci_commit($connection);
}
/**
* Implements hook_uninstall().
*/
function migrate_example_oracle_uninstall() {
global $conf;
$connection = @oci_connect($conf['oracle_db']['username'], $conf['oracle_db']['password'],
$conf['oracle_db']['connection_string'], 'UTF8');
if (!$connection) {
$e = oci_error();
throw new Exception($e['message']);
}
// Get rid of the test data
// This SQL from http://dbaforums.org/oracle/index.php?showtopic=4990.
$query = "begin execute immediate 'drop table ORACLE_CONTENT'; exception when others then null; end;";
$result = oci_parse($connection, $query);
if (!$result) {
$e = oci_error($connection);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
$status = oci_execute($result);
if (!$status) {
$e = oci_error($result);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
}
/**
* Implements hook_requirements().
*/
function migrate_example_oracle_requirements($phase) {
$requirements = array();
$t = get_t();
switch ($phase) {
case 'install':
// Check that the OCI8 extension is loaded
$requirements['oci8'] = array('title' => $t('Oracle extension'));
if (!extension_loaded('oci8')) {
$requirements['oci8']['description'] = $t('Migrating from an Oracle
database requires that you have the !link extension loaded in PHP.',
array('!link' => l('oci8', 'http://us.php.net/manual/en/book.oci8.php')));
$requirements['oci8']['severity'] = REQUIREMENT_ERROR;
break;
}
$sample_setting =
'<pre>
$conf[\'oracle_db\'] = array(
\'username\' => \'Oracle_username\',
\'password\' => \'Oracle_password\',
\'connection_string\' => \'//Oracle_host/SID\',
);
</pre>';
// Check that there is an Oracle database configured for use
$requirements['oracle_db'] = array('title' => $t('Oracle configuration'));
global $conf;
if (empty($conf['oracle_db']) || empty($conf['oracle_db']['username']) ||
empty($conf['oracle_db']['password']) || empty($conf['oracle_db']['connection_string'])) {
$requirements['oracle_db']['description'] = $t('You must define $conf[\'oracle_db\']
(in your site\'s settings.php file) to point to the Oracle database where
you want test data to be stored: ' . $sample_setting);
$requirements['oracle_db']['severity'] = REQUIREMENT_ERROR;
break;
}
// Check that we can connect to the Oracle db.
$requirements['oracle_connect'] = array('title' => $t('Oracle connection available'));
$connection = oci_connect($conf['oracle_db']['username'], $conf['oracle_db']['password'],
$conf['oracle_db']['connection_string'], 'UTF8');
if (!$connection) {
$e = oci_error();
$requirements['oracle_connect']['description'] = $t('Could not connect to configured
Oracle database at !conn_string. Oracle error message: !message',
array('!conn_string' => $conf['oracle_db']['connection_string'],
'!message' => $e['message']));
$requirements['oracle_connect']['severity'] = REQUIREMENT_ERROR;
break;
}
// Check for necessary privileges
$requirements['oracle_privs'] = array('title' => $t('Necessary Oracle privileges are assigned'));
$statement = oci_parse($connection, 'SELECT * FROM SESSION_PRIVS');
if (!$statement) {
$e = oci_error($connection);
$requirements['oracle_connect']['description'] = $e['message'];
$requirements['oracle_connect']['severity'] = REQUIREMENT_ERROR;
break;
}
$result = oci_execute($statement);
if (!$result) {
$e = oci_error($statement);
$requirements['oracle_connect']['description'] = $e['message'];
$requirements['oracle_connect']['severity'] = REQUIREMENT_ERROR;
break;
}
$ok = FALSE;
while ($row = oci_fetch_object($statement)) {
if ($row->PRIVILEGE == 'CREATE TABLE') {
$ok = TRUE;
break;
}
}
if (!$ok) {
$requirements['oracle_privs']['description'] = $t('The specified
username !username does not have the CREATE TABLE privilege. This privilege
is necessary to create test tables in the Oracle database.',
array('!username' => $conf['oracle_db']['username']));
$requirements['oracle_privs']['severity'] = REQUIREMENT_ERROR;
break;
}
break;
case 'update':
break;
case 'runtime':
break;
}
return $requirements;
}

88
sites/all/modules/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.migrate.inc Normal file
View File

@@ -0,0 +1,88 @@
<?php
/**
* @file
* Examples and test fodder for migration from Oracle sources. To use this example
* (and to run the corresponding tests) you must define a connection to an Oracle database
* in your settings.php. E.g.,
*
* $conf['oracle_db'] = array(
* 'username' => 'DRUPAL',
* 'password' => 'DRUPAL',
* 'connection_string' => '//oracledb/orcl',
* );
*
* The username must have the CREATE TABLE privilege, so test data can be stored for the
* example to import.
*
* See http://us.php.net/manual/en/function.oci-connect.php for more information on
* connection_string.
*/
/**
* Migration class to test importing from Oracle into nodes.
*/
class MigrateExampleOracleNode extends Migration {
public function __construct() {
parent::__construct();
$this->description = t('Example migration from Oracle into nodes.');
// Note that Oracle by default upper-cases all identifiers, so use upper-case
// for the key name and for source fields below.
$this->map = new MigrateSQLMap($this->machineName,
array(
'OID' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Content ID',
)
),
MigrateDestinationNode::getKeySchema()
);
// Source fields available from the Oracle table.
$fields = array(
'OID' => t('Source id'),
'TITLE' => t('Title'),
'BODY' => t('Description'),
'MAINIMAGE' => t('Main image'),
'CREATED' => t('Creation date'),
'UPDATED' => t('Updated date'),
);
// Oracle will usually (depending on server configuration) return only a
// date (such as 01-MAY-11) for datetime fields - you need to use TO_CHAR()
// to extract time information as well.
$query = "SELECT OID, TITLE, BODY, MAINIMAGE, TO_CHAR(CREATED, 'yyyy/mm/dd hh24:mi:ss') CREATED,
TO_CHAR(UPDATED, 'yyyy/mm/dd hh24:mi:ss') UPDATED
FROM ORACLE_CONTENT";
$count_query = "SELECT COUNT(*) FROM ORACLE_CONTENT";
// Per above, the connection info should be defined in settings.php.
global $conf;
$this->source = new MigrateSourceOracle($conf['oracle_db'], $query,
$count_query, $fields);
$this->destination = new MigrateDestinationNode('migrate_example_oracle');
// Basic fields
$this->addFieldMapping('title', 'TITLE');
$this->addFieldMapping('uid')
->defaultValue(1);
$this->addFieldMapping('body', 'BODY');
$this->addFieldMapping('field_mainimage', 'MAINIMAGE')
->description('An image blob in the DB')
->arguments(array(
'file_function' => 'file_blob',
// Alternatively, specify a column here for dynamic file name.
'source_path' => 'druplicon.png',
));
$this->addFieldMapping('created', 'CREATED');
$this->addFieldMapping('changed', 'UPDATED');
// Unmapped destination fields
$this->addUnmigratedDestinations(array('is_new', 'status', 'promote',
'revision', 'language', 'sticky', 'revision_uid', 'path'));
}
}

49
sites/all/modules/migrate/migrate_example/migrate_example_oracle/migrate_example_oracle.module Normal file
View File

@@ -0,0 +1,49 @@
<?php
include_once('migrate_example_oracle.features.inc');
/*
* Implementation of hook_migrate_api().
*/
function migrate_example_oracle_migrate_api() {
$api = array(
'api' => 2,
);
return $api;
}
/**
* Generate a text string of reproducible contents for a given length.
*
* @param int $length
* Number of characters to generate.
*
* $return
* String of the given length.
*/
function migrate_example_oracle_generate($length) {
$base = 'word '; // Five characters long
$multiplier = ($length / 5) + 1; // 80% chance of going a bit long, thus substr below
$data = str_repeat($base, $multiplier);
$data = substr($data, 0, $length);
return $data;
}
/**
* Return an array of data rows for testing Oracle import. Note that 4000 is a magic
* number for Oracle LOB datatypes, so we testing lengths above and below that limit.
*/
function migrate_example_oracle_sample_data() {
$image = file_get_contents('misc/druplicon.png');
return array(
array('oid' => 3, 'title' => 'Sample title', 'body' => 'Sample body',
'mainimage' => $image, 'created' => '2011/05/01 01:02:03',
'updated' => '2011/06/30 04:05:06'),
array('oid' => 5, 'title' => 'Another title', 'body' => migrate_example_oracle_generate(3900),
'mainimage' => $image, 'created' => '2011/08/12 07:08:09',
'updated' => '2011/12/25 10:11:12'),
array('oid' => 7, 'title' => 'Yet another title', 'body' => migrate_example_oracle_generate(4500),
'mainimage' => $image, 'created' => '2012/01/01 13:14:15',
'updated' => '2012/03/14 16:17:18'),
);
}

1236
sites/all/modules/migrate/migrate_example/wine.inc Normal file
View File

File diff suppressed because it is too large Load Diff

1253
sites/all/modules/migrate/migrate_example/wine.install.inc Normal file
View File

File diff suppressed because it is too large Load Diff

7
sites/all/modules/migrate/migrate_example/xml/0001.xml Normal file
View File

@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<producer>
<name>Lolonis Winery</name>
<description>Makers of Ladybug Red</description>
<authorid>3</authorid>
<region>Redwood Valley</region>
</producer>

4
sites/all/modules/migrate/migrate_example/xml/index.xml Normal file
View File

@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
<content>
<sourceid>0001</sourceid>
</content>

11
sites/all/modules/migrate/migrate_example/xml/positions.xml Normal file
View File

@@ -0,0 +1,11 @@
<?xml version="1.0" encoding="UTF-8"?>
<positions>
<position>
<sourceid>5</sourceid>
<name>Taster</name>
</position>
<position>
<sourceid>18</sourceid>
<name>Vintner</name>
</position>
</positions>

17
sites/all/modules/migrate/migrate_example/xml/producers.xml Normal file
View File

@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<producers>
<producer>
<sourceid>0002</sourceid>
<name>Blue Sky Winery</name>
<description>Makers of Warm Sun Blush</description>
<authorid>1</authorid>
<region>Redwood Valley</region>
</producer>
<producer>
<sourceid>0003</sourceid>
<name>Meriam Winery</name>
<description>Makers of Extra Dry Chardonnay</description>
<authorid>9</authorid>
<region>Redwood Valley</region>
</producer>
</producers>

17
sites/all/modules/migrate/migrate_example/xml/producers2.xml Normal file
View File

@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<producers>
<producer>
<sourceid>0005</sourceid>
<name>Boston Winery</name>
<description>Boston's only winery</description>
<authorid>1</authorid>
<region>Elba</region>
</producer>
<producer>
<sourceid>0008</sourceid>
<name>Nashoba Valley Winery</name>
<description>Methode champenoise</description>
<authorid>9</authorid>
<region>Chile</region>
</producer>
</producers>

10
sites/all/modules/migrate/migrate_example_baseball/README.txt Normal file
View File

@@ -0,0 +1,10 @@
An example migration from comma separated value files into Drupal nodes. Also a
good example of a DynamicMigration (i.e. the same migration class handles
multiple migrations).
We currently depend on Features module just for easy export of a content type.
The data comes from http://www.retrosheet.org/gamelogs/index.html:
The information used here was obtained free of charge from and is copyrighted by
Retrosheet. Interested parties may contact Retrosheet at "www.retrosheet.org".

986
sites/all/modules/migrate/migrate_example_baseball/migrate_example_baseball.features.field.inc Normal file
View File

@@ -0,0 +1,986 @@
<?php
/**
* Implementation of hook_field_default_fields().
*/
function migrate_example_baseball_field_default_fields() {
$fields = array();
// Exported field: 'node-migrate_example_baseball-body'
$fields['node-migrate_example_baseball-body'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(
'0' => 'node',
),
'field_name' => 'body',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(),
'translatable' => '1',
'type' => 'text_with_summary',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'hidden',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => '0',
),
'teaser' => array(
'label' => 'hidden',
'module' => 'text',
'settings' => array(
'trim_length' => 600,
),
'type' => 'text_summary_or_trimmed',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'body',
'label' => 'Body',
'required' => FALSE,
'settings' => array(
'display_summary' => TRUE,
'text_processing' => 1,
'user_register_form' => FALSE,
),
'widget' => array(
'module' => 'text',
'settings' => array(
'rows' => 20,
'summary_rows' => 5,
),
'type' => 'text_textarea_with_summary',
'weight' => '-4',
),
'widget_type' => 'text_textarea_with_summary',
),
);
// Exported field: 'node-migrate_example_baseball-field_attendance'
$fields['node-migrate_example_baseball-field_attendance'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_attendance',
'foreign keys' => array(),
'indexes' => array(),
'module' => 'number',
'settings' => array(),
'translatable' => '1',
'type' => 'number_integer',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'number',
'settings' => array(
'decimal_separator' => '.',
'prefix_suffix' => TRUE,
'scale' => 0,
'thousand_separator' => ' ',
),
'type' => 'number_integer',
'weight' => '7',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_attendance',
'label' => 'Attendance',
'required' => 0,
'settings' => array(
'max' => '',
'min' => '',
'prefix' => '',
'suffix' => '',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 0,
'module' => 'number',
'settings' => array(),
'type' => 'number',
'weight' => '5',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_duration'
$fields['node-migrate_example_baseball-field_duration'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_duration',
'foreign keys' => array(),
'indexes' => array(),
'module' => 'number',
'settings' => array(),
'translatable' => '1',
'type' => 'number_integer',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'number',
'settings' => array(
'decimal_separator' => '.',
'prefix_suffix' => TRUE,
'scale' => 0,
'thousand_separator' => ' ',
),
'type' => 'number_integer',
'weight' => '8',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_duration',
'label' => 'Duration',
'required' => 0,
'settings' => array(
'max' => '',
'min' => '',
'prefix' => '',
'suffix' => 'minutes',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 0,
'module' => 'number',
'settings' => array(),
'type' => 'number',
'weight' => '6',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_home_batters'
$fields['node-migrate_example_baseball-field_home_batters'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '9',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_home_batters',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(
'max_length' => '255',
),
'translatable' => '1',
'type' => 'text',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'above',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => '10',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_home_batters',
'label' => 'Home batters',
'required' => 0,
'settings' => array(
'text_processing' => '0',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 1,
'module' => 'text',
'settings' => array(
'size' => '60',
),
'type' => 'text_textfield',
'weight' => '7',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_home_game_number'
$fields['node-migrate_example_baseball-field_home_game_number'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_home_game_number',
'foreign keys' => array(),
'indexes' => array(),
'module' => 'number',
'settings' => array(),
'translatable' => '1',
'type' => 'number_integer',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'number',
'settings' => array(
'decimal_separator' => '.',
'prefix_suffix' => TRUE,
'scale' => 0,
'thousand_separator' => ' ',
),
'type' => 'number_integer',
'weight' => '3',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_home_game_number',
'label' => 'Home game number',
'required' => 0,
'settings' => array(
'max' => '',
'min' => '',
'prefix' => '',
'suffix' => '',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 0,
'module' => 'number',
'settings' => array(),
'type' => 'number',
'weight' => '1',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_home_pitcher'
$fields['node-migrate_example_baseball-field_home_pitcher'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_home_pitcher',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(
'max_length' => '255',
),
'translatable' => '1',
'type' => 'text',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => '11',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_home_pitcher',
'label' => 'Home starting pitcher',
'required' => 0,
'settings' => array(
'text_processing' => '0',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 1,
'module' => 'text',
'settings' => array(
'size' => '60',
),
'type' => 'text_textfield',
'weight' => '9',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_home_score'
$fields['node-migrate_example_baseball-field_home_score'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_home_score',
'foreign keys' => array(),
'indexes' => array(),
'module' => 'number',
'settings' => array(),
'translatable' => '1',
'type' => 'number_integer',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'number',
'settings' => array(
'decimal_separator' => '.',
'prefix_suffix' => TRUE,
'scale' => 0,
'thousand_separator' => ' ',
),
'type' => 'number_integer',
'weight' => '4',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_home_score',
'label' => 'Home score',
'required' => 0,
'settings' => array(
'max' => '',
'min' => '',
'prefix' => '',
'suffix' => '',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 0,
'module' => 'number',
'settings' => array(),
'type' => 'number',
'weight' => '2',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_home_team'
$fields['node-migrate_example_baseball-field_home_team'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_home_team',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(
'max_length' => '255',
),
'translatable' => '1',
'type' => 'text',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => '2',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_home_team',
'label' => 'Home team',
'required' => 0,
'settings' => array(
'text_processing' => '0',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 1,
'module' => 'text',
'settings' => array(
'size' => '60',
),
'type' => 'text_textfield',
'weight' => '0',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_outs'
$fields['node-migrate_example_baseball-field_outs'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_outs',
'foreign keys' => array(),
'indexes' => array(),
'module' => 'number',
'settings' => array(),
'translatable' => '1',
'type' => 'number_integer',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'number',
'settings' => array(
'decimal_separator' => '.',
'prefix_suffix' => TRUE,
'scale' => 0,
'thousand_separator' => ' ',
),
'type' => 'number_integer',
'weight' => '6',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_outs',
'label' => 'Outs',
'required' => 0,
'settings' => array(
'max' => '',
'min' => '',
'prefix' => '',
'suffix' => '',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 0,
'module' => 'number',
'settings' => array(),
'type' => 'number',
'weight' => '4',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_park'
$fields['node-migrate_example_baseball-field_park'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_park',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(
'max_length' => '255',
),
'translatable' => '1',
'type' => 'text',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => '1',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_park',
'label' => 'Park',
'required' => 0,
'settings' => array(
'text_processing' => '0',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 1,
'module' => 'text',
'settings' => array(
'size' => '60',
),
'type' => 'text_textfield',
'weight' => '-2',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_start_date'
$fields['node-migrate_example_baseball-field_start_date'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_start_date',
'foreign keys' => array(),
'indexes' => array(),
'module' => 'number',
'settings' => array(),
'translatable' => '1',
'type' => 'number_integer',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'number',
'settings' => array(
'decimal_separator' => '.',
'prefix_suffix' => TRUE,
'scale' => 0,
'thousand_separator' => ' ',
),
'type' => 'number_integer',
'weight' => '14',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_start_date',
'label' => 'Start date',
'required' => 0,
'settings' => array(
'max' => '',
'min' => '',
'prefix' => '',
'suffix' => '',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 0,
'module' => 'number',
'settings' => array(),
'type' => 'number',
'weight' => '12',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_visiting_batters'
$fields['node-migrate_example_baseball-field_visiting_batters'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '9',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_visiting_batters',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(
'max_length' => '255',
),
'translatable' => '1',
'type' => 'text',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'above',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => '9',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_visiting_batters',
'label' => 'Visiting batters',
'required' => 0,
'settings' => array(
'text_processing' => '0',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 1,
'module' => 'text',
'settings' => array(
'size' => '60',
),
'type' => 'text_textfield',
'weight' => '8',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_visiting_pitcher'
$fields['node-migrate_example_baseball-field_visiting_pitcher'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_visiting_pitcher',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(
'max_length' => '255',
),
'translatable' => '1',
'type' => 'text',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => '12',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_visiting_pitcher',
'label' => 'Visiting starting pitcher',
'required' => 0,
'settings' => array(
'text_processing' => '0',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 1,
'module' => 'text',
'settings' => array(
'size' => '60',
),
'type' => 'text_textfield',
'weight' => '10',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_visiting_score'
$fields['node-migrate_example_baseball-field_visiting_score'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_visiting_score',
'foreign keys' => array(),
'indexes' => array(),
'module' => 'number',
'settings' => array(),
'translatable' => '1',
'type' => 'number_integer',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'number',
'settings' => array(
'decimal_separator' => '.',
'prefix_suffix' => TRUE,
'scale' => 0,
'thousand_separator' => ' ',
),
'type' => 'number_integer',
'weight' => '5',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_visiting_score',
'label' => 'Visiting score',
'required' => 0,
'settings' => array(
'max' => '',
'min' => '',
'prefix' => '',
'suffix' => '',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 0,
'module' => 'number',
'settings' => array(),
'type' => 'number',
'weight' => '3',
),
),
);
// Exported field: 'node-migrate_example_baseball-field_visiting_team'
$fields['node-migrate_example_baseball-field_visiting_team'] = array(
'field_config' => array(
'active' => '1',
'cardinality' => '1',
'deleted' => '0',
'entity_types' => array(),
'field_name' => 'field_visiting_team',
'foreign keys' => array(
'format' => array(
'columns' => array(
'format' => 'format',
),
'table' => 'filter_format',
),
),
'indexes' => array(
'format' => array(
'0' => 'format',
),
),
'module' => 'text',
'settings' => array(
'max_length' => '255',
),
'translatable' => '1',
'type' => 'text',
),
'field_instance' => array(
'bundle' => 'migrate_example_baseball',
'default_value' => NULL,
'deleted' => '0',
'description' => '',
'display' => array(
'default' => array(
'label' => 'inline',
'module' => 'text',
'settings' => array(),
'type' => 'text_default',
'weight' => '13',
),
'teaser' => array(
'label' => 'above',
'settings' => array(),
'type' => 'hidden',
'weight' => 0,
),
),
'entity_type' => 'node',
'field_name' => 'field_visiting_team',
'label' => 'Visiting team',
'required' => 0,
'settings' => array(
'text_processing' => '0',
'user_register_form' => FALSE,
),
'widget' => array(
'active' => 1,
'module' => 'text',
'settings' => array(
'size' => '60',
),
'type' => 'text_textfield',
'weight' => '11',
),
),
);
return $fields;
}

18
sites/all/modules/migrate/migrate_example_baseball/migrate_example_baseball.features.inc Normal file
View File

@@ -0,0 +1,18 @@
<?php
/**
* Implementation of hook_node_info().
*/
function migrate_example_baseball_node_info() {
$items = array(
'migrate_example_baseball' => array(
'name' => t('migrate_example_baseball'),
'base' => 'node_content',
'description' => t('A baseball box score'),
'has_title' => '1',
'title_label' => t('Title'),
'help' => '',
),
);
return $items;
}

32
sites/all/modules/migrate/migrate_example_baseball/migrate_example_baseball.info Normal file
View File

@@ -0,0 +1,32 @@
core = "7.x"
dependencies[] = "features"
dependencies[] = "migrate"
dependencies[] = "number"
description = "Import baseball box scores."
features[field][] = "node-migrate_example_baseball-body"
features[field][] = "node-migrate_example_baseball-field_attendance"
features[field][] = "node-migrate_example_baseball-field_duration"
features[field][] = "node-migrate_example_baseball-field_home_batters"
features[field][] = "node-migrate_example_baseball-field_home_game_number"
features[field][] = "node-migrate_example_baseball-field_home_pitcher"
features[field][] = "node-migrate_example_baseball-field_home_score"
features[field][] = "node-migrate_example_baseball-field_home_team"
features[field][] = "node-migrate_example_baseball-field_outs"
features[field][] = "node-migrate_example_baseball-field_park"
features[field][] = "node-migrate_example_baseball-field_start_date"
features[field][] = "node-migrate_example_baseball-field_visiting_batters"
features[field][] = "node-migrate_example_baseball-field_visiting_pitcher"
features[field][] = "node-migrate_example_baseball-field_visiting_score"
features[field][] = "node-migrate_example_baseball-field_visiting_team"
features[node][] = "migrate_example_baseball"
files[] = "migrate_example_baseball.migrate.inc"
name = "migrate_example_baseball"
package = "Migrate Examples"
php = "5.2.4"
; Information added by drupal.org packaging script on 2012-06-02
version = "7.x-2.4"
core = "7.x"
project = "migrate"
datestamp = "1338661580"

71
sites/all/modules/migrate/migrate_example_baseball/migrate_example_baseball.install Normal file
View File

@@ -0,0 +1,71 @@
<?php
/**
* @file
* Set up the migration baseball example module.
*/
function migrate_example_baseball_enable() {
$path = dirname(__FILE__) . '/data';
migrate_example_baseball_get_files($path);
for ($i=0; $i<=9; $i++) {
$file = 'gl200' . $i . '.txt';
Migration::registerMigration('GameBaseball',
pathinfo($file, PATHINFO_FILENAME),
array('source_file' => $path . '/' . $file));
}
}
/**
* Obtain our sample data from Retrosheet.
*
* @param $path
*/
function migrate_example_baseball_get_files($path) {
// Don't replace old upper-case names
if (!file_exists("$path/GL2000.TXT") && !file_exists("$path/gl2000.txt")) {
file_prepare_directory($path, FILE_CREATE_DIRECTORY | FILE_MODIFY_PERMISSIONS);
$result = copy('http://www.retrosheet.org/gamelogs/gl2000_09.zip',
$path . '/gl2000_09.zip');
if ($result) {
$zip = new ZipArchive();
$zip->open($path . '/gl2000_09.zip');
$zip->extractTo($path);
$zip->close();
unlink("$path/gl2000_09.zip");
}
}
}
function migrate_example_baseball_uninstall() {
$bundle = 'migrate_example_baseball';
$field_names = array('field_park', 'field_home_team', 'field_home_game_number',
'field_home_score', 'field_visiting_score', 'field_outs', 'field_attendance',
'field_duration', 'field_home_batters', 'field_visiting_batters',
'field_home_pitcher', 'field_visiting_pitcher', 'field_visiting_team',
'field_start_date');
foreach ($field_names as $field_name) {
$instance = field_info_instance('node', $field_name, $bundle);
field_delete_instance($instance);
field_delete_field($field_name);
}
node_type_delete($bundle);
}
function migrate_example_baseball_disable() {
for ($i=0; $i<=9; $i++) {
$file = 'gl200' . $i . '.txt';
Migration::deregisterMigration(pathinfo($file, PATHINFO_FILENAME));
$filename = dirname(__FILE__) . '/data/' . $file;
if (file_exists($filename)) {
unlink($filename);
}
}
}
/**
* Get a copy of the sample CSV data if necessary.
*/
function migrate_example_baseball_update_7201() {
migrate_example_baseball_get_files(dirname(__FILE__) . '/data');
}

133
sites/all/modules/migrate/migrate_example_baseball/migrate_example_baseball.migrate.inc Normal file
View File

@@ -0,0 +1,133 @@
<?php
/**
* @file
* A baseball game migration example.
*/
/*
* You must implement hook_migrate_api(), setting the API level to 2, for
* your migration classes to be recognized by the Migrate module.
*/
function migrate_example_baseball_migrate_api() {
$api = array(
'api' => 2,
);
return $api;
}
/**
* A dynamic migration that is reused for each source CSV file.
*/
class GameBaseball extends DynamicMigration {
public function __construct(array $arguments) {
$this->arguments = $arguments;
parent::__construct();
$this->description = t('Import box scores from CSV file.');
// Create a map object for tracking the relationships between source rows
$this->map = new MigrateSQLMap($this->machineName,
array(
'start_date' => array('type' => 'varchar',
'length' => 8,
'not null' => TRUE,
'description' => 'Start date',
),
'home_team' => array('type' => 'varchar',
'length' => 255,
'not null' => TRUE,
'description' => 'Home team',
),
'home_game_number' => array('type' => 'int',
'not null' => TRUE,
'description' => 'Home team game number',
),
),
MigrateDestinationNode::getKeySchema()
);
// Create a MigrateSource object, which manages retrieving the input data.
$this->source = new MigrateSourceCSV($arguments['source_file'], $this->csvcolumns(), array(), $this->fields());
$this->destination = new MigrateDestinationNode('migrate_example_baseball');
$this->addFieldMapping('title', 'title')
->description('See prepareRow().');
$this->addFieldMapping('field_start_date', 'start_date');
$this->addFieldMapping('field_park', 'park_id');
$this->addFieldMapping('field_visiting_team', 'visiting_team');
$this->addFieldMapping('field_home_team', 'home_team');
$this->addFieldMapping('field_home_game_number', 'home_game_number');
$this->addFieldMapping('field_home_score', 'home_score');
$this->addFieldMapping('field_visiting_score', 'visiting_score');
$this->addFieldMapping('field_outs', 'outs');
$this->addFieldMapping('field_attendance', 'attendance');
$this->addFieldMapping('field_duration', 'duration')
->defaultValue(NULL);
$this->addFieldMapping('field_home_pitcher', 'home_pitcher');
$this->addFieldMapping('field_visiting_pitcher', 'visiting_pitcher');
$this->addFieldMapping('field_home_batters', 'home_batters')
->separator(',')
->description('See prepareRow().');
$this->addFieldMapping('field_visiting_batters', 'visiting_batters')
->separator(',')
->description('See prepareRow().');
for ($i=1; $i <= 9; $i++ ) {
$this->addFieldMapping(NULL, "visiting_batter_$i")
->description('Not needed since we use the multi-value field: visiting_batters.');
$this->addFieldMapping(NULL, "home_batter_$i")
->description('Not needed since we use the multi-value field: home_batters.');
}
}
function csvcolumns() {
// Note: Remember to subtract 1 from column number at http://www.retrosheet.org/gamelogs/glfields.txt
$columns[0] = array('start_date', 'Date of game');
$columns[3] = array('visiting_team', 'Visiting team');
$columns[6] = array('home_team', 'Home team');
$columns[8] = array('home_game_number', 'Home team game number');
$columns[9] = array('home_score', 'Home score');
$columns[10] = array('visiting_score', 'Visiting score');
$columns[11] = array('outs', 'Length of game in outs');
$columns[16] = array('park_id', 'Ballpark ID');
$columns[17] = array('attendance', 'Attendance');
$columns[18] = array('duration', 'Duration in minutes');
for ($i=1; $i <= 9; $i++ ) {
$columns[103+3*$i] = array("visiting_batter_$i", "Visiting batter $i");
$columns[130+3*$i] = array("home_batter_$i", "Home batter $i");
}
$columns[102] = array('visiting_pitcher', 'Visiting starting pitcher');
$columns[104] = array('home_pitcher', 'Home starting pitcher');
return $columns;
}
function prepareRow($row) {
// Collect all the batters into one multi-value field.
for ($i=1; $i <= 9; $i++ ) {
$key = "visiting_batter_$i";
$visiting_batters[] = $row->$key;
$key = "home_batter_$i";
$home_batters[] = $row->$key;
}
$row->visiting_batters = implode(',', $visiting_batters);
$row->home_batters = implode(',', $home_batters);
$row->title = "$row->home_team vs. $row->visiting_team. " . gmdate('M d, Y', strtotime($row->start_date));
}
/**
* Construct the machine name from the source file name.
*/
protected function generateMachineName($class_name = NULL) {
return drupal_strtolower(pathinfo($this->arguments['source_file'],
PATHINFO_FILENAME));
}
function fields() {
return array(
'title' => 'A composite field made during prepareRow().',
'home_batters' => 'An array of batters, populated during prepareRow().',
'visiting_batters' => 'An array of batters, populated during prepareRow().',
);
}
}

3
sites/all/modules/migrate/migrate_example_baseball/migrate_example_baseball.module Normal file
View File

@@ -0,0 +1,3 @@
<?php
include_once('migrate_example_baseball.features.inc');

18
sites/all/modules/migrate/migrate_ui/migrate_ui.css Normal file
View File

@@ -0,0 +1,18 @@
table.migrate-dashboard tr.migrate-running {
background-color: #CFC;
}
.migrate-running {
background-color: #CFC;
}
.migrate-option-separator {
margin-bottom:0.2em;
padding-bottom: 0.2em;
border-bottom: 1px solid #aaa;
}
#migrate-migration-info td.migrate-error {
background: #ffc9c9;
}

14
sites/all/modules/migrate/migrate_ui/migrate_ui.info Normal file
View File

@@ -0,0 +1,14 @@
name = "Migrate UI"
description = "UI for managing migration processes"
package = "Development"
;configure = admin/config/development/migrate
core = 7.x
dependencies[] = migrate
files[] = migrate_ui.module
; Information added by drupal.org packaging script on 2012-06-02
version = "7.x-2.4"
core = "7.x"
project = "migrate"
datestamp = "1338661580"

59
sites/all/modules/migrate/migrate_ui/migrate_ui.js Normal file
View File

@@ -0,0 +1,59 @@
(function ($) {
/**
* Provide the summary information for the migration detail vertical tabs.
*/
Drupal.behaviors.migrateUISummary = {
attach: function (context) {
// The drupalSetSummary method required for this behavior is not available
// on the Blocks administration page, so we need to make sure this
// behavior is processed only if setSummary is defined.
if (typeof jQuery.fn.drupalSetSummary == 'undefined') {
return;
}
$('fieldset#edit-overview', context).drupalSetSummary(function (context) {
if (!$('#owner', context).children()) {
return '<span class="error">' + Drupal.t('Missing client owner.') + '</span>';
}
});
$('fieldset#edit-destination', context).drupalSetSummary(function (context) {
total = $('tr', context).length - 2;
unmapped = $('td.migrate-error', context).length / 2;
mapped = total - unmapped;
msg = Drupal.formatPlural(mapped, '1 mapping.', '@count mapped.');
if (unmapped) {
msg = '<span class="error">' + Drupal.formatPlural(unmapped, '1 unmapped', '@count unmapped') + '</span>' + '. ' + msg;
}
return msg;
});
$('fieldset#edit-source', context).drupalSetSummary(function (context) {
total = $('tr', context).length - 2;
unmapped = $('td.migrate-error', context).length / 2;
mapped = total - unmapped;
msg = Drupal.formatPlural(mapped, '1 mapping.', '@count mapped.');
if (unmapped) {
msg = '<span class="error">' + Drupal.formatPlural(unmapped, '1 unmapped', '@count unmapped') + '</span>' + '. ' + msg;
}
return msg;
});
$('fieldset.migrate-mapping').each(function ($context) {
msg = Drupal.t('By priority: ');
var levels= {1:'OK',2:'Low',3:'Medium',4:'Blocker'};
for (level in levels) {
txt = '';
if (count = $(this).find('td.migrate-priority-' + level).length / 5) {
txt = count + ' ' + levels[level];
if (level > 1) {
txt = '<span class="error">' + txt + '</span>';
}
msg = msg + txt + '. ';
}
}
$(this).drupalSetSummary(msg);
}
)}
}
})(jQuery);

89
sites/all/modules/migrate/migrate_ui/migrate_ui.module Normal file
View File

@@ -0,0 +1,89 @@
<?php
define('MIGRATE_ACCESS_BASIC', 'migration information');
function migrate_ui_help($path, $arg) {
switch ($path) {
case 'admin/migrate':
return t('The current status of each migration defined in this system. Click on a migration name for details on its configuration.');
}
}
/**
* Implementation of hook_permission().
*/
function migrate_ui_permission() {
return array(
MIGRATE_ACCESS_BASIC => array(
'title' => t('Basic access to migration information'),
),
);
}
/**
* Implementation of hook_menu().
*/
function migrate_menu() {
$items = array();
$items['admin/content/migrate'] = array(
'title' => 'Migrate',
'type' => MENU_LOCAL_TASK | MENU_NORMAL_ITEM,
'description' => 'Monitor the creation of Drupal content from source data',
'page callback' => 'migrate_ui_dashboard',
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui/migrate_ui.pages.inc',
);
$items['admin/content/migrate/dashboard'] = array(
'title' => 'Migrate',
'type' => MENU_DEFAULT_LOCAL_TASK,
'weight' => -10,
);
$items['admin/content/migrate/configure'] = array(
'title' => 'Configure',
'type' => MENU_LOCAL_TASK,
'description' => 'Configure migration handlers',
'page callback' => 'migrate_ui_configure',
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui/migrate_ui.pages.inc',
'weight' => 10,
);
$items['admin/content/migrate/messages/%migration'] = array(
'title callback' => 'migration_title',
'title arguments' => array(4),
'description' => 'View messages from a migration',
'page callback' => 'migrate_ui_messages',
'page arguments' => array(4),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui/migrate_ui.pages.inc',
);
$items['admin/content/migrate/%migration'] = array(
'title callback' => 'migration_title',
'title arguments' => array(3),
'page callback' => 'drupal_get_form',
'page arguments' => array('migrate_migration_info', 3),
'access arguments' => array(MIGRATE_ACCESS_BASIC),
'file' => 'migrate_ui/migrate_ui.pages.inc',
);
if (FALSE) {
// Not working yet
migrate_ui_menu_add($items);
}
return $items;
}
// A menu load callback.
function migration_load($machine_name) {
if ($machine_name) {
return Migration::getInstance($machine_name);
}
}
function migration_title($migration) {
if (is_string($migration)) {
$migration = migration_load($migration);
}
return $migration->getMachineName();
}

841
sites/all/modules/migrate/migrate_ui/migrate_ui.pages.inc Normal file
View File

@@ -0,0 +1,841 @@
<?php
/**
* @file
*/
/**
* Menu callback
*/
function migrate_ui_dashboard() {
drupal_set_title(t('Migrate'));
return drupal_get_form('migrate_ui_dashboard_form');
}
/**
* Form for reviewing migrations.
*/
function migrate_ui_dashboard_form($form, &$form_state) {
$build = array();
$build['overview'] = array(
'#prefix' => '<div>',
'#markup' => migrate_overview(),
'#suffix' => '</div>',
);
$header = array(
'status' => array('data' => t('Status')),
'machinename' => array('data' => t('Migration')),
'importrows' => array('data' => t('Total rows')),
'imported' => array('data' => t('Imported')),
'unimported' => array('data' => t('Unimported')),
'messages' => array('data' => t('Messages')),
'lastthroughput' => array('data' => t('Throughput')),
'lastimported' => array('data' => t('Last imported')),
);
$migrations = migrate_migrations();
$rows = array();
foreach ($migrations as $migration) {
$row = array();
$has_counts = TRUE;
if (method_exists($migration, 'sourceCount')) {
$total = $migration->sourceCount();
if ($total < 0) {
$has_counts = FALSE;
$total = t('N/A');
}
}
else {
$has_counts = FALSE;
$total = t('N/A');
}
if (method_exists($migration, 'importedCount')) {
$imported = $migration->importedCount();
$processed = $migration->processedCount();
}
else {
$has_counts = FALSE;
$imported = t('N/A');
}
if ($has_counts) {
$unimported = $total - $processed;
}
else {
$unimported = t('N/A');
}
$status = $migration->getStatus();
switch ($status) {
case MigrationBase::STATUS_IDLE:
$status = t('Idle');
break;
case MigrationBase::STATUS_IMPORTING:
$status = t('Importing');
break;
case MigrationBase::STATUS_ROLLING_BACK:
$status = t('Rolling back');
break;
case MigrationBase::STATUS_STOPPING:
$status = t('Stopping');
break;
case MigrationBase::STATUS_DISABLED:
$status = t('Disabled');
break;
default:
$status = t('Unknown');
break;
}
$row['status'] = $status;
$machine_name = $migration->getMachineName();
$row['machinename'] =
l($machine_name, 'admin/content/migrate/' . $machine_name);
$row['importrows'] = $total;
$row['imported'] = $imported;
$row['unimported'] = $unimported;
if (is_subclass_of($migration, 'Migration')) {
$num_messages = $migration->messageCount();
$row['messages'] = $num_messages ? l($num_messages, 'admin/content/migrate/messages/' . $machine_name) : 0;
}
else {
$row['messages'] = t('N/A');
}
if (method_exists($migration, 'getLastThroughput')) {
$rate = $migration->getLastThroughput();
if ($rate == '') {
$row['lastthroughput'] = t('Unknown');
}
elseif ($status == MigrationBase::STATUS_IDLE) {
$row['lastthroughput'] = t('!rate/min', array('!rate' => $rate));
}
else {
if ($rate > 0) {
$row['lastthroughput'] = t('!rate/min, !time remaining', array('!rate' => $rate, '!time' => format_interval((60*$unimported) / $rate)));
}
else {
$row['lastthroughput'] = t('!rate/min, unknown time remaining', array('!rate' => $rate));
}
}
}
else {
$row['lastthroughput'] = t('N/A');
}
$row['lastimported'] = $migration->getLastImported();
$rows[$machine_name] = $row;
}
$build['dashboard'] = array(
'#type' => 'tableselect',
'#header' => $header,
'#options' => $rows,
'#empty' => t('No migrations'),
);
// Build the 'Update options' form.
$build['operations'] = array(
'#type' => 'fieldset',
'#title' => t('Operations'),
);
$options = array(
'import' => t('Import'),
'rollback' => t('Rollback'),
'rollback_and_import' => t('Rollback and import'),
'stop' => t('Stop'),
'reset' => t('Reset'),
);
$build['operations']['operation'] = array(
'#type' => 'select',
'#title' => t('Operation'),
'#title_display' => 'invisible',
'#options' => $options,
);
$build['operations']['submit'] = array(
'#type' => 'submit',
'#value' => t('Execute'),
'#validate' => array('migrate_ui_dashboard_validate'),
'#submit' => array('migrate_ui_dashboard_submit'),
);
$build['operations']['description'] = array(
'#prefix' => '<p>',
'#markup' => t(
'Choose an operation to run on all migrations selected above:
<ul>
<li>Import - Imports all previously unimported records from the source, plus
any records marked for update, into destination Drupal objects.</li>
<li>Rollback - Deletes all Drupal objects created by the migration.</li>
<li>Rollback and import - Performs the Rollback operation, immediately
followed by the Import operation.</li>
<li>Stop - Cleanly interrupts any import or rollback processes that may
currently be running.</li>
<li>Reset - Sometimes a migration process may fail to stop cleanly, and be
left stuck in an Importing or Rolling Back status. Choose Reset to clear
the status and permit other operations to proceed.</li>
</ul>'
),
'#postfix' => '</p>',
);
$build['operations']['options'] = array(
'#type' => 'fieldset',
'#title' => t('Options'),
'#collapsible' => TRUE,
'#collapsed' => TRUE,
);
$build['operations']['options']['update'] = array(
'#type' => 'checkbox',
'#title' => t('Update'),
'#description' => t('Check this box to update all previously-migrated content
in addition to importing new content. Leave unchecked to only import
new content'),
);
$build['operations']['options']['force'] = array(
'#type' => 'checkbox',
'#title' => t('Ignore dependencies'),
'#description' => t('Check this box to ignore dependencies when running imports
- all migrations will run whether or not their dependent migrations have
completed.'),
);
$build['operations']['options']['limit'] = array(
'#tree' => TRUE,
'#type' => 'fieldset',
'#attributes' => array('class' => array('container-inline')),
'value' => array(
'#type' => 'textfield',
'#title' => t('Limit to:'),
'#size' => 10,
),
'unit' => array(
'#type' => 'select',
'#options' => array(
'items' => t('items'),
'seconds' => t('seconds'),
),
'#description' => t('Set a limit of how many items to process for
each migration, or how long each should run.'),
),
);
return $build;
}
/**
* Validate callback for the dashboard form.
*/
function migrate_ui_dashboard_validate($form, &$form_state) {
// Error if there are no items to select.
if (!is_array($form_state['values']['dashboard']) || !count(array_filter($form_state['values']['dashboard']))) {
form_set_error('', t('No items selected.'));
}
}
/**
* Submit callback for the dashboard form.
*/
function migrate_ui_dashboard_submit($form, &$form_state) {
$operation = $form_state['values']['operation'];
$limit = $form_state['values']['limit'];
$update = $form_state['values']['update'];
$force = $form_state['values']['force'];
$machine_names = array_filter($form_state['values']['dashboard']);
$operations = array();
// Rollback in reverse order.
if (in_array($operation, array('rollback', 'rollback_and_import'))) {
$machine_names = array_reverse($machine_names);
foreach ($machine_names as $machine_name) {
$operations[] = array('migrate_ui_batch', array('rollback', $machine_name, $limit));
}
// Reset order of machines names in preparation for final operation.
$machine_names = array_reverse($machine_names);
$operation = $operation == 'rollback_and_import' ? 'import' : NULL;
}
// Perform non-rollback operation, if one exists.
if ($operation) {
foreach ($machine_names as $machine_name) {
$migration = Migration::getInstance($machine_name);
// Update (if necessary) once, before starting
if ($update && method_exists($migration, 'prepareUpdate')) {
$migration->prepareUpdate();
}
$operations[] = array('migrate_ui_batch', array($operation, $machine_name, $limit, $force));
}
}
$batch = array(
'operations' => $operations,
'title' => t('Migration processing'),
'file' => drupal_get_path('module', 'migrate_ui') . '/migrate_ui.pages.inc',
'init_message' => t('Starting migration process'),
'progress_message' => t(''),
'error_message' => t('An error occurred. Some or all of the migrate processing has failed.'),
'finished' => 'migrate_ui_batch_finish',
);
batch_set($batch);
}
/**
* Process all enabled migration processes in a browser, using the Batch API
* to break it into manageable chunks.
*
* @param $operation
* Operation to perform - 'import', 'rollback', 'stop', or 'reset'.
* @param $machine_name
* Machine name of migration to process.
* @param $limit
* An array indicating the number of items to import or rollback, or the
* number of seconds to process. Should include 'unit' (either 'items' or
* 'seconds') and 'value'.
* @param $context
* Batch API context structure
*/
function migrate_ui_batch($operation, $machine_name, $limit, $force = FALSE, &$context) {
// If we got a stop message, skip everything else
if (isset($context['results']['stopped'])) {
$context['finished'] = 1;
return;
}
$migration = Migration::getInstance($machine_name);
// Messages generated by migration processes will be captured in this global
global $_migrate_messages;
$_migrate_messages = array();
Migration::setDisplayFunction('migrate_ui_capture_message');
// Perform the requested operation
switch ($operation) {
case 'import':
$result = $migration->processImport(array('limit' => $limit, 'force' => $force));
break;
case 'rollback':
$result = $migration->processRollback(array('limit' => $limit, 'force' => $force));
break;
case 'stop':
$migration->stopProcess();
$result = Migration::RESULT_COMPLETED;
break;
case 'reset':
$migration->resetStatus();
$result = Migration::RESULT_COMPLETED;
break;
}
switch ($result) {
case Migration::RESULT_INCOMPLETE:
// Default to half-done, in case we can't get a more precise fix
$context['finished'] = .5;
if (method_exists($migration, 'sourceCount')) {
$total = $migration->sourceCount();
if ($total > 0 && method_exists($migration, 'importedCount')) {
$processed = $migration->processedCount();
switch ($operation) {
case 'import':
$to_update = $migration->updateCount();
$context['finished'] = ($processed-$to_update)/$total;
break;
case 'rollback':
$context['finished'] = ($total-$processed)/$total;
break;
}
}
}
break;
case MigrationBase::RESULT_SKIPPED:
$_migrate_messages[] = t("Skipped !name due to unfulfilled dependencies: !depends",
array(
'!name' => $machine_name,
'!depends' => implode(", ", $migration->incompleteDependencies()),
));
$context['finished'] = 1;
break;
case MigrationBase::RESULT_STOPPED:
$context['finished'] = 1;
// Skip any further operations
$context['results']['stopped'] = TRUE;
break;
default:
$context['finished'] = 1;
break;
}
// Add any messages generated in this batch to the cumulative list
foreach ($_migrate_messages as $message) {
$context['results'][] = $message;
}
// While in progress, show the cumulative list of messages
$full_message = '';
foreach ($context['results'] as $message) {
$full_message .= $message . '<br />';
}
$context['message'] = $full_message;
}
/**
* Batch API finished callback - report results
*
* @param $success
* Ignored
* @param $results
* List of results from batch processing
* @param $operations
* Ignored
*/
function migrate_ui_batch_finish($success, $results, $operations) {
unset($results['stopped']);
foreach ($results as $result) {
drupal_set_message($result);
}
}
function migrate_ui_capture_message($message, $level) {
if ($level != 'debug') {
global $_migrate_messages;
$_migrate_messages[] = $message;
}
}
/**
* Menu callback for messages page
*/
function migrate_ui_messages($migration) {
$build = $rows = array();
$header = array(
array('data' => t('Source ID'), 'field' => 'sourceid1', 'sort' => 'asc'),
array('data' => t('Level'), 'field' => 'level'),
array('data' => t('Message'), 'field' => 'message'),
);
if (is_string($migration)) {
$migration = migration_load($migration);
}
// TODO: need a general MigrateMap API
$messages = $migration->getMap()->getConnection()
->select($migration->getMap()->getMessageTable(), 'msg')
->extend('PagerDefault')
->extend('TableSort')
->orderByHeader($header)
->limit(500)
->fields('msg')
->execute();
foreach ($messages as $message) {
$classes[] = $message->level <= MigrationBase::MESSAGE_WARNING ? 'migrate-error' : '';
$rows[] = array(
array('data' => $message->sourceid1, 'class' => $classes), // TODO: deal with compound keys
array('data' => $migration->getMessageLevelName($message->level), 'class' => $classes),
array('data' => $message->message, 'class' => $classes),
);
unset($classes);
}
$build['messages'] = array(
'#theme' => 'table',
'#header' => $header,
'#rows' => $rows,
'#empty' => t('No messages'),
'#attached' => array(
'css' => array(drupal_get_path('module', 'migrate_ui') . '/migrate_ui.css'),
),
);
$build['migrate_ui_pager'] = array('#theme' => 'pager');
return $build;
}
/**
* Menu callback function for migration view page.
*/
function migrate_migration_info($form, $form_state, $migration) {
if (is_string($migration)) {
$migration = migration_load($migration);
}
$has_mappings = method_exists($migration, 'getFieldMappings');
$form = array();
if ($has_mappings) {
$field_mappings = $migration->getFieldMappings();
// Identify what destination and source fields are mapped
foreach ($field_mappings as $mapping) {
$source_field = $mapping->getSourceField();
$destination_field = $mapping->getDestinationField();
$source_fields[$source_field] = $source_field;
$destination_fields[$destination_field] = $destination_field;
}
$form['detail'] = array(
'#type' => 'vertical_tabs',
'#attached' => array(
'js' => array(drupal_get_path('module', 'migrate_ui') . '/migrate_ui.js'),
'css' => array(drupal_get_path('module', 'migrate_ui') . '/migrate_ui.css'),
),
);
}
else {
$form['detail'] = array(
'#type' => 'fieldset',
);
}
$form['overview'] = array(
'#type' => 'fieldset',
'#title' => t('Overview'),
'#group' => 'detail',
);
$team = array();
foreach ($migration->getTeam() as $member) {
$email_address = $member->getEmailAddress();
$team[$member->getGroup()][] =
$member->getName() . ' <' . l($email_address, 'mailto:' . $email_address) . '>';
}
foreach ($team as $group => $list) {
$form['overview'][$group] = array(
'#type' => 'item',
'#title' => $group,
'#markup' => implode(', ', $list),
);
}
$dependencies = $migration->getHardDependencies();
if (count($dependencies) > 0) {
$form['overview']['dependencies'] = array(
'#title' => t('Dependencies') ,
'#markup' => implode(', ', $dependencies),
'#type' => 'item',
);
}
$soft_dependencies = $migration->getSoftDependencies();
if (count($soft_dependencies) > 0) {
$form['overview']['soft_dependencies'] = array(
'#title' => t('Soft Dependencies'),
'#markup' => implode(', ', $soft_dependencies),
'#type' => 'item',
);
}
if ($has_mappings) {
switch ($migration->getSystemOfRecord()) {
case Migration::SOURCE:
$system_of_record = t('Source data');
break;
case Migration::DESTINATION:
$system_of_record = t('Destination data');
break;
default:
$system_of_record = t('Unknown');
break;
}
$form['overview']['system_of_record'] = array(
'#type' => 'item',
'#title' => t('System of record:'),
'#markup' => $system_of_record,
);
}
$form['overview']['description'] = array(
'#title' => t('Description:'),
'#markup' => $migration->getDescription(),
'#type' => 'item',
);
if ($has_mappings) {
// Destination field information
$form['destination'] = array(
'#type' => 'fieldset',
'#title' => t('Destination'),
'#group' => 'detail',
'#description' =>
t('<p>These are the fields available in the destination of this
migration. The machine names listed here are those available to be used
as the first parameter to $this->addFieldMapping() in your Migration
class constructor. <span class="error">Unmapped fields are red</span>.</p>'),
);
$destination = $migration->getDestination();
$form['destination']['type'] = array(
'#type' => 'item',
'#title' => t('Type'),
'#markup' => (string)$destination,
);
$dest_key = $destination->getKeySchema();
$header = array(t('Machine name'), t('Description'));
$rows = array();
foreach ($destination->fields($migration) as $machine_name => $description) {
$classes = array();
if (isset($dest_key[$machine_name])) {
// Identify primary key
$machine_name .= ' ' . t('(PK)');
}
else {
// Add class for mapped/unmapped. Used in summary.
$classes[] = !isset($destination_fields[$machine_name]) ? 'migrate-error' : '';
}
$rows[] = array(array('data' => $machine_name, 'class' => $classes), array('data' => $description, 'class' => $classes));
}
$classes = array();
$form['destination']['fields'] = array(
'#theme' => 'table',
'#header' => $header,
'#rows' => $rows,
'#empty' => t('No fields'),
);
// TODO: Get source_fields from arguments
$form['source'] = array(
'#type' => 'fieldset',
'#title' => t('Source'),
'#group' => 'detail',
'#description' =>
t('<p>These are the fields available from the source of this
migration. The machine names listed here are those available to be used
as the second parameter to $this->addFieldMapping() in your Migration
class constructor. <span class="error">Unmapped fields are red</span>.</p>'),
);
$source = $migration->getSource();
$form['source']['query'] = array(
'#type' => 'item',
'#title' => t('Query'),
'#markup' => '<pre>' . $source . '</pre>',
);
$source_key = $migration->getMap()->getSourceKey();
$header = array(t('Machine name'), t('Description'));
$rows = array();
foreach ($source->fields() as $machine_name => $description) {
if (isset($source_key[$machine_name])) {
// Identify primary key
$machine_name .= ' ' . t('(PK)');
}
else {
// Add class for mapped/unmapped. Used in summary.
$classes = !isset($source_fields[$machine_name]) ? 'migrate-error' : '';
}
$rows[] = array(array('data' => $machine_name, 'class' => $classes), array('data' => $description, 'class' => $classes));
}
$classes = array();
$form['source']['fields'] = array(
'#theme' => 'table',
'#header' => $header,
'#rows' => $rows,
'#empty' => t('No fields'),
);
$header = array(t('Destination'), t('Source'), t('Default'), t('Description'), t('Priority'));
// First group the mappings
$descriptions = array();
$source_fields = $source->fields();
$destination_fields = $destination->fields($migration);
foreach ($field_mappings as $mapping) {
// Validate source and destination fields actually exist
$source_field = $mapping->getSourceField();
$destination_field = $mapping->getDestinationField();
if (!is_null($source_field) && !isset($source_fields[$source_field])) {
drupal_set_message(t('"!source" was used as source field in the
"!destination" mapping but is not in list of source fields', array(
'!source' => $source_field,
'!destination' => $destination_field
)),
'warning');
}
if (!is_null($destination_field) && !isset($destination_fields[$destination_field])) {
drupal_set_message(t('"!destination" was used as destination field in
"!source" mapping but is not in list of destination fields', array(
'!source' => $source_field,
'!destination' => $destination_field)),
'warning');
}
$descriptions[$mapping->getIssueGroup()][] = $mapping;
}
// Put out each group header
foreach ($descriptions as $group => $mappings) {
$form[$group] = array(
'#type' => 'fieldset',
'#title' => t('Mapping: !group', array('!group' => $group)),
'#group' => 'detail',
'#attributes' => array('class' => array('migrate-mapping')),
);
$rows = array();
foreach ($mappings as $mapping) {
$default = $mapping->getDefaultValue();
if (is_array($default)) {
$default = implode(',', $default);
}
$issue_priority = $mapping->getIssuePriority();
if (!is_null($issue_priority)) {
$classes[] = 'migrate-priority-' . $issue_priority;
$priority = MigrateFieldMapping::$priorities[$issue_priority];
$issue_pattern = $migration->getIssuePattern();
$issue_number = $mapping->getIssueNumber();
if (!is_null($issue_pattern) && !is_null($issue_number)) {
$priority .= ' (' . l(t('#') . $issue_number, str_replace(':id:', $issue_number,
$issue_pattern)) . ')';
}
if ($issue_priority != MigrateFieldMapping::ISSUE_PRIORITY_OK) {
$classes[] = 'migrate-error';
}
}
else {
$priority = t('OK');
$classes[] = 'migrate-priority-' . 1;
}
$row = array(
array('data' => $mapping->getDestinationField(), 'class' => $classes),
array('data' => $mapping->getSourceField(), 'class' => $classes),
array('data' => $default, 'class' => $classes),
array('data' => $mapping->getDescription(), 'class' => $classes),
array('data' => $priority, 'class' => $classes),
);
$rows[] = $row;
$classes = array();
}
$form[$group]['table'] = array(
'#theme' => 'table',
'#header' => $header,
'#rows' => $rows,
);
}
}
return $form;
}
/**
* Menu callback
*/
function migrate_ui_configure() {
drupal_set_title(t('Migrate configuration'));
return drupal_get_form('migrate_ui_configure_form');
}
/**
* Form for reviewing migrations.
*/
function migrate_ui_configure_form($form, &$form_state) {
$build = array();
$build['description'] = array(
'#prefix' => '<div>',
'#markup' => t('In some cases, such as when a handler for a contributed module is
implemented in both migrate_extras and the module itself, you may need to disable
a particular handler. In this case, you may uncheck the undesired handler below.'),
'#suffix' => '</div>',
);
$build['destination'] = array(
'#type' => 'fieldset',
'#title' => t('Destination handlers'),
'#collapsible' => TRUE,
);
$header = array(
'module' => array('data' => t('Module')),
'class' => array('data' => t('Class')),
'types' => array('data' => t('Destination types handled')),
);
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
$class_list = _migrate_class_list('MigrateDestinationHandler');
$rows = array();
$default_values = array();
foreach ($class_list as $class_name => $handler) {
$row = array();
$module = db_select('registry', 'r')
->fields('r', array('module'))
->condition('name', $class_name)
->condition('type', 'class')
->execute()
->fetchField();
$row['module'] = $module;
$row['class'] = $class_name;
$row['types'] = implode(', ', $handler->getTypesHandled());
$default_values[$class_name] = !in_array($class_name, $disabled);
$rows[$class_name] = $row;
}
$build['destination']['destination_handlers'] = array(
'#type' => 'tableselect',
'#header' => $header,
'#options' => $rows,
'#default_value' => $default_values,
'#empty' => t('No destination handlers found'),
);
$build['field'] = array(
'#type' => 'fieldset',
'#title' => t('Field handlers'),
'#collapsible' => TRUE,
);
$header = array(
'module' => array('data' => t('Module')),
'class' => array('data' => t('Class')),
'types' => array('data' => t('Field types handled')),
);
$class_list = _migrate_class_list('MigrateFieldHandler');
$rows = array();
$default_values = array();
foreach ($class_list as $class_name => $handler) {
$row = array();
$module = db_select('registry', 'r')
->fields('r', array('module'))
->condition('name', $class_name)
->condition('type', 'class')
->execute()
->fetchField();
$row['module'] = $module;
$row['class'] = $class_name;
$row['types'] = implode(', ', $handler->getTypesHandled());
$default_values[$class_name] = !in_array($class_name, $disabled);
$rows[$class_name] = $row;
}
$build['field']['field_handlers'] = array(
'#type' => 'tableselect',
'#header' => $header,
'#options' => $rows,
'#default_value' => $default_values,
'#empty' => t('No field handlers found'),
);
$build['submit'] = array(
'#type' => 'submit',
'#value' => t('Save handler statuses'),
'#submit' => array('migrate_ui_configure_submit'),
);
return $build;
}
/**
* Submit callback for the dashboard form.
*/
function migrate_ui_configure_submit($form, &$form_state) {
$disabled = array();
foreach ($form_state['values']['destination_handlers'] as $class => $value) {
if (!$value) {
$disabled[] = $class;
}
}
foreach ($form_state['values']['field_handlers'] as $class => $value) {
if (!$value) {
$disabled[] = $class;
}
}
variable_set('migrate_disabled_handlers', serialize($disabled));
if (!empty($disabled)) {
drupal_set_message(t('The following handler classes are disabled: @classes',
array('@classes' => implode(', ', $disabled))));
}
else {
drupal_set_message(t('No handler classes are currently disabled.'));
}
}

295
sites/all/modules/migrate/plugins/destinations/comment.inc Normal file
View File

@@ -0,0 +1,295 @@
<?php
/**
* @file
* Support for comment destinations.
*/
// TODO:
// Make sure this works with updates, explicit destination keys
/**
* Destination class implementing migration into comments.
*/
class MigrateDestinationComment extends MigrateDestinationEntity {
static public function getKeySchema() {
return array(
'cid' => array(
'type' => 'int',
'unsigned' => TRUE,
'description' => 'ID of destination entity',
),
);
}
/**
* Save the original setting of comment_maintain_node_statistics
* @var boolean
*/
protected $maintainNodeStatistics;
/**
* Return an options array for comment destinations.
*
* @param string $language
* Default language for comments created via this destination class.
* @param string $text_format
* Default text format for comments created via this destination class.
*/
static public function options($language, $text_format) {
return compact('language', 'text_format');
}
/**
* Basic initialization
*
* @param string $bundle
* A.k.a. the content type (page, article, etc.) of the ... comment?.
* @param array $options
* Options applied to comments.
*/
public function __construct($bundle, array $options = array()) {
parent::__construct('comment', $bundle, $options);
}
/**
* Returns a list of fields available to be mapped for comments attached to
* a particular bundle (node type)
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array();
// First the core (comment table) properties
$fields['cid'] = t('Comment: <a href="@doc">Existing comment ID</a>',
array('@doc' => 'http://drupal.org/node/1349714#cid'));
$fields['nid'] = t('Comment: <a href="@doc">Node (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#nid'));
$fields['uid'] = t('Comment: <a href="@doc">User (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#uid'));
$fields['pid'] = t('Comment: <a href="@doc">Parent (by Drupal ID)</a>',
array('@doc' => 'http://drupal.org/node/1349714#pid'));
$fields['subject'] = t('Comment: <a href="@doc">Subject</a>',
array('@doc' => 'http://drupal.org/node/1349714#subject'));
$fields['created'] = t('Comment: <a href="@doc">Created timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349714#created'));
$fields['changed'] = t('Comment: <a href="@doc">Modified timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349714#changed'));
$fields['status'] = t('Comment: <a href="@doc">Status</a>',
array('@doc' => 'http://drupal.org/node/1349714#status'));
$fields['hostname'] = t('Comment: <a href="@doc">Hostname/IP address</a>',
array('@doc' => 'http://drupal.org/node/1349714#hostname'));
$fields['name'] = t('Comment: <a href="@doc">User name (not username)</a>',
array('@doc' => 'http://drupal.org/node/1349714#name'));
$fields['mail'] = t('Comment: <a href="@doc">Email address</a>',
array('@doc' => 'http://drupal.org/node/1349714#mail'));
$fields['homepage'] = t('Comment: <a href="@doc">Homepage</a>',
array('@doc' => 'http://drupal.org/node/1349714#homepage'));
$fields['language'] = t('Comment: <a href="@doc">Language</a>',
array('@doc' => 'http://drupal.org/node/1349714#language'));
$fields['thread'] = t('Comment: <a href="@doc">Thread</a>',
array('@doc' => 'http://drupal.org/node/1349714#thread'));
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
$fields += migrate_handler_invoke_all('Comment', 'fields', $this->entityType, $this->bundle, $migration);
return $fields;
}
/**
* Delete a batch of comments at once.
*
* @param $cids
* Array of comment IDs to be deleted.
*/
public function bulkRollback(array $cids) {
migrate_instrument_start('comment_delete_multiple');
$this->prepareRollback($cids);
$result = comment_delete_multiple($cids);
$this->completeRollback($cids);
migrate_instrument_stop('comment_delete_multiple');
return $result;
}
/**
* Import a single comment.
*
* @param $comment
* Comment object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields (cid only in this case) of the comment that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $comment, stdClass $row) {
$migration = Migration::currentMigration();
// Updating previously-migrated content?
if (isset($row->migrate_map_destid1)) {
if (isset($comment->cid)) {
if ($comment->cid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming cid !cid and map destination nid !destid1 don't match",
array('!cid' => $comment->cid, '!destid1' => $row->migrate_map_destid1)));
}
}
else {
$comment->cid = $row->migrate_map_destid1;
}
}
// Fix up timestamps
if (isset($comment->created)) {
$comment->created = MigrationBase::timestamp($comment->created);
}
if (isset($comment->changed)) {
$comment->changed = MigrationBase::timestamp($comment->changed);
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($comment->cid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination cid provided'));
}
$rawcomment = $comment;
$old_comment = comment_load($comment->cid);
if (empty($old_comment)) {
throw new MigrateException(t('System-of-record is DESTINATION, but commend !cid does not exist',
array('!cid' => $comment->cid)));
}
if (!isset($comment->nid)) {
$comment->nid = $old_comment->nid;
}
if (!isset($comment->created)) {
$comment->created = $old_comment->created;
}
if (!isset($comment->changed)) {
$comment->changed = $old_comment->changed;
}
$this->prepare($comment, $row);
foreach ($rawcomment as $field => $value) {
$old_comment->$field = $comment->$field;
}
$comment = $old_comment;
}
else {
// Set some default properties.
$defaults = array(
'language' => $this->language,
'node_type' => $this->bundle,
'subject' => '',
'status' => COMMENT_PUBLISHED,
'uid' => 0,
'cid' => 0,
'pid' => 0,
);
foreach ($defaults as $field => $value) {
if (!isset($comment->$field)) {
$comment->$field = $value;
}
}
$this->prepare($comment, $row);
// Make sure we have a nid
if (!isset($comment->nid) || !$comment->nid) {
throw new MigrateException(t('No node ID provided for comment'));
}
// comment_save() hardcodes hostname, so if we're trying to set it we
// need to save it and apply it after
if (isset($comment->hostname)) {
$hostname = $comment->hostname;
}
}
if (isset($comment->cid) && $comment->cid) {
$updating = TRUE;
}
else {
$updating = FALSE;
}
migrate_instrument_start('comment_save');
comment_save($comment);
migrate_instrument_stop('comment_save');
if (isset($hostname) && isset($comment->cid) && $comment->cid > 0) {
db_update('comment')
->fields(array('hostname' => $hostname))
->condition('cid', $comment->cid)
->execute();
}
$this->complete($comment, $row);
if (isset($comment->cid) && $comment->cid > 0) {
$return = array($comment->cid);
if ($updating) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
}
else {
$return = FALSE;
}
return $return;
}
public function preImport() {
// If maintaining node statistics is enabled, temporarily disable it
$this->maintainNodeStatistics =
variable_get('comment_maintain_node_statistics', TRUE);
if ($this->maintainNodeStatistics) {
$GLOBALS['conf']['comment_maintain_node_statistics'] = FALSE;
}
}
public function postImport() {
// If originally enabled, re-enable and rebuild the stats
if ($this->maintainNodeStatistics) {
$GLOBALS['conf']['comment_maintain_node_statistics'] = TRUE;
// Copied from devel_rebuild_node_comment_statistics
// Empty table
db_truncate('node_comment_statistics')->execute();
// TODO: DBTNG. Ignore keyword is Mysql only? Is only used in the rare case when
// two comments on the same node share same timestamp.
$sql = "
INSERT IGNORE INTO {node_comment_statistics} (nid, cid, last_comment_timestamp, last_comment_name, last_comment_uid, comment_count) (
SELECT c.nid, c.cid, c.created, c.name, c.uid, c2.comment_count FROM {comment} c
JOIN (
SELECT c.nid, MAX(c.created) AS created, COUNT(*) AS comment_count FROM {comment} c WHERE status=:published GROUP BY c.nid
) AS c2 ON c.nid = c2.nid AND c.created=c2.created
)";
db_query($sql, array(':published' => COMMENT_PUBLISHED));
// Insert records into the node_comment_statistics for nodes that are missing.
$query = db_select('node', 'n');
$query->leftJoin('node_comment_statistics', 'ncs', 'ncs.nid = n.nid');
$query->addField('n', 'changed', 'last_comment_timestamp');
$query->addField('n', 'uid', 'last_comment_uid');
$query->addField('n', 'nid');
$query->addExpression('0', 'comment_count');
$query->addExpression('NULL', 'last_comment_name');
$query->isNull('ncs.comment_count');
db_insert('node_comment_statistics')
->from($query)
->execute();
}
}
}
class MigrateCommentNodeHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('node'));
}
public function fields($entity_type, $bundle) {
$fields = array();
$fields['comment'] = t('Whether comments may be posted to the node');
return $fields;
}
}

174
sites/all/modules/migrate/plugins/destinations/entity.inc Normal file
View File

@@ -0,0 +1,174 @@
<?php
/**
* @file
* Defines base for migration destinations implemented as Drupal entities.
*/
/**
* Abstract base class for entity-based destination handling. Holds common
* Field API-related functions.
*/
abstract class MigrateDestinationEntity extends MigrateDestination {
/**
* The entity type (node, user, taxonomy_term, etc.) of the destination.
*
* @var string
*/
protected $entityType;
public function getEntityType() {
return $this->entityType;
}
/**
* The bundle (node type, vocabulary, etc.) of the destination.
*
* @var string
*/
protected $bundle;
public function getBundle() {
return $this->bundle;
}
/**
* Default language for text fields in this destination.
*
* @var string
*/
protected $language;
public function getLanguage() {
return $this->language;
}
/**
* Default input format for text fields in this destination.
*
* @var int
*/
protected $textFormat;
public function getTextFormat() {
return $this->textFormat;
}
/**
* Simply save the key schema.
*
* @param array $key_schema
*/
public function __construct($entity_type, $bundle, array $options = array()) {
parent::__construct();
$this->entityType = $entity_type;
$this->bundle = $bundle;
$this->language = isset($options['language']) ? $options['language'] : LANGUAGE_NONE;
$this->textFormat = isset($options['text_format']) ? $options['text_format'] : filter_fallback_format();
}
public function __toString() {
// TODO: Link to configuration page
if ($this->entityType == $this->bundle) {
$output = t('%type', array('%type' => $this->entityType));
}
else {
$output = t('%type (%bundle)',
array('%type' => $this->entityType, '%bundle' => $this->bundle));
}
// TODO: Non-default language, input format
return $output;
}
/**
* Give handlers a shot at cleaning up before an entity has been rolled back.
*
* @param $entity_id
* ID of the entity about to be deleted..
*/
public function prepareRollback($entity_id) {
$migration = Migration::currentMigration();
// Call any general entity handlers (in particular, the builtin field handler)
migrate_handler_invoke_all('Entity', 'prepareRollback', $entity_id);
// Then call any entity-specific handlers
migrate_handler_invoke_all($this->entityType, 'prepareRollback', $entity_id);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($entity_id);
}
}
/**
* Give handlers a shot at cleaning up after an entity has been rolled back.
*
* @param $entity_id
* ID of the entity which has been deleted.
*/
public function completeRollback($entity_id) {
$migration = Migration::currentMigration();
// Call any general entity handlers (in particular, the builtin field handler)
migrate_handler_invoke_all('Entity', 'completeRollback', $entity_id);
// Then call any entity-specific handlers
migrate_handler_invoke_all($this->entityType, 'completeRollback', $entity_id);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($entity_id);
}
}
/**
* Give handlers a shot at modifying the object before saving it.
*
* @param $entity
* Entity object to build. Prefilled with any fields mapped in the Migration.
* @param $source_row
* Raw source data object - passed through to prepare handlers.
*/
public function prepare($entity, stdClass $source_row) {
// Add source keys for debugging and identification of migrated data by hooks.
/* TODO: Restore
foreach ($migration->sourceKeyMap() as $field_name => $key_name) {
$keys[$key_name] = $source_row->$field_name;
}
*/
$migration = Migration::currentMigration();
$entity->migrate = array(
// 'source_keys' => $keys,
'machineName' => $migration->getMachineName(),
);
// Call any general entity handlers (in particular, the builtin field handler)
migrate_handler_invoke_all('Entity', 'prepare', $entity, $source_row);
// Then call any entity-specific handlers
migrate_handler_invoke_all($this->entityType, 'prepare', $entity, $source_row);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($entity, $source_row);
}
}
/**
* Give handlers a shot at modifying the object (or taking additional action)
* after saving it.
*
* @param $object
* Entity object to build. This is the complete object after saving.
* @param $source_row
* Raw source data object - passed through to complete handlers.
*/
public function complete($entity, stdClass $source_row) {
// Call any general entity handlers (in particular, the builtin field handler)
migrate_handler_invoke_all('Entity', 'complete', $entity, $source_row);
// Then call any entity-specific handlers
migrate_handler_invoke_all($this->entityType, 'complete', $entity, $source_row);
// Then call any complete handler for this specific Migration.
$migration = Migration::currentMigration();
if (method_exists($migration, 'complete')) {
try {
$migration->complete($entity, $source_row);
}
catch (Exception $e) {
// If we catch any errors here, save the messages without letting
// the exception prevent the saving of the entity being recorded.
$migration->saveMessage($e->getMessage());
}
}
}
}

671
sites/all/modules/migrate/plugins/destinations/fields.inc Normal file
View File

@@ -0,0 +1,671 @@
<?php
/**
* @file
* Support for processing entity fields
*/
class MigrateFieldsEntityHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('entity'));
}
/**
* Implementation of MigrateDestinationHandler::fields().
*
* @param $entity_type
* The entity type (node, user, etc.) for which to list fields.
* @param $bundle
* The bundle (article, blog, etc.), if any, for which to list fields.
* @param Migration $migration
* Optionally, the migration providing the context.
* @return array
* An array keyed by field name, with field descriptions as values.
*/
public function fields($entity_type, $bundle, $migration = NULL) {
$fields = array();
$field_instance_info = field_info_instances($entity_type, $bundle);
foreach ($field_instance_info as $machine_name => $instance) {
$field_info = field_info_field($machine_name);
$type = $field_info['type'];
$fields[$machine_name] = t('Field:') . ' ' . $instance['label'] .
' (' . $field_info['type'] . ')';
// Look for subfields
$class_list = _migrate_class_list('MigrateFieldHandler');
$disabled = unserialize(variable_get('migrate_disabled_handlers', serialize(array())));
foreach ($class_list as $class_name => $handler) {
if (!in_array($class_name, $disabled) && $handler->handlesType($type)
&& method_exists($handler, 'fields')) {
migrate_instrument_start($class_name . '->fields');
$subfields = call_user_func(array($handler, 'fields'), $type,
$machine_name, $migration);
migrate_instrument_stop($class_name . '->fields');
foreach ($subfields as $subfield_name => $subfield_label) {
$fields[$machine_name . ':' . $subfield_name] = $subfield_label;
}
}
}
}
return $fields;
}
public function prepare($entity, stdClass $row) {
migrate_instrument_start('MigrateDestinationEntity->prepareFields');
// Look for Field API fields attached to this destination and handle appropriately
$migration = Migration::currentMigration();
$destination = $migration->getDestination();
$entity_type = $destination->getEntityType();
$bundle = $destination->getBundle();
$instances = field_info_instances($entity_type, $bundle);
foreach ($instances as $machine_name => $instance) {
if (property_exists($entity, $machine_name)) {
// Normalize to an array
if (!is_array($entity->$machine_name)) {
$entity->$machine_name = array($entity->$machine_name);
}
$field_info = field_info_field($machine_name);
$entity->$machine_name = migrate_field_handler_invoke_all($entity, $field_info,
$instance, $entity->$machine_name);
}
}
migrate_instrument_stop('MigrateDestinationEntity->prepareFields');
}
public function complete($entity, stdClass $row) {
migrate_instrument_start('MigrateDestinationEntity->completeFields');
// Look for Field API fields attached to this destination and handle appropriately
$migration = Migration::currentMigration();
$destination = $migration->getDestination();
$entity_type = $destination->getEntityType();
$bundle = $destination->getBundle();
$instances = field_info_instances($entity_type, $bundle);
foreach ($instances as $machine_name => $instance) {
if (property_exists($entity, $machine_name)) {
// Normalize to an array
if (!is_array($entity->$machine_name)) {
$entity->$machine_name = array($entity->$machine_name);
}
$field_info = field_info_field($machine_name);
migrate_field_handler_invoke_all($entity, $field_info,
$instance, $entity->$machine_name, 'complete');
}
}
migrate_instrument_stop('MigrateDestinationEntity->completeFields');
}
}
abstract class MigrateFieldHandler extends MigrateHandler {
// Derived classes are expected to implement one or both of the prepare/complete
// handlers.
// abstract public function prepare($entity, array $field_info, array $instance, array $values);
// abstract public function complete($entity, array $field_info, array $instance, array $values);
/**
* Determine the language of the field
*
* @param $entity
* @param $field_info
* @param $arguments
* @return string language code
*/
function getFieldLanguage($entity, $field_info, array $arguments) {
$migration = Migration::currentMigration();
switch (TRUE) {
case !field_is_translatable($migration->getDestination()->getEntityType(), $field_info):
return LANGUAGE_NONE;
case isset($arguments['language']):
return $arguments['language'];
case !empty($entity->language) && $entity->language != LANGUAGE_NONE:
return $entity->language;
break;
default:
return $migration->getDestination()->getLanguage();
}
}
}
/**
* Base class for creating field handlers for fields with a single value.
*
* To use this class just extend it and pass key where the field's value should
* be stored to the constructor, then register the type(s):
* @code
* class MigrateLinkFieldHandler extends MigrateSimpleFieldHandler {
* public function __construct() {
* parent::__construct('url');
* $this->registerTypes(array('link'));
* }
* }
* @endcode
*/
abstract class MigrateSimpleFieldHandler extends MigrateFieldHandler {
protected $fieldValueKey = 'value';
protected $skipEmpty = FALSE;
/**
* Construct a simple field handler.
*
* @param $options
* Array of options (rather than unamed parameters so you don't have to
* what TRUE or FALSE means). The following keys are used:
* - 'value_key' string with the name of the key in the fields value array.
* - 'skip_empty' Boolean indicating that empty values should not be saved.
*/
public function __construct($options = array()) {
if (isset($options['value_key'])) {
$this->fieldValueKey = $options['value_key'];
}
if (isset($options['skip_empty'])) {
$this->skipEmpty = $options['skip_empty'];
}
}
public function prepare($entity, array $field_info, array $instance, array $values) {
$arguments = array();
if (isset($values['arguments'])) {
$arguments = $values['arguments'];
unset($values['arguments']);
}
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
// Let the derived class skip empty values.
if ($this->skipEmpty) {
$values = array_filter($values, array($this, 'notNull'));
}
// Setup the Field API array for saving.
$delta = 0;
foreach ($values as $value) {
if (is_array($language)) {
$current_language = $language[$delta];
}
else {
$current_language = $language;
}
$return[$current_language][] = array($this->fieldValueKey => $value);
$delta++;
}
return isset($return) ? $return : NULL;
}
/**
* Returns TRUE only for values which are not NULL.
*
* @param $value
* @return bool
*/
protected function notNull($value) {
return !is_null($value);
}
}
class MigrateTextFieldHandler extends MigrateFieldHandler {
public function __construct() {
$this->registerTypes(array('text', 'text_long', 'text_with_summary'));
}
static function arguments($summary = NULL, $format = NULL, $language = NULL) {
$arguments = array();
if (!is_null($summary)) {
$arguments['summary'] = $summary;
}
if (!is_null($format)) {
$arguments['format'] = $format;
}
if (!is_null($language)) {
$arguments['language'] = $language;
}
return $arguments;
}
public function fields($type) {
$fields = array();
if ($type == 'text_with_summary') {
$fields['summary'] = t('Subfield: Summary of field contents');
}
$fields += array(
'format' => t('Subfield: Text format for the field'),
'language' => t('Subfield: Language for the field'),
);
return $fields;
}
public function prepare($entity, array $field_info, array $instance, array $values) {
if (isset($values['arguments'])) {
$arguments = $values['arguments'];
unset($values['arguments']);
}
else {
$arguments = array();
}
$migration = Migration::currentMigration();
$destination = $migration->getDestination();
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
$max_length = isset($field_info['settings']['max_length']) ?
$field_info['settings']['max_length'] : 0;
// Setup the standard Field API array for saving.
$delta = 0;
foreach ($values as $value) {
$item = array();
if (isset($arguments['summary'])) {
if (is_array($arguments['summary'])) {
$item['summary'] = $arguments['summary'][$delta];
}
else {
$item['summary'] = $arguments['summary'];
}
}
if (isset($arguments['format'])) {
if (is_array($arguments['format'])) {
$format = $arguments['format'][$delta];
}
else {
$format = $arguments['format'];
}
}
else {
$format = $destination->getTextFormat();
}
$item['format'] = $item['value_format'] = $format;
// Make sure the value will fit
if ($max_length) {
$item['value'] = drupal_substr($value, 0, $max_length);
if (!empty($arguments['track_overflow'])) {
$value_length = drupal_strlen($value);
if ($value_length > $max_length) {
$migration->saveMessage(
t('Value for field !field exceeds max length of !max_length, actual length is !length',
array('!field' => $instance['field_name'], '!max_length' => $max_length,
'!length' => $value_length)),
Migration::MESSAGE_INFORMATIONAL);
}
}
}
else {
$item['value'] = $value;
}
if (is_array($language)) {
$current_language = $language[$delta];
}
else {
$current_language = $language;
}
$return[$current_language][] = $item;
$delta++;
}
return isset($return) ? $return : NULL;
}
}
class MigrateValueFieldHandler extends MigrateSimpleFieldHandler {
public function __construct() {
parent::__construct(array(
'value_key' => 'value',
'skip_empty' => FALSE,
));
$this->registerTypes(array('value', 'list', 'list_boolean', 'list_integer',
'list_float', 'list_text', 'number_integer', 'number_decimal', 'number_float'));
}
}
class MigrateTaxonomyTermReferenceFieldHandler extends MigrateFieldHandler {
public function __construct() {
$this->registerTypes(array('taxonomy_term_reference'));
}
public function fields($type) {
return array(
'source_type' => t('Option: Set to \'tid\' when the value is a source ID'),
'create_term' => t('Option: Set to TRUE to create referenced terms when necessary')
);
}
public function prepare($entity, array $field_info, array $instance, array $values) {
if (isset($values['arguments'])) {
$arguments = $values['arguments'];
unset($values['arguments']);
}
else {
$arguments = array();
}
if (empty($values[0])) {
$values = array();
}
$tids = array();
if (isset($arguments['source_type']) && $arguments['source_type'] == 'tid') {
// Nothing to do. We have tids already.
$tids = $values;
}
elseif ($values) {
// Get the vocabulary for this term
if (isset($field_info['settings']['allowed_values'][0]['vid'])) {
$vid = $field_info['settings']['allowed_values'][0]['vid'];
}
else {
$vocab_name = $field_info['settings']['allowed_values'][0]['vocabulary'];
$names = taxonomy_vocabulary_get_names();
$vid = $names[$vocab_name]->vid;
}
// Cannot use taxonomy_term_load_multiple() since we have an array of names.
// It wants a singular value. This query may return case-insensitive
// matches.
$existing_terms = db_select('taxonomy_term_data', 'td')
->fields('td', array('tid', 'name'))
->condition('td.name', $values, 'IN')
->condition('td.vid', $vid)
->execute()
->fetchAllKeyed(1, 0);
foreach ($values as $value) {
if (isset($existing_terms[$value])) {
$tids[] = $existing_terms[$value];
}
elseif (!empty($arguments['create_term'])) {
$new_term = new stdClass();
$new_term->vid = $vid;
$new_term->name = $value;
taxonomy_term_save($new_term);
$tids[] = $new_term->tid;
}
}
}
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
$result = array();
$delta = 0;
foreach ($tids as $tid) {
if (is_array($language)) {
$current_language = $language[$delta];
}
else {
$current_language = $language;
}
$result[$current_language][] = array('tid' => $tid);
$delta++;
}
return $result;
}
}
/**
* The next generation of file field handler. This class focuses on the file
* field itself, and offloads understanding of obtaining the actual file and
* dealing with the file entity to an embedded MigrateFileInterface instance.
*/
abstract class MigrateFileFieldBaseHandler extends MigrateFieldHandler {
/**
* Implementation of MigrateFieldHandler::fields().
*
* @param $type
* The file field type - 'file', 'image', etc.
* @param $parent_field
* Name of the parent field.
* @param Migration $migration
* The migration context for the parent field. We can look at the mappings
* and determine which subfields are relevant.
* @return array
*/
public function fields($type, $parent_field, $migration = NULL) {
$fields = array(
'file_class' => t('Option: <a href="@doc">Implementation of MigrateFile to use</a>',
array('@doc' => 'http://drupal.org/node/1540106#file_class')),
'language' => t('Subfield: Language for the field'),
);
// If we can identify the file class mapped to this field, pick up the
// subfields specific to that class.
if ($migration) {
$field_mappings = $migration->getFieldMappings();
$class_mapping = $parent_field . ':file_class';
if (isset($field_mappings[$class_mapping])) {
$mapping = $field_mappings[$class_mapping];
$file_class = $mapping->getDefaultValue();
}
}
if (!isset($file_class)) {
$file_class = 'MigrateFileUri';
}
$fields += call_user_func(array($file_class, 'fields'));
return $fields;
}
/**
* Implementation of MigrateFieldHandler::prepare().
*
* Prepare file data for saving as a Field API file field.
*
* @return array
* Field API array suitable for inserting in the destination object.
*/
public function prepare($entity, array $field_info, array $instance, array $values) {
if (isset($values['arguments'])) {
$arguments = $values['arguments'];
unset($values['arguments']);
}
else {
$arguments = array();
}
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
$migration = Migration::currentMigration();
// One can override the source class via CLI or drushrc.php (the
// option is named file_function for historical reasons)
if ($migration->getOption('file_function')) {
$file_class = $migration->getOption('file_function');
}
elseif (!empty($arguments['file_class'])) {
$file_class = $arguments['file_class'];
}
else {
$file_class = 'MigrateFileUri';
}
// If a destination directory (relative to the Drupal public files directory)
// is not explicitly provided, use the default for the field.
if (empty($arguments['destination_dir'])) {
$arguments['destination_dir'] = $this->destinationDir($field_info, $instance);
}
$return = array();
$delta = 0;
// Note that what $value represents depends on the file class -
// MigrateFileUri expects a filespec/URI, MigrateFileFid expects a file ID,
// etc.
foreach ($values as $value) {
if ($value) {
// If the parent entity doesn't have an explicit uid, give ownership
// to the anonymous account
$owner = isset($entity->uid) ? $entity->uid : 0;
// Call the MigrateFileInterface implementation to do the real work
$source = new $file_class($arguments);
$file = $source->processFile($value, $owner);
// Assuming we got back a valid file ID, build the proper field
// array out of it. We assume that if we did not get back a fid, the
// MigrateFile class has saved a message indicating why.
if ($file) {
$field_array = array('fid' => $file->fid);
$return[$language][] = $this->buildFieldArray($field_array, $arguments, $delta);
}
}
$delta++;
}
return $return;
}
/**
* Determine where the migrated file should go.
*
* @param $field_info
* Field API info on the general field.
* @param $instance
* Field API info on the field instance for this entity type.
* @return string
* Directory relative to the Drupal public files directory.
*/
protected function destinationDir($field_info, $instance) {
$destination_dir = file_field_widget_uri($field_info, $instance);
return $destination_dir;
}
/**
* Add any type-specific subfields to a file field array.
*
* @param $field_array
* The field array so far (generally will just contain a fid).
* @param $arguments
* Array of arguments passed to the field handler, from which we'll extract
* our own subfields.
* @param $delta
* Index of field values being worked on, for pulling the corresponding
* subfield values if we have an array of them.
*/
abstract protected function buildFieldArray($field_array, $arguments, $delta);
}
/**
* Handle for file fields.
*/
class MigrateFileFieldHandler extends MigrateFileFieldBaseHandler {
public function __construct() {
$this->registerTypes(array('file'));
}
/**
* Implementation of MigrateFieldHandler::fields().
* Note that file and image fields support slightly different field lists.
*
* @param $type
* The file field type - 'file' or 'image'
* @param $parent_field
* Name of the parent field.
* @param Migration $migration
* The migration context for the parent field. We can look at the mappings
* and determine which subfields are relevant.
* @return array
*/
public function fields($type, $parent_field, $migration = NULL) {
$fields = parent::fields($type, $parent_field, $migration);
$fields += array(
'description' => t('Subfield: String to be used as the description value'),
'display' => t('Subfield: String to be used as the display value'),
);
return $fields;
}
/**
* Implementation of MigrateFileFieldBaseHandler::buildFieldArray().
*/
protected function buildFieldArray($field_array, $arguments, $delta) {
if (isset($arguments['description'])) {
if (is_array($arguments['description'])) {
$field_array['description'] = $arguments['description'][$delta];
}
else {
$field_array['description'] = $arguments['description'];
}
}
else {
$field_array['description'] = '';
}
if (isset($arguments['display'])) {
if (is_array($arguments['display'])) {
$field_array['display'] = $arguments['display'][$delta];
}
else {
$field_array['display'] = $arguments['display'];
}
}
else {
$field_array['display'] = 1;
}
return $field_array;
}
}
/**
* Handle for image fields;
*/
class MigrateImageFieldHandler extends MigrateFileFieldBaseHandler {
public function __construct() {
$this->registerTypes(array('image'));
}
/**
* Implementation of MigrateFieldHandler::fields().
* Note that file and image fields support slightly different field lists.
*
* @param $type
* The file field type - 'file' or 'image'
* @param $parent_field
* Name of the parent field.
* @param Migration $migration
* The migration context for the parent field. We can look at the mappings
* and determine which subfields are relevant.
* @return array
*/
public function fields($type, $parent_field, $migration = NULL) {
$fields = parent::fields($type, $parent_field, $migration);
$fields += array(
'alt' => t('Subfield: String to be used as the alt value'),
'title' => t('Subfield: String to be used as the title value'),
);
return $fields;
}
/**
* Implementation of MigrateFileFieldBaseHandler::buildFieldArray().
*/
protected function buildFieldArray($field_array, $arguments, $delta) {
if (isset($arguments['alt'])) {
if (is_array($arguments['alt'])) {
$field_array['alt'] = $arguments['alt'][$delta];
}
else {
$field_array['alt'] = $arguments['alt'];
}
}
if (isset($arguments['title'])) {
if (is_array($arguments['title'])) {
$field_array['title'] = $arguments['title'][$delta];
}
else {
$field_array['title'] = $arguments['title'];
}
}
return $field_array;
}
}
class MigrateNodeReferenceFieldHandler extends MigrateSimpleFieldHandler {
public function __construct() {
parent::__construct(array(
'value_key' => 'nid',
'skip_empty' => TRUE,
));
$this->registerTypes(array('node_reference'));
}
}
class MigrateUserReferenceFieldHandler extends MigrateSimpleFieldHandler {
public function __construct() {
parent::__construct(array(
'value_key' => 'uid',
'skip_empty' => TRUE,
));
$this->registerTypes(array('user_reference'));
}
}

588
sites/all/modules/migrate/plugins/destinations/file.inc Normal file
View File

@@ -0,0 +1,588 @@
<?php
/**
* @file
* Support for file entity as destination. Note that File Fields have their
* own destination in fields.inc
*/
/**
* Interface for taking some value representing a file and returning
* a Drupal file entity (creating the entity if necessary).
*/
interface MigrateFileInterface {
/**
* Return a list of subfields and options specific to this implementation,
* keyed by name.
*/
public static function fields();
/**
* Create or link to a Drupal file entity.
*
* @param $value
* A class-specific value (URI, pre-existing file ID, file blob, ...)
* representing file content.
*
* @param $owner
* uid of an account to be recorded as the file owner.
*
* @return object
* File entity being created or referenced.
*/
public function processFile($value, $owner);
}
/**
* Handle the degenerate case where we already have a file ID.
*/
class MigrateFileFid implements MigrateFileInterface {
/**
* Implementation of MigrateFileInterface::fields().
*
* @return array
*/
static public function fields() {
return array();
}
/**
* Implementation of MigrateFileInterface::processFile().
*
* @param $value
* An existing file entity ID (fid).
* @param $owner
* User ID (uid) to be the owner of the file. Ignored in this case.
* @return int
* The file entity corresponding to the fid that was passed in.
*/
public function processFile($value, $owner) {
return file_load($value);
}
}
/**
* Base class for creating core file entities.
*/
abstract class MigrateFile implements MigrateFileInterface {
/**
* Extension of the core FILE_EXISTS_* constants, offering an alternative to
* reuse the existing file if present as-is (core only offers the options of
* replacing it or renaming to avoid collision).
*/
const FILE_EXISTS_REUSE = -1;
/**
* The destination directory within Drupal.
*
* @var string
*/
protected $destinationDir = 'public://';
/**
* The filename relative to destinationDir to which to save the current file.
*
* @var string
*/
protected $destinationFile = '';
/**
* How to handle destination filename collisions.
*
* @var int
*/
protected $fileReplace = FILE_EXISTS_RENAME;
/**
* Set to TRUE to prevent file deletion on rollback.
*
* @var bool
*/
protected $preserveFiles = FALSE;
/**
* An optional file object to use as a default starting point for building the
* file entity.
*
* @var stdClass
*/
protected $defaultFile;
public function __construct($arguments = array(), $default_file = NULL) {
if (isset($arguments['destination_dir'])) {
$this->destinationDir = $arguments['destination_dir'];
}
if (isset($arguments['destination_file'])) {
$this->destinationFile = $arguments['destination_file'];
}
if (isset($arguments['file_replace'])) {
$this->fileReplace = $arguments['file_replace'];
}
if (isset($arguments['preserve_files'])) {
$this->preserveFiles = $arguments['preserve_files'];
}
if ($default_file) {
$this->defaultFile = $default_file;
}
else {
$this->defaultFile = new stdClass;
}
}
/**
* Implementation of MigrateFileInterface::fields().
*
* @return array
*/
static public function fields() {
return array(
'destination_dir' => t('Subfield: <a href="@doc">Path within Drupal files directory to store file</a>',
array('@doc' => 'http://drupal.org/node/1540106#destination_dir')),
'destination_file' => t('Subfield: <a href="@doc">Path within destination_dir to store the file.</a>',
array('@doc' => 'http://drupal.org/node/1540106#destination_file')),
'file_replace' => t('Option: <a href="@doc">Value of $replace in that file function. Does not apply to file_fast(). Defaults to FILE_EXISTS_RENAME.</a>',
array('@doc' => 'http://drupal.org/node/1540106#file_replace')),
'preserve_files' => t('Option: <a href="@doc">Boolean indicating whether files should be preserved or deleted on rollback</a>',
array('@doc' => 'http://drupal.org/node/1540106#preserve_files')),
);
}
/**
* Setup a file entity object suitable for saving.
*
* @param $destination
* Path to the Drupal copy of the file.
* @param $owner
* Uid of the file owner.
* @return stdClass
* A file object ready to be saved.
*/
protected function createFileEntity($destination, $owner) {
$file = clone $this->defaultFile;
$file->uri = $destination;
$file->uid = $owner;
if (!isset($file->filename)) {
$file->filename = drupal_basename($destination);
}
if (!isset($file->filemime)) {
$file->filemime = file_get_mimetype($destination);
}
if (!isset($file->status)) {
$file->status = FILE_STATUS_PERMANENT;
}
// If we are replacing an existing file re-use its database record.
if ($this->fileReplace == FILE_EXISTS_REPLACE) {
$existing_files = file_load_multiple(array(), array('uri' => $destination));
if (count($existing_files)) {
$existing = reset($existing_files);
$file->fid = $existing->fid;
$file->filename = $existing->filename;
}
}
return $file;
}
/**
* By whatever appropriate means, put the file in the right place.
*
* @param $destination
* Destination path within Drupal.
* @return bool
* TRUE if the file is successfully saved, FALSE otherwise.
*/
abstract protected function copyFile($destination);
/**
* Default implementation of MigrateFileInterface::processFiles().
*
* @param $value
* The URI or local filespec of a file to be imported.
* @param $owner
* User ID (uid) to be the owner of the file.
* @return object
* The file entity being created or referenced.
*/
public function processFile($value, $owner) {
$migration = Migration::currentMigration();
// Determine the final path we want in Drupal - start with our preferred path.
$destination = file_stream_wrapper_uri_normalize(
$this->destinationDir . '/' .
ltrim($this->destinationFile, "/\\"));
// Our own file_replace behavior - if the file exists, use it without
// replacing it
if ($this->fileReplace == self::FILE_EXISTS_REUSE) {
// See if we this file already (we'll reuse a file entity if it exists).
if (file_exists($destination)) {
$file = $this->createFileEntity($destination, $owner);
// File entity didn't already exist, create it
if (empty($file->fid)) {
$file = file_save($file);
}
return $file;
}
// No existing one to reuse, reset to REPLACE
$this->fileReplace = FILE_EXISTS_REPLACE;
}
// Prepare the destination directory.
if (!file_prepare_directory(drupal_dirname($destination),
FILE_CREATE_DIRECTORY | FILE_MODIFY_PERMISSIONS)) {
$migration->saveMessage(t('Could not create destination directory for !dest',
array('!dest' => $destination)));
return FALSE;
}
// Determine whether we can perform this operation based on overwrite rules.
$destination = file_destination($destination, $this->fileReplace);
if ($destination === FALSE) {
$migration->saveMessage(t('The file could not be copied because ' .
'file %dest already exists in the destination directory.',
array('%dest' => $destination)));
return FALSE;
}
// Make sure the .htaccess files are present.
file_ensure_htaccess();
// Put the file where it needs to be.
if (!$this->copyFile($destination)) {
return FALSE;
}
// Set the permissions on the new file.
drupal_chmod($destination);
// Create and save the file entity.
$file = file_save($this->createFileEntity($destination, $owner));
// Prevent deletion of the file on rollback if requested.
if (is_object($file)) {
if (!empty($this->preserveFiles)) {
// We do this directly instead of calling file_usage_add, to force the
// count to 1 - otherwise, updates will increment the counter and the file
// will never be deletable
db_merge('file_usage')
->key(array(
'fid' => $file->fid,
'module' => 'migrate',
'type' => 'file',
'id' => $file->fid,
))
->fields(array('count' => 1))
->execute();
}
return $file;
}
else {
return FALSE;
}
}
}
/**
* Handle cases where we're handed a URI, or local filespec, representing a file
* to be imported to Drupal.
*/
class MigrateFileUri extends MigrateFile {
/**
* The source directory for the file, relative to which the value (source
* file) will be taken.
*
* @var string
*/
protected $sourceDir = '';
/**
* The full path to the source file.
*
* @var string
*/
protected $sourcePath = '';
public function __construct($arguments = array(), $default_file = NULL) {
parent::__construct($arguments, $default_file);
if (isset($arguments['source_dir'])) {
$this->sourceDir = rtrim($arguments['source_dir'], "/\\");
}
}
/**
* Implementation of MigrateFileInterface::fields().
*
* @return array
*/
static public function fields() {
return parent::fields() +
array(
'source_dir' => t('Subfield: <a href="@doc">Path to source file.</a>',
array('@doc' => 'http://drupal.org/node/1540106#source_dir')),
);
}
/**
* Implementation of MigrateFile::copyFile().
*
* @param $destination
* Destination within Drupal.
*
* @return bool
* TRUE if the copy succeeded, FALSE otherwise.
*/
protected function copyFile($destination) {
// Perform the copy operation.
if (!@copy($this->sourcePath, $destination)) {
throw new MigrateException(t('The specified file %file could not be copied to ' .
'%destination.',
array('%file' => $this->sourcePath, '%destination' => $destination)));
}
else {
return TRUE;
}
}
/**
* Implementation of MigrateFileInterface::processFiles().
*
* @param $value
* The URI or local filespec of a file to be imported.
* @param $owner
* User ID (uid) to be the owner of the file.
* @return object
* The file entity being created or referenced.
*/
public function processFile($value, $owner) {
// Identify the full path to the source file
if (!empty($this->sourceDir)) {
$this->sourcePath = rtrim($this->sourceDir, "/\\") . '/' . ltrim($value, "/\\");
}
else {
$this->sourcePath = $value;
}
if (empty($this->destinationFile)) {
$this->destinationFile = basename($this->sourcePath);
}
// MigrateFile has most of the smarts - the key is that it will call back
// to our copyFile() implementation.
$file = parent::processFile($value, $owner);
return $file;
}
}
/**
* Handle cases where we're handed a blob (i.e., the actual contents of a file,
* such as image data) to be stored as a real file in Drupal.
*/
class MigrateFileBlob extends MigrateFile {
/**
* The file contents we will be writing to a real file.
*
* @var
*/
protected $fileContents;
/**
* Implementation of MigrateFile::copyFile().
*
* @param $destination
* Drupal destination path.
* @return bool
* TRUE if the file contents were successfully written, FALSE otherwise.
*/
protected function copyFile($destination) {
if (file_put_contents($destination, $this->fileContents)) {
return TRUE;
}
else {
$migration = Migration::currentMigration();
$migration->saveMessage(t('Failed to write blob data to %destination',
array('%destination' => $destination)));
return FALSE;
}
}
/**
* Implementation of MigrateFileInterface::processFile().
*
* @param $value
* The file contents to be saved as a file.
* @param $owner
* User ID (uid) to be the owner of the file.
* @return object
* File entity being created or referenced.
*/
public function processFile($value, $owner) {
$this->fileContents = $value;
$file = parent::processFile($value, $owner);
return $file;
}
}
/**
* Destination class implementing migration into the files table.
*/
class MigrateDestinationFile extends MigrateDestinationEntity {
/**
* File class (MigrateFileUri etc.) doing the dirty wrk.
*
* @var string
*/
protected $fileClass;
/**
* Implementation of MigrateDestination::getKeySchema().
*
* @return array
*/
static public function getKeySchema() {
return array(
'fid' => array(
'type' => 'int',
'unsigned' => TRUE,
'description' => 'file_managed ID',
),
);
}
/**
* Basic initialization
*
* @param array $options
* Options applied to files.
*/
public function __construct($bundle = 'file', $file_class = 'MigrateFileUri',
$options = array()) {
parent::__construct('file', $bundle, $options);
$this->fileClass = $file_class;
}
/**
* Returns a list of fields available to be mapped for the entity type (bundle)
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array();
// First the core properties
$fields['fid'] = t('File: Existing file ID');
$fields['uid'] = t('File: Uid of user associated with file');
$fields['value'] = t('File: Representation of the source file (usually a URI)');
$fields['timestamp'] = t('File: UNIX timestamp for the date the file was added');
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
$fields += migrate_handler_invoke_all('File', 'fields', $this->entityType, $this->bundle, $migration);
// Plus anything provided by the file class
$fields += call_user_func(array($this->fileClass, 'fields'));
return $fields;
}
/**
* Delete a file entry.
*
* @param array $fid
* Fid to delete, arrayed.
*/
public function rollback(array $fid) {
migrate_instrument_start('file_load');
$file = file_load(reset($fid));
migrate_instrument_stop('file_load');
if ($file) {
// If we're not preserving the file, make sure we do the job completely.
migrate_instrument_start('file_delete');
file_delete($file, TRUE);
migrate_instrument_stop('file_delete');
}
}
/**
* Import a single file record.
*
* @param $file
* File object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields (fid only in this case) of the file that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $file, stdClass $row) {
// Updating previously-migrated content?
$migration = Migration::currentMigration();
if (isset($row->migrate_map_destid1)) {
if (isset($file->fid)) {
if ($file->fid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming fid !fid and map destination fid !destid1 don't match",
array('!fid' => $file->fid, '!destid1' => $row->migrate_map_destid1)));
}
}
else {
$file->fid = $row->migrate_map_destid1;
}
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($file->fid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination fid provided'));
}
$old_file = file_load($file->fid);
}
// Invoke migration prepare handlers
$this->prepare($file, $row);
if (isset($file->fid)) {
$updating = TRUE;
}
else {
$updating = FALSE;
}
if (!isset($file->uid)) {
$file->uid = 1;
}
// file_save() unconditionally sets timestamp - if we have an explicit
// value we want, we need to set it manually after file_save.
if (isset($file->timestamp)) {
$timestamp = MigrationBase::timestamp($file->timestamp);
}
$file_class = $this->fileClass;
$source = new $file_class((array)$file, $file);
$file = $source->processFile($file->value, $file->uid);
if (is_object($file) && isset($file->fid)) {
$this->complete($file, $row);
if (isset($timestamp)) {
db_update('file_managed')
->fields(array('timestamp' => $timestamp))
->condition('fid', $file->fid)
->execute();
$file->timestamp = $timestamp;
}
$return = array($file->fid);
if ($updating) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
}
else {
$return = FALSE;
}
return $return;
}
}

188
sites/all/modules/migrate/plugins/destinations/menu.inc Normal file
View File

@@ -0,0 +1,188 @@
<?php
/**
* @file
* Support for menu destinations.
*/
/**
* Destination class implementing migration into {menu_custom}.
*/
class MigrateDestinationMenu extends MigrateDestination {
static public function getKeySchema() {
return array(
'menu_name' => array(
'type' => 'varchar',
'length' => 32,
'not null' => TRUE,
'default' => '',
'description' => 'Primary Key: Unique key for menu. This is used as a block delta so length is 32.',
),
);
}
public function __construct() {
parent::__construct();
}
public function __toString() {
$output = t('Menu');
return $output;
}
/**
* Returns a list of fields available to be mapped for menus.
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array(
'menu_name' => t('The menu name. Primary key.'),
'title' => t('The human-readable name of the menu.'),
'description' => t('A description of the menu'),
);
return $fields;
}
/**
* Import a single row.
*
* @param $menu
* Menu object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $menu, stdClass $row) {
// Invoke migration prepare handlers
$this->prepare($menu, $row);
// Menus are handled as arrays, so clone the object to an array.
$menu = clone $menu;
$menu = (array) $menu;
// Check to see if this is a new menu.
$update = FALSE;
if ($data = menu_load($menu['menu_name'])) {
$update = TRUE;
}
// menu_save() provides no return callback, so we can't really test this
// without running a menu_load() check.
migrate_instrument_start('menu_save');
menu_save($menu);
migrate_instrument_stop('menu_save');
// Return the new id or FALSE on failure.
if ($data = menu_load($menu['menu_name'])) {
// Increment the count if the save succeeded.
if ($update) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
// Return the primary key to the mapping table.
$return = array($data['menu_name']);
}
else {
$return = FALSE;
}
// Invoke migration complete handlers.
$menu = (object) $data;
$this->complete($menu, $row);
return $return;
}
/**
* Implementation of MigrateDestination::prepare().
*/
public function prepare($menu, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$menu->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('menu', 'prepare', $menu, $row);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($menu, $row);
}
}
public function complete($menu, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$menu->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('menu', 'complete', $menu, $row);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'complete')) {
$migration->complete($menu, $row);
}
}
/**
* Delete a single menu.
*
* @param $id
* Array of fields representing the key (in this case, just menu_name).
*/
public function rollback(array $id) {
$menu_name = reset($id);
migrate_instrument_start('menu_delete');
$this->prepareRollback($menu_name);
if ($menu = menu_load($menu_name)) {
menu_delete($menu);
}
$this->completeRollback($menu_name);
migrate_instrument_stop('menu_delete');
}
/**
* Give handlers a shot at cleaning up before a menu has been rolled back.
*
* @param $menu_name
* ID of the menu about to be deleted.
*/
public function prepareRollback($menu_name) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('menu', 'prepareRollback', $menu_name);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($menu_name);
}
}
/**
* Give handlers a shot at cleaning up after a menu has been rolled back.
*
* @param $menu_name
* ID of the menu which has been deleted.
*/
public function completeRollback($menu_name) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('menu', 'completeRollback', $menu_name);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($menu_name);
}
}
}

235
sites/all/modules/migrate/plugins/destinations/menu_links.inc Normal file
View File

@@ -0,0 +1,235 @@
<?php
/**
* @file
* Support for menu link destinations.
*/
/**
* Destination class implementing migration into {menu_links}.
*/
class MigrateDestinationMenuLinks extends MigrateDestination {
static public function getKeySchema() {
return array(
'mlid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'ID of destination link',
),
);
}
public function __construct() {
parent::__construct();
}
public function __toString() {
$output = t('Menu links');
return $output;
}
/**
* Returns a list of fields available to be mapped for menu links.
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array(
'menu_name' => t('The menu name. All links with the same menu name (such as \'navigation\') are part of the same menu.'),
'mlid' => t('The menu link ID (mlid) is the integer primary key.'),
'plid' => t('The parent link ID (plid) is the mlid of the link above in the hierarchy, or zero if the link is at the top level in its menu.'),
'link_path' => t('The Drupal path or external path this link points to.'),
'router_path' => t('For links corresponding to a Drupal path (external = 0), this connects the link to a {menu_router}.path for joins.'),
'link_title' => t('The text displayed for the link, which may be modified by a title callback stored in {menu_router}.'),
'options' => t('A serialized array of options to be passed to the url() or l() function, such as a query string or HTML attributes.'),
'module' => t('The name of the module that generated this link.'),
'hidden' => t('A flag for whether the link should be rendered in menus. (1 = a disabled menu item that may be shown on admin screens, -1 = a menu callback, 0 = a normal, visible link)'),
'external' => t('A flag to indicate if the link points to a full URL starting with a protocol, like http:// (1 = external, 0 = internal).'),
'has_children' => t('Flag indicating whether any links have this link as a parent (1 = children exist, 0 = no children).'),
'expanded' => t('Flag for whether this link should be rendered as expanded in menus - expanded links always have their child links displayed, instead of only when the link is in the active trail (1 = expanded, 0 = not expanded)'),
'weight' => t('Link weight among links in the same menu at the same depth.'),
'depth' => t('The depth relative to the top level. A link with plid == 0 will have depth == 1.'),
'customized' => t('A flag to indicate that the user has manually created or edited the link (1 = customized, 0 = not customized).'),
'p1' => t('The first mlid in the materialized path. If N = depth, then pN must equal the mlid. If depth > 1 then p(N-1) must equal the plid. All pX where X > depth must equal zero. The columns p1 .. p9 are also called the parents.'),
'p2' => t('The second mlid in the materialized path. See p1.'),
'p3' => t('The third mlid in the materialized path. See p1.'),
'p4' => t('The fourth mlid in the materialized path. See p1.'),
'p5' => t('The fifth mlid in the materialized path. See p1.'),
'p6' => t('The sixth mlid in the materialized path. See p1.'),
'p7' => t('The seventh mlid in the materialized path. See p1.'),
'p8' => t('The eighth mlid in the materialized path. See p1.'),
'p9' => t('The ninth mlid in the materialized path. See p1.'),
'updated' => t('Flag that indicates that this link was generated during the update from Drupal 5.'),
);
return $fields;
}
/**
* Import a single row.
*
* @param $menu_link
* Menu link object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $menu_link, stdClass $row) {
// Updating previously-migrated content
if (isset($row->migrate_map_destid1)) {
$menu_link->mlid = $row->migrate_map_destid1;
}
// Invoke migration prepare handlers
// @todo derive existing mlids?
$this->prepare($menu_link, $row);
// Menu links are handled as arrays, so clone the object to an array.
$item = clone $menu_link;
$item = (array) $item;
migrate_instrument_start('menu_link_save');
// Check to see if this is a new menu item.
$update = FALSE;
if (isset($item['mlid'])) {
$update = TRUE;
$mlid = menu_link_save($item);
}
else {
// menu_link_save() should return an mlid integer.
$mlid = menu_link_save($item);
}
migrate_instrument_stop('menu_link_save');
// Return the new id or FALSE on failure.
if (!empty($mlid)) {
// Increment the count if the save succeeded.
if ($update) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
// Return the primary key to the mapping table.
$return = array($mlid);
}
else {
$return = FALSE;
}
// Invoke migration complete handlers.
$menu_link = (object) menu_link_load($mlid);
$this->complete($menu_link, $row);
return $return;
}
/**
* Implementation of MigrateDestination::prepare().
*/
public function prepare($menu_link, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$menu_link->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('menu_links', 'prepare', $menu_link, $row);
// Then call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($menu_link, $row);
}
}
/**
* Implementation of MigrateDestination::complete().
*/
public function complete($menu_link, stdClass $row) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
$menu_link->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any general handlers.
migrate_handler_invoke_all('menu_links', 'complete', $menu_link, $row);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'complete')) {
$migration->complete($menu_link, $row);
}
}
/**
* Implementation of MigrateDestination::postImport().
*/
public function postImport() {
// Clear the cache after all menu links are imported.
menu_cache_clear_all();
}
/**
* Delete a single menu item.
*
* @param $id
* Array of fields representing the key (in this case, just mlid).
*/
public function rollback($id) {
$mlid = reset($id);
migrate_instrument_start('menu_link_delete');
$this->prepareRollback($mlid);
// @todo: any error checking here? For example, menu.inc has:
// if ($menu = menu_load($menu_name)) { menu_delete($menu) }
menu_link_delete($mlid);
$this->completeRollback($mlid);
migrate_instrument_stop('menu_link_delete');
}
/**
* Give handlers a shot at cleaning up before a menu has been rolled back.
*
* @param $mlid
* ID of the menu link about to be deleted.
*/
public function prepareRollback($mlid) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('menu_links', 'prepareRollback', $mlid);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'prepareRollback')) {
$migration->prepareRollback($mlid);
}
}
/**
* Give handlers a shot at cleaning up after a menu has been rolled back.
*
* @param $mlid
* ID of the menu link which has been deleted.
*/
public function completeRollback($mlid) {
// We do nothing here but allow child classes to act.
$migration = Migration::currentMigration();
// Call any general handlers.
migrate_handler_invoke_all('menu_links', 'completeRollback', $mlid);
// Then call any complete handler for this specific Migration.
if (method_exists($migration, 'completeRollback')) {
$migration->completeRollback($mlid);
}
}
/**
* Implementation of MigrateDestination::postRollback().
*/
public function postRollback() {
// Clear the cache after all menu links are rolled back.
menu_cache_clear_all();
}
}

296
sites/all/modules/migrate/plugins/destinations/node.inc Normal file
View File

@@ -0,0 +1,296 @@
<?php
/**
* @file
* Support for node destinations.
*/
// TODO:
// Make sure this works with updates, explicit destination keys
/**
* Destination class implementing migration into nodes.
*/
class MigrateDestinationNode extends MigrateDestinationEntity {
static public function getKeySchema() {
return array(
'nid' => array(
'type' => 'int',
'unsigned' => TRUE,
'description' => 'ID of destination node',
),
);
}
/**
* Return an options array for node destinations.
*
* @param string $language
* Default language for nodes created via this destination class.
* @param string $text_format
* Default text format for nodes created via this destination class.
*/
static public function options($language, $text_format) {
return compact('language', 'text_format');
}
/**
* Basic initialization
*
* @param string $bundle
* A.k.a. the content type (page, article, etc.) of the node.
* @param array $options
* Options applied to nodes.
*/
public function __construct($bundle, array $options = array()) {
parent::__construct('node', $bundle, $options);
}
/**
* Returns a list of fields available to be mapped for the node type (bundle)
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array();
// First the core (node table) properties
$fields['nid'] = t('Node: <a href="@doc">Existing node ID</a>',
array('@doc' => 'http://drupal.org/node/1349696#nid'));
$node_type = node_type_load($this->bundle);
if ($node_type->has_title) {
$fields['title'] = t('Node: <a href="@doc">',
array('@doc' => 'http://drupal.org/node/1349696#title'))
. $node_type->title_label . '</a>';
}
$fields['uid'] = t('Node: <a href="@doc">Authored by (uid)</a>',
array('@doc' => 'http://drupal.org/node/1349696#uid'));
$fields['created'] = t('Node: <a href="@doc">Created timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349696#created'));
$fields['changed'] = t('Node: <a href="@doc">Modified timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349696#changed'));
$fields['status'] = t('Node: <a href="@doc">Published</a>',
array('@doc' => 'http://drupal.org/node/1349696#status'));
$fields['promote'] = t('Node: <a href="@doc">Promoted to front page</a>',
array('@doc' => 'http://drupal.org/node/1349696#promote'));
$fields['sticky'] = t('Node: <a href="@doc">Sticky at top of lists</a>',
array('@doc' => 'http://drupal.org/node/1349696#sticky'));
$fields['revision'] = t('Node: <a href="@doc">Create new revision</a>',
array('@doc' => 'http://drupal.org/node/1349696#revision'));
$fields['log'] = t('Node: <a href="@doc">Revision Log message</a>',
array('@doc' => 'http://drupal.org/node/1349696#log'));
$fields['language'] = t('Node: <a href="@doc">Language (fr, en, ...)</a>',
array('@doc' => 'http://drupal.org/node/1349696#language'));
$fields['tnid'] = t('Node: <a href="@doc">The translation set id for this node</a>',
array('@doc' => 'http://drupal.org/node/1349696#tnid'));
$fields['revision_uid'] = t('Node: <a href="@doc">Modified (uid)</a>',
array('@doc' => 'http://drupal.org/node/1349696#revision_uid'));
$fields['is_new'] = t('Option: <a href="@doc">Indicates a new node with the specified nid should be created</a>',
array('@doc' => 'http://drupal.org/node/1349696#is_new'));
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
$fields += migrate_handler_invoke_all('Node', 'fields', $this->entityType, $this->bundle, $migration);
return $fields;
}
/**
* Delete a batch of nodes at once.
*
* @param $nids
* Array of node IDs to be deleted.
*/
public function bulkRollback(array $nids) {
migrate_instrument_start('node_delete_multiple');
$this->prepareRollback($nids);
node_delete_multiple($nids);
$this->completeRollback($nids);
migrate_instrument_stop('node_delete_multiple');
}
/**
* Import a single node.
*
* @param $node
* Node object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields (nid only in this case) of the node that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $node, stdClass $row) {
// Updating previously-migrated content?
$migration = Migration::currentMigration();
if (isset($row->migrate_map_destid1)) {
// Make sure is_new is off
$node->is_new = FALSE;
if (isset($node->nid)) {
if ($node->nid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming nid !nid and map destination nid !destid1 don't match",
array('!nid' => $node->nid, '!destid1' => $row->migrate_map_destid1)));
}
}
else {
$node->nid = $row->migrate_map_destid1;
}
// Get the existing vid, tnid so updates don't generate notices
$values = db_select('node', 'n')
->fields('n', array('vid', 'tnid'))
->condition('nid', $node->nid)
->execute()
->fetchAssoc();
if (empty($values)) {
throw new MigrateException(t("Incoming node ID !nid no longer exists",
array('!nid' => $node->nid)));
}
$node->vid = $values['vid'];
if (empty($row->tnid)) {
$node->tnid = $values['tnid'];
}
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($node->nid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination nid provided'));
}
$old_node = node_load($node->nid);
if (empty($old_node)) {
throw new MigrateException(t('System-of-record is DESTINATION, but node !nid does not exist',
array('!nid' => $node->nid)));
}
if (!isset($node->created)) {
$node->created = $old_node->created;
}
if (!isset($node->vid)) {
$node->vid = $old_node->vid;
}
if (!isset($node->status)) {
$node->status = $old_node->status;
}
if (!isset($node->uid)) {
$node->uid = $old_node->uid;
}
}
elseif (!isset($node->type)) {
// Default the type to our designated destination bundle (by doing this
// conditionally, we permit some flexibility in terms of implementing
// migrations which can affect more than one type).
$node->type = $this->bundle;
}
// Set some required properties.
if ($migration->getSystemOfRecord() == Migration::SOURCE) {
if (!isset($node->language)) {
$node->language = $this->language;
}
// Apply defaults, allow standard node prepare hooks to fire.
// node_object_prepare() will blow these away, so save them here and
// stuff them in later if need be.
if (isset($node->created)) {
$created = MigrationBase::timestamp($node->created);
}
else {
// To keep node_object_prepare() from choking
$node->created = REQUEST_TIME;
}
if (isset($node->changed)) {
$changed = MigrationBase::timestamp($node->changed);
}
if (isset($node->uid)) {
$uid = $node->uid;
}
node_object_prepare($node);
if (isset($created)) {
$node->created = $created;
}
// No point to resetting $node->changed here, node_save() will overwrite it
if (isset($uid)) {
$node->uid = $uid;
}
}
// Invoke migration prepare handlers
$this->prepare($node, $row);
if (!isset($node->revision)) {
$node->revision = 0; // Saves disk space and writes. Can be overridden.
}
// Trying to update an existing node
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
// Incoming data overrides existing data, so only copy non-existent fields
foreach ($old_node as $field => $value) {
// An explicit NULL in the source data means to wipe to old value (i.e.,
// don't copy it over from $old_node)
if (property_exists($node, $field) && $node->$field === NULL) {
// Ignore this field
}
elseif (!isset($node->$field)) {
$node->$field = $old_node->$field;
}
}
}
if (isset($node->nid) && !(isset($node->is_new) && $node->is_new)) {
$updating = TRUE;
}
else {
$updating = FALSE;
}
migrate_instrument_start('node_save');
node_save($node);
migrate_instrument_stop('node_save');
if (isset($node->nid)) {
if ($updating) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
// Unfortunately, http://drupal.org/node/722688 was not accepted, so fix
// the changed timestamp
if (isset($changed)) {
db_update('node')
->fields(array('changed' => $changed))
->condition('nid', $node->nid)
->execute();
$node->changed = $changed;
}
// Potentially fix uid and timestamp in node_revisions.
$query = db_update('node_revision')
->condition('vid', $node->vid);
if (isset($changed)) {
$fields['timestamp'] = $changed;
}
$revision_uid = isset($node->revision_uid) ? $node->revision_uid : $node->uid;
if ($revision_uid != $GLOBALS['user']->uid) {
$fields['uid'] = $revision_uid;
}
if (!empty($fields)) {
// We actually have something to update.
$query->fields($fields);
$query->execute();
if (isset($changed)) {
$node->timestamp = $changed;
}
}
$return = array($node->nid);
}
else {
$return = FALSE;
}
$this->complete($node, $row);
return $return;
}
}

29
sites/all/modules/migrate/plugins/destinations/path.inc Normal file
View File

@@ -0,0 +1,29 @@
<?php
/**
* @file
* Support for paths in core Drupal objects
*/
class MigratePathEntityHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('entity'));
}
public function fields() {
if (module_exists('path')) {
return array('path' => t('Node: Path alias'));
}
else {
return array();
}
}
public function prepare($entity, stdClass $row) {
if (module_exists('path') && isset($entity->path)) {
$path = $entity->path;
$entity->path = array();
$entity->path['alias'] = $path;
}
}
}

121
sites/all/modules/migrate/plugins/destinations/poll.inc Normal file
View File

@@ -0,0 +1,121 @@
<?php
/**
* @file
* Support for poll nodes.
*
* Each poll node will have multiple choices, and multiple votes. It's
* not practical to bring this information in through your migration's
* source query, you need to pull it separately in prepareRow():
*
* @code
* ...
* $this->addFieldMapping('active')
* ->defaultValue(1);
* $this->addFieldMapping('runtime', 'seconds_to_run')
* $this->addFieldMapping('choice', 'src_choices')
* ->description('src_choices populated in prepareRow()');
* $this->addFieldMapping('votes', 'src_votes')
* ->description('src_votes populated in prepareRow()');
* ...
* public function prepareRow($row);
* $choices = Database::getConnection('default', 'legacy')
* ->select('src_poll_choice', 'c')
* ->fields('c', array('choice_label', 'choice_order', 'choice_total))
* ->condition('c.choiceid', $row->src_contentid);
* ->execute();
* $row->src_choices = array();
* foreach ($choices as $choice) {
* $row->src_choices[] = array(
* 'chtext' => $choice->choice_label,
* 'chvotes' => $choice->choice_total,
* 'weight' => $choice->choice_order,
* );
* }
* // Note that we won't know until much later what the chid is for each
* // choice, so it's best to tie the votes to choices by text.
* $query = Database::getConnection('default', 'legacy')
* ->select('src_poll_vote', 'v')
* ->fields('v', array('choice_uid', 'hostname', 'timestamp))
* ->condition('v.choiceid', $row->src_contentid);
* $votes = $query->innerJoin('src_poll_choice', 'c', 'v.choice_id=c.choice_id')
* ->fields('c', array('choice_label'))
* ->execute();
* $row->src_votes = array();
* foreach ($votes as $vote) {
* $row->src_votes[] = array(
* 'chtext' => $choice->choice_label,
* 'uid' => $choice->choice_uid,
* 'hostname' => $choice->hostname,
* 'timestamp' => $choice->timestamp,
* );
* }
* return TRUE;
* }
* @endcode
*/
class MigratePollEntityHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('node'));
}
public function fields($entity_type, $bundle) {
if ($bundle == 'poll') {
$fields = array(
'active' => t('Poll: Active status'),
'runtime' => t('Poll: How long the poll runs for in seconds'),
'choice' => t('Poll: Choices. Each choice is an array with chtext, chvotes, and weight keys.'),
'votes' => t('Poll: Votes. Each vote is an array with chid (or chtext), uid, hostname, and timestamp keys'),
);
}
else {
$fields = array();
}
return $fields;
}
public function complete($entity, stdClass $row) {
if ($entity->type == 'poll') {
// Update settings overridden by !user_access('administer nodes') check in
// poll_insert().
db_update('poll')
->fields(array('active' => $entity->active))
->condition('nid', $entity->nid)
->execute();
// Update vote summary count, again overridden by
// !user_access('administer nodes') check in poll_insert().
foreach ($row->choice as $choice) {
// Have no mapping tracking for chid, so assume choice text is unique.
db_update('poll_choice')
->fields(array('chvotes' => $choice['chvotes'], 'weight' => $choice['weight']))
->condition('nid', $entity->nid)
->condition('chtext', $choice['chtext'])
->execute();
}
// Insert actual votes.
foreach ($row->votes as $vote) {
$chid = $vote['chid'];
if (!isset($chid)) {
$result = db_select('poll_choice', 'pc')
->fields('pc', array('chid'))
->condition('pc.nid', $entity->nid)
->condition('pc.chtext', $vote['chtext'])
->execute();
$chid = $result->fetchField();
}
db_insert('poll_vote')
->fields(array(
'chid' => $chid,
'nid' => $entity->nid,
'uid' => $vote['uid'],
'hostname' => $vote['hostname'],
'timestamp' => $vote['timestamp'],
))
->execute();
}
}
}
}

42
sites/all/modules/migrate/plugins/destinations/statistics.inc Normal file
View File

@@ -0,0 +1,42 @@
<?php
/**
* @file
* Support for node_counter statistics in core Drupal nodes.
*/
class MigrateStatisticsEntityHandler extends MigrateDestinationHandler {
public function __construct() {
$this->registerTypes(array('node'));
}
public function fields() {
if (module_exists('statistics')) {
$fields = array(
'totalcount' => t('Node: The total number of times the node has been viewed.'),
'daycount' => t('Node: The total number of times the node has been viewed today.'),
'timestamp' => t('Node: The most recent time the node has been viewed.'),
);
}
else {
$fields = array();
}
return $fields;
}
public function complete($node, stdClass $row) {
if (module_exists('statistics') && isset($node->nid)) {
$totalcount = isset($node->totalcount) ? $node->totalcount : 0;
$daycount = isset($node->daycount) ? $node->daycount : 0;
$timestamp = isset($node->timestamp) ? $node->timestamp : 0;
db_merge('node_counter')
->key(array('nid' => $node->nid))
->fields(array(
'totalcount' => $totalcount,
'daycount' => $daycount,
'timestamp' => $timestamp,
))
->execute();
}
}
}

210
sites/all/modules/migrate/plugins/destinations/table.inc Normal file
View File

@@ -0,0 +1,210 @@
<?php
/**
* @file
* Support for tables defined through the Schema API.
*/
/**
* Destination class implementing migration into a single table defined through
* the Schema API.
*/
class MigrateDestinationTable extends MigrateDestination {
/**
* The schema of the current table.
*
* @var array
*/
protected $schema = NULL;
/**
* The name of the current table.
*
* @var string
*/
protected $tableName = NULL;
public function __construct($table_name) {
$this->schema = drupal_get_schema($table_name);
$this->tableName = $table_name;
}
static public function getKeySchema($table_name = NULL) {
if (empty($table_name)) {
return array();
}
$schema = drupal_get_schema($table_name);
$keys = array();
foreach ($schema['primary key'] as $primary_key) {
// We can't have any form of serial fields here, since the mapping table
// already has it's own.
$schema['fields'][$primary_key]['auto_increment'] = FALSE;
if ($schema['fields'][$primary_key]['type'] == 'serial') {
$schema['fields'][$primary_key]['type'] = 'int';
}
$keys[$primary_key] = $schema['fields'][$primary_key];
}
return $keys;
}
public function __toString() {
$output = t('Table !name', array('!name' => $this->tableName));
return $output;
}
/**
* Delete a single row.
*
* @param $id
* Primary key values.
*/
public function rollback(array $id) {
migrate_instrument_start('table rollback');
$delete = db_delete($this->tableName);
$keys = array_keys(self::getKeySchema($this->tableName));
$i = 0;
foreach ($id as $value) {
$key = $keys[$i++];
$delete->condition($key, $value);
}
$delete->execute();
migrate_instrument_stop('table rollback');
}
/**
* Import a single row.
*
* @param $entity
* Object object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $entity, stdClass $row) {
if (empty($this->schema['primary key'])) {
throw new MigrateException(t("The destination table has no primary key defined."));
}
// Only filled when doing an update.
$primary_key = array();
$migration = Migration::currentMigration();
// Updating previously-migrated content?
if (isset($row->migrate_map_destid1)) {
$i = 1;
foreach ($this->schema['primary key'] as $key) {
$primary_key[] = $key;
$destination_id = $row->{'migrate_map_destid' . $i};
if (isset($entity->{$key})) {
if ($entity->{$key} != $destination_id) {
throw new MigrateException(t("Incoming id !id and map destination id !destid don't match",
array('!id' => $entity->{$key}, '!destid' => $destination_id)));
}
}
else {
$entity->{$key} = $destination_id;
}
$i++;
}
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
foreach ($this->schema['primary key'] as $key) {
$primary_key[] = $key;
if (!isset($entity->{$key})) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination id provided'));
}
}
$select = db_select($this->tableName)
->fields($this->tableName);
foreach ($this->schema['primary key'] as $key) {
$select->condition($key, $entity->{$key});
}
$old_entity = $select->execute()->fetchObject();
if (empty($old_entity)) {
throw new MigrateException(t('System-of-record is DESTINATION, but the destination entity does not exist'));
}
foreach ($entity as $field => $value) {
$old_entity->$field = $entity->$field;
}
$entity = $old_entity;
}
$this->prepare($entity, $row);
$status = drupal_write_record($this->tableName, $entity, $primary_key);
$this->complete($entity, $row);
if ($status) {
$id = array();
foreach ($this->schema['primary key'] as $key) {
$id[] = $entity->{$key};
}
// Increment the number of updated or inserted records by checking the
// result of drupal_write_record.
($status == SAVED_NEW) ? $this->numCreated++ : $this->numUpdated++;
return $id;
}
}
/**
* Returns a list of fields available to be mapped.
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array();
foreach ($this->schema['fields'] as $column => $schema) {
$fields[$column] = t('Type: !type', array('!type' => $schema['type']));
}
return $fields;
}
/**
* Give handlers a shot at modifying the object before saving it.
*
* @param $entity
* Entity object to build. Prefilled with any fields mapped in the Migration.
* @param $source_row
* Raw source data object - passed through to prepare handlers.
*/
public function prepare($entity, stdClass $source_row) {
$migration = Migration::currentMigration();
$entity->migrate = array(
'machineName' => $migration->getMachineName(),
);
// Call any prepare handler for this specific Migration.
if (method_exists($migration, 'prepare')) {
$migration->prepare($entity, $source_row);
}
}
/**
* Give handlers a shot at modifying the object (or taking additional action)
* after saving it.
*
* @param $object
* Entity object to build. This is the complete object after saving.
* @param $source_row
* Raw source data object - passed through to complete handlers.
*/
public function complete($entity, stdClass $source_row) {
$migration = Migration::currentMigration();
// Call any complete handler for this specific Migration.
if (method_exists($migration, 'complete')) {
$migration->complete($entity, $source_row);
}
}
}

83
sites/all/modules/migrate/plugins/destinations/table_copy.inc Normal file
View File

@@ -0,0 +1,83 @@
<?php
/**
* @file
* Copies data_row into a table using drupal_write_record()
*/
/**
* Destination class implementing migration into a single table.
*/
class MigrateDestinationTableCopy extends MigrateDestination {
public function __construct($tableName, $keySchema) {
parent::__construct();
$this->tableName = $tableName;
$this->keySchema = $keySchema;
}
public function __toString() {
$output = t('Table copy');
return $output;
}
/**
* Delete a batch of rows at once.
*
* @param $ids
* Array of IDs to be deleted.
*/
public function bulkRollback(array $ids) {
migrate_instrument_start('table_copy bulkRollback');
db_delete($this->tableName)
->condition(key($this->keySchema), $ids, 'IN')
->execute();
migrate_instrument_stop('table_copy bulkRollback');
}
/**
* Import a single row.
*
* @param $entity
* Object object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $entity, stdClass $row) {
$migration = MigrationBase::currentMigration();
$fields = clone $row;
$keys = array_keys($this->keySchema);
$values = array();
foreach ($keys as $key) {
$values[] = $row->$key;
}
unset($fields->migrate_map_destid1);
unset($fields->needs_update);
$query = db_merge($this->tableName)->key($keys, $values)->fields((array)$fields);
try {
$status = $query->execute();
if ($status == MergeQuery::STATUS_INSERT) {
$this->numCreated++;
}
else {
$this->numUpdated++;
}
return $values;
}
catch (MigrateException $e) {
$migration->saveMessage($e->getMessage(), $e->getLevel());
Migration::displayMessage($e->getMessage());
}
catch (Exception $e) {
$this->handleException($e);
}
}
public function fields($migration = NULL) {
return array();
}
}

267
sites/all/modules/migrate/plugins/destinations/term.inc Normal file
View File

@@ -0,0 +1,267 @@
<?php
/**
* @file
* Support for taxonomy term destinations.
*/
// TODO:
// Make sure this works with updates, explicit destination keys
// taxonomy_term_save() is doing a cache_clear_all and an automatic insertion for parent.
/**
* Destination class implementing migration into terms.
*/
class MigrateDestinationTerm extends MigrateDestinationEntity {
static public function getKeySchema() {
return array(
'tid' => array(
'type' => 'int',
'unsigned' => TRUE,
'description' => 'ID of destination term',
),
);
}
/**
* Return an options array for term destinations.
*
* @param string $language
* Default language for terms created via this destination class.
* @param string $text_format
* Default text format for terms created via this destination class.
*/
static public function options($language, $text_format) {
return compact('language', 'text_format');
}
/**
* Basic initialization
*
* @param array $options
* Options applied to terms.
*/
public function __construct($bundle, array $options = array()) {
parent::__construct('taxonomy_term', $bundle, $options);
}
/**
* Returns a list of fields available to be mapped for this vocabulary (bundle)
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array();
// First the core (taxonomy_term_data table) properties
$fields['tid'] = t('Term: <a href="@doc">Existing term ID</a>',
array('@doc' => 'http://drupal.org/node/1349702#tid'));
$fields['name'] = t('Term: <a href="@doc">Name</a>',
array('@doc' => 'http://drupal.org/node/1349702#name'));
$fields['description'] = t('Term: <a href="@doc">Description</a>',
array('@doc' => 'http://drupal.org/node/1349702#description'));
$fields['parent'] = t('Term: <a href="@doc">Parent (by Drupal term ID)</a>',
array('@doc' => 'http://drupal.org/node/1349702#parent'));
// TODO: Remove parent_name, implement via arguments
$fields['parent_name'] = t('Term: <a href="@doc">Parent (by name)</a>',
array('@doc' => 'http://drupal.org/node/1349702#parent_name'));
$fields['format'] = t('Term: <a href="@doc">Format</a>',
array('@doc' => 'http://drupal.org/node/1349702#format'));
$fields['weight'] = t('Term: <a href="@doc">Weight</a>',
array('@doc' => 'http://drupal.org/node/1349702#weight'));
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('entity', 'fields', $this->entityType, $this->bundle, $migration);
$fields += migrate_handler_invoke_all('taxonomy_term', 'fields', $this->entityType, $this->bundle, $migration);
return $fields;
}
/**
* Delete a migrated term
*
* @param $ids
* Array of fields representing the key (in this case, just tid).
*/
public function rollback(array $key) {
$tid = reset($key);
/*
* This load() happens soon delete() anyway. We load here in order to
* avoid notices when term has already been deleted. That is easily possible
* considering how deleting a term parent also deletes children in same call.
*/
migrate_instrument_start('taxonomy_term_load');
if (taxonomy_term_load($tid)) {
migrate_instrument_stop('taxonomy_term_load');
migrate_instrument_start('taxonomy_term_delete');
$this->prepareRollback($tid);
$result = (bool) taxonomy_term_delete($tid);
$this->completeRollback($tid);
migrate_instrument_stop('taxonomy_term_delete');
}
else {
migrate_instrument_stop('taxonomy_term_load');
// If it didn't exist, consider this a success
$result = TRUE;
}
return $result;
}
/**
* Import a single term.
*
* @param $term
* Term object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields (tid only in this case) of the term that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $term, stdClass $row) {
$migration = Migration::currentMigration();
// Updating previously-migrated content?
if (isset($row->migrate_map_destid1)) {
$term->tid = $row->migrate_map_destid1;
if (isset($term->tid)) {
if ($term->tid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming tid !tid and map destination nid !destid1 don't match",
array('!tid' => $term->tid, '!destid1' => $row->migrate_map_destid1)));
}
}
else {
$term->tid = $row->migrate_map_destid1;
}
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($term->tid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination tid provided'));
}
$rawterm = $term;
$this->prepare($term, $row);
$old_term = taxonomy_term_load($term->tid);
if (empty($old_term)) {
throw new MigrateException(t('System-of-record is DESTINATION, but term !tid does not exist',
array('!tid' => $term->tid)));
}
foreach ($rawterm as $field => $value) {
$old_term->$field = $term->$field;
}
$term = $old_term;
}
else {
// Default to bundle if no vocabulary machine name provided
if (!isset($term->vocabulary_machine_name)) {
$term->vocabulary_machine_name = $this->bundle;
}
// vid is required
if (empty($term->vid)) {
static $vocab_map = array();
if (!isset($vocab_map[$term->vocabulary_machine_name])) {
// The keys of the returned array are vids
$vocabs = taxonomy_vocabulary_load_multiple(array(),
array('machine_name' => $term->vocabulary_machine_name));
$vids = array_keys($vocabs);
if (isset($vids[0])) {
$vocab_map[$term->vocabulary_machine_name] = $vids[0];
}
else {
$migration->saveMessage(t('No vocabulary found with machine_name !name',
array('!name' => $term->vocabulary_machine_name)));
return FALSE;
}
}
$term->vid = $vocab_map[$term->vocabulary_machine_name];
}
// Look up parent name if provided
if (isset($term->parent_name) && trim($term->parent_name)) {
// Look for the name in the same vocabulary.
// Note that hierarchies may have multiples of the same name...
$terms = taxonomy_term_load_multiple(array(),
array('name' => trim($term->parent_name), 'vid' => $term->vid));
$tids = array_keys($terms);
$term->parent = array($tids[0]);
unset($term->parent_name);
}
if (empty($term->parent)) {
$term->parent = array(0);
}
if (is_array($term->parent) && isset($term->parent['arguments'])) {
// Unset arguments here to avoid duplicate entries in the
// term_hierarchy table.
unset($term->parent['arguments']);
}
if (!isset($term->format)) {
$term->format = $this->textFormat;
}
$this->prepare($term, $row);
// See if the term, with the same parentage, already exists - if so,
// load it
$candidates = taxonomy_term_load_multiple(array(),
array('name' => trim($term->name), 'vid' => $term->vid));
foreach ($candidates as $candidate) {
$parents = taxonomy_get_parents($candidate->tid);
// We need to set up $parents as a simple array of tids
if (empty($parents)) {
$parents = array(0);
}
else {
// Parents array is tid => term object, make into list of tids
$new_parents = array();
foreach ($parents as $parent) {
$new_parents[] = $parent->tid;
}
$parents = $new_parents;
}
if ($term->parent == $parents) {
// We've found a matching term, we'll use that
$term = $candidate;
break;
}
}
}
// Trying to update an existing term
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
$existing_term = taxonomy_term_load($term->tid);
if ($existing_term) {
// Incoming data overrides existing data, so only copy non-existent fields
foreach ($existing_term as $field => $value) {
if (!isset($term->$field)) {
$term->$field = $existing_term->$field;
}
}
}
}
if (isset($term->tid)) {
$updating = TRUE;
}
else {
$updating = FALSE;
}
migrate_instrument_start('taxonomy_term_save');
$status = taxonomy_term_save($term);
migrate_instrument_stop('taxonomy_term_save');
$this->complete($term, $row);
if (isset($term->tid)) {
if ($updating) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
$return = array($term->tid);
}
else {
$return = FALSE;
}
return $return;
}
}

341
sites/all/modules/migrate/plugins/destinations/user.inc Normal file
View File

@@ -0,0 +1,341 @@
<?php
/**
* @file
* Support for user destinations.
*/
// TODO:
// Make sure this works with updates, explicit destination keys
// Speed up password generation a ton: $conf['password_count_log2'] = 1;
/**
* Destination class implementing migration into users.
*/
class MigrateDestinationUser extends MigrateDestinationEntity {
/**
* Indicates whether incoming passwords are md5-encrypted - if so, we will
* rehash them similarly to the D6->D7 upgrade path.
*
* @var boolean
*/
protected $md5Passwords = FALSE;
static public function getKeySchema() {
return array(
'uid' => array(
'type' => 'int',
'unsigned' => TRUE,
'description' => 'ID of destination user',
),
);
}
/**
* Return an options array for user destinations.
*
* @param string $language
* Default language for usrs created via this destination class.
* @param string $text_format
* Default text format for users created via this destination class.
* @param boolean $md5_passwords
* Set TRUE to indicate incoming passwords are md5-encrypted.
*/
static public function options($language, $text_format, $md5_passwords) {
return compact('language', 'text_format', 'md5_passwords');
}
/**
* Basic initialization
*
* @param array $options
* Options applied to comments.
*/
public function __construct(array $options = array()) {
parent::__construct('user', 'user', $options);
if (!empty($options['md5_passwords'])) {
$this->md5Passwords = $options['md5_passwords'];
}
// Reduce hash count so import runs in a reasonable time (use same value as
// the standard Drupal 6=>Drupal 7 upgrade path).
global $conf;
$conf['password_count_log2'] = 11;
}
/**
* Returns a list of fields available to be mapped for users
*
* @param Migration $migration
* Optionally, the migration containing this destination.
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields($migration = NULL) {
$fields = array();
// First the core (users table) properties
$fields['uid'] = t('User: <a href="@doc">Existing user ID</a>',
array('@doc' => 'http://drupal.org/node/1349632#uid'));
$fields['mail'] = t('User: <a href="@doc">Email address</a>',
array('@doc' => 'http://drupal.org/node/1349632#mail'));
$fields['name'] = t('User: <a href="@doc">Username</a>',
array('@doc' => 'http://drupal.org/node/1349632#name'));
$fields['pass'] = t('User: <a href="@doc">Password (plain text)</a>',
array('@doc' => 'http://drupal.org/node/1349632#pass'));
$fields['status'] = t('User: <a href="@doc">Status</a>',
array('@doc' => 'http://drupal.org/node/1349632#status'));
$fields['created'] = t('User: <a href="@doc">Registered timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#created'));
$fields['access'] = t('User: <a href="@doc">Last access timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#access'));
$fields['login'] = t('User: <a href="@doc">Last login timestamp</a>',
array('@doc' => 'http://drupal.org/node/1349632#login'));
$fields['roles'] = t('User: <a href="@doc">Role IDs</a>',
array('@doc' => 'http://drupal.org/node/1349632#roles'));
$fields['role_names'] = t('User: <a href="@doc">Role Names</a>',
array('@doc' => 'http://drupal.org/node/1349632#role_names'));
$fields['picture'] = t('User: <a href="@doc">Picture</a>',
array('@doc' => 'http://drupal.org/node/1349632#picture'));
$fields['signature'] = t('User: <a href="@doc">Signature</a>',
array('@doc' => 'http://drupal.org/node/1349632#signature'));
$fields['signature_format'] = t('User: <a href="@doc">Signature format</a>',
array('@doc' => 'http://drupal.org/node/1349632#signature_format'));
$fields['timezone'] = t('User: <a href="@doc">Timezone</a>',
array('@doc' => 'http://drupal.org/node/1349632#timezone'));
$fields['language'] = t('User: <a href="@doc">Language</a>',
array('@doc' => 'http://drupal.org/node/1349632#language'));
$fields['theme'] = t('User: <a href="@doc">Default theme</a>',
array('@doc' => 'http://drupal.org/node/1349632#theme'));
$fields['init'] = t('User: <a href="@doc">Init</a>',
array('@doc' => 'http://drupal.org/node/1349632#init'));
$fields['is_new'] = t('Option: <a href="@doc">Indicates a new user with the specified uid should be created</a>',
array('@doc' => 'http://drupal.org/node/1349632#is_new'));
// Then add in anything provided by handlers
$fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle, $migration);
$fields += migrate_handler_invoke_all('User', 'fields', $this->entityType, $this->bundle, $migration);
return $fields;
}
/**
* Delete a batch of users at once.
*
* @param $uids
* Array of user IDs to be deleted.
*/
public function bulkRollback(array $uids) {
migrate_instrument_start('user_delete_multiple');
$this->prepareRollback($uids);
user_delete_multiple($uids);
$this->completeRollback($uids);
migrate_instrument_stop('user_delete_multiple');
}
/**
* Import a single user.
*
* @param $account
* Account object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields (uid only in this case) of the user that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $account, stdClass $row) {
$migration = Migration::currentMigration();
// Updating previously-migrated content?
if (isset($row->migrate_map_destid1)) {
// Make sure is_new is off
$account->is_new = FALSE;
if (isset($account->uid)) {
if ($account->uid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming uid !uid and map destination uid !destid1 don't match",
array('!uid' => $account->uid, '!destid1' => $row->migrate_map_destid1)));
}
}
else {
$account->uid = $row->migrate_map_destid1;
}
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($account->uid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination uid provided'));
}
$old_account = user_load($account->uid, TRUE);
if (empty($old_account)) {
throw new MigrateException(t('System-of-record is DESTINATION, but user !uid does not exist',
array('!uid' => $account->uid)));
}
}
else {
$old_account = $account;
}
// Roles must be arrays keyed by the role id, which isn't how the data
// naturally comes in. Fix them up.
// First, if names instead of IDs are presented, translate them
if (!empty($account->role_names)) {
$role_names = is_array($account->role_names) ? $account->role_names : array($account->role_names);
foreach ($role_names as $role_name) {
$role = user_role_load_by_name($role_name);
if ($role) {
$account->roles[] = $role->rid;
}
}
}
if (!empty($account->roles)) {
if (!is_array($account->roles)) {
$account->roles = array($account->roles);
}
$account->roles = drupal_map_assoc($account->roles);
}
if (empty($account->roles) && empty($old_account->roles)) {
$account->roles = array();
}
$this->prepare($account, $row);
if (isset($account->uid) && !(isset($account->is_new) && $account->is_new)) {
$updating = TRUE;
}
else {
$updating = FALSE;
}
// While user_save is happy to see a fid in $account->picture on insert,
// when updating an existing account it wants a file object.
if ($updating && ($fid = $account->picture)) {
$account->picture = file_load($fid);
}
// Normalize MD5 passwords to lowercase, as generated by Drupal 6 and previous
if ($this->md5Passwords) {
$account->pass = drupal_strtolower($account->pass);
}
// If any datetime values were included, ensure that they're in timestamp format.
if (isset($account->created)) {
$account->created = MigrationBase::timestamp($account->created);
}
if (isset($account->access)) {
$account->access = MigrationBase::timestamp($account->access);
}
if (isset($account->login)) {
$account->login = MigrationBase::timestamp($account->login);
}
migrate_instrument_start('user_save');
$newaccount = user_save($old_account, (array)$account);
migrate_instrument_stop('user_save');
if ($newaccount) {
if ($this->md5Passwords && !empty($account->pass)) {
// Ape the Drupal 6 -> Drupal 7 upgrade, which encrypts the MD5 text in the
// modern way, and marks it with a prepended U so it recognizes and fixes it
// up at login time.
$password = 'U' . $newaccount->pass;
db_update('users')
->fields(array('pass' => $password))
->condition('uid', $newaccount->uid)
->execute();
}
if ($updating) {
$this->numUpdated++;
}
else {
$this->numCreated++;
}
$this->complete($newaccount, $row);
$return = array($newaccount->uid);
}
else {
$return = FALSE;
}
return $return;
}
}
class MigrateDestinationRole extends MigrateDestinationTable {
public function __construct() {
parent::__construct('role');
}
/**
* Get the key definition for the role table.
*
* @param $dummy
* PHP is picky - it throws E_STRICT notices if we don't have a parameter
* because MigrateDestinationTable has one.
*/
static public function getKeySchema($dummy = NULL) {
return MigrateDestinationTable::getKeySchema('role');
}
/**
* Delete a single row.
*
* @param $id
* Primary key values.
*/
public function rollback(array $id) {
migrate_instrument_start('role rollback');
$rid = reset($id);
user_role_delete((int)$rid);
migrate_instrument_stop('role rollback');
}
/**
* Import a single row.
*
* @param $entity
* Object object to build. Prefilled with any fields mapped in the Migration.
* @param $row
* Raw source data object - passed through to prepare/complete handlers.
* @return array
* Array of key fields of the object that was saved if
* successful. FALSE on failure.
*/
public function import(stdClass $entity, stdClass $row) {
$migration = Migration::currentMigration();
// Updating previously-migrated content?
if (isset($row->migrate_map_destid1)) {
if (isset($entity->rid)) {
if ($entity->rid != $row->migrate_map_destid1) {
throw new MigrateException(t("Incoming id !id and map destination id !destid don't match",
array('!id' => $entity->rid, '!destid' => $row->migrate_map_destid1)));
}
else {
$entity->rid = $row->migrate_map_destid1;
}
}
}
if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
if (!isset($entity->rid)) {
throw new MigrateException(t('System-of-record is DESTINATION, but no destination id provided'));
}
$old_entity = user_role_load($entity->rid);
foreach ($entity as $field => $value) {
$old_entity->$field = $entity->$field;
}
$entity = $old_entity;
}
$this->prepare($entity, $row);
user_role_save($entity);
$this->complete($entity, $row);
if (!empty($entity->rid)) {
$id = array($entity->rid);
}
else {
$id = FALSE;
}
return $id;
}
}

208
sites/all/modules/migrate/plugins/sources/csv.inc Normal file
View File

@@ -0,0 +1,208 @@
<?php
/**
* @file
* Define a MigrateSource for importing from comma separated values files.
*/
/**
* Implementation of MigrateSource, to handle imports from CSV files.
*
* If the CSV file contains non-ASCII characters, make sure it includes a
* UTF BOM (Byte Order Marker) so they are interpreted correctly.
*/
class MigrateSourceCSV extends MigrateSource {
/**
* List of available source fields.
*
* @var array
*/
protected $fields = array();
/**
* Parameters for the fgetcsv() call.
*
* @var array
*/
protected $fgetcsv = array();
/**
* File handle for the CSV file being iterated.
*
* @var resource
*/
protected $csvHandle = NULL;
/**
* The number of rows in the CSV file before the data starts.
*
* @var integer
*/
protected $headerRows = 0;
/**
* Simple initialization.
*
* @param string $path
* The path to the source file
* @param array $csvcolumns
* Keys are integers. values are array(field name, description).
* @param array $options
* Options applied to this source.
* @param array $fields
* Optional - keys are field names, values are descriptions. Use to override
* the default descriptions, or to add additional source fields which the
* migration will add via other means (e.g., prepareRow()).
*/
public function __construct($path, array $csvcolumns = array(), array $options = array(), array $fields = array()) {
parent::__construct($options);
$this->file = $path;
if (!empty($options['header_rows'])) {
$this->headerRows = $options['header_rows'];
}
else {
$this->headerRows = 0;
}
$this->options = $options;
$this->fields = $fields;
// fgetcsv specific options
foreach (array('length' => NULL, 'delimiter' => ',', 'enclosure' => '"', 'escape' => '\\') as $key => $default) {
$this->fgetcsv[$key] = isset($options[$key]) ? $options[$key] : $default;
}
// One can either pass in an explicit list of column names to use, or if we have
// a header row we can use the names from that
if ($this->headerRows && empty($csvcolumns)) {
$this->csvcolumns = array();
$this->csvHandle = fopen($this->file, 'r');
// Skip all but the last header
for ($i = 0; $i < $this->headerRows - 1; $i++) {
$this->getNextLine();
}
$row = $this->getNextLine();
foreach ($row as $header) {
$header = trim($header);
$this->csvcolumns[] = array($header, $header);
}
fclose($this->csvHandle);
unset($this->csvHandle);
}
else {
$this->csvcolumns = $csvcolumns;
}
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
return $this->file;
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
$fields = array();
foreach ($this->csvcolumns as $values) {
$fields[$values[0]] = $values[1];
}
// Any caller-specified fields with the same names as extracted fields will
// override them; any others will be added
if ($this->fields) {
$fields = $this->fields + $fields;
}
return $fields;
}
/**
* Return a count of all available source records.
*/
public function computeCount() {
// If the data may have embedded newlines, the file line count won't reflect
// the number of CSV records (one record will span multiple lines). We need
// to scan with fgetcsv to get the true count.
if (!empty($this->options['embedded_newlines'])) {
$result = fopen($this->file, 'r');
// Skip all but the last header
for ($i = 0; $i < $this->headerRows; $i++) {
fgets($result);
}
$count = 0;
while ($this->getNextLine()) {
$count++;
}
fclose($result);
}
else {
// TODO. If this takes too much time/memory, use exec('wc -l')
$count = count(file($this->file));
$count -= $this->headerRows;
}
return $count;
}
/**
* Implementation of MigrateSource::performRewind().
*
* @return void
*/
public function performRewind() {
// Close any previously-opened handle
if (!is_null($this->csvHandle)) {
fclose($this->csvHandle);
}
// Load up the first row, skipping the header(s) if necessary
$this->csvHandle = fopen($this->file, 'r');
for ($i = 0; $i < $this->headerRows; $i++) {
$this->getNextLine();
}
}
/**
* Implementation of MigrateSource::getNextRow().
* Return the next line of the source CSV file as an object.
*
* @return null|object
*/
public function getNextRow() {
$row = $this->getNextLine();
if ($row) {
// Set meaningful keys for the columns mentioned in $this->csvcolumns().
foreach ($this->csvcolumns as $int => $values) {
list($key, $description) = $values;
// Copy value to more descriptive string based key and then unset original.
$row[$key] = isset($row[$int]) ? $row[$int] : NULL;
unset($row[$int]);
}
return (object)$row;
}
else {
fclose($this->csvHandle);
$this->csvHandle = NULL;
return NULL;
}
}
protected function getNextLine() {
// escape parameter was added in PHP 5.3.
if (version_compare(phpversion(), '5.3', '<')) {
$row = fgetcsv($this->csvHandle, $this->fgetcsv['length'],
$this->fgetcsv['delimiter'], $this->fgetcsv['enclosure']);
}
else {
$row = fgetcsv($this->csvHandle, $this->fgetcsv['length'],
$this->fgetcsv['delimiter'], $this->fgetcsv['enclosure'],
$this->fgetcsv['escape']);
}
return $row;
}
}

175
sites/all/modules/migrate/plugins/sources/files.inc Normal file
View File

@@ -0,0 +1,175 @@
<?php
/**
* @file
* Support for migration from files sources.
*/
/**
* Implementation of MigrateList, for retrieving a list of IDs to be migrated
* from a directory listing. Each item is a file, it's ID is the path.
*/
class MigrateListFiles extends MigrateList {
protected $listDirs;
protected $baseDir;
protected $fileMask;
protected $directoryOptions;
/**
* Constructor.
*
* @param $list_dirs
* Array of directory paths that will be scanned for files. No trailing
* slash. For example:
* array(
* '/var/html_source/en/news',
* '/var/html_source/fr/news',
* '/var/html_source/zh/news',
* );
* @param $base_dir
* The base dir is the part of the path that will be excluded when making
* an ID for each file. To continue the example from above, you want base_dir
* to be = '/var/html_source', so that the files will have IDs in the format
* '/en/news/news_2011_03_4.html'.
* @param $file_mask
* Passed on and used to filter for certain types of files. Use a regular
* expression, for example '/(.*\.htm$|.*\.html$)/i' to match all .htm and
* .html files, case insensitive.
* @param $options
* Options that will be passed on to file_scan_directory(). See docs of that
* core Drupal function for more information.
*/
public function __construct($list_dirs, $base_dir, $file_mask = NULL, $options = array()) {
parent::__construct();
$this->listDirs = $list_dirs;
$this->baseDir = $base_dir;
$this->fileMask = $file_mask;
$this->directoryOptions = $options;
}
/**
* Our public face is the directories we're getting items from.
*/
public function __toString() {
if (is_array($this->listDirs)) {
return implode(',', $this->listDirs);
}
else {
return $this->listDirs;
}
}
/**
* Retrieve a list of files based on parameters passed for the migration.
*/
public function getIdList() {
$files = array();
foreach ($this->listDirs as $dir) {
migrate_instrument_start("Retrieve $dir");
$files = array_merge(file_scan_directory($dir, $this->fileMask, $this->directoryOptions), $files);
migrate_instrument_stop("Retrieve $dir");
}
if (isset($files)) {
return $this->getIDsFromFiles($files);
}
Migration::displayMessage(t('Loading of !listuri failed:', array('!listuri' => $this->listUri)));
return NULL;
}
/**
* Given an array generated from file_scan_directory(), parse out the IDs for
* processing and return them as an array.
*/
protected function getIDsFromFiles(array $files) {
$ids = array();
foreach ($files as $file) {
$ids[] = str_replace($this->baseDir, '', (string) $file->uri);
}
return array_unique($ids);
}
/**
* Return a count of all available IDs from the source listing.
*/
public function computeCount() {
$count = 0;
$files = $this->getIdList();
if ($files) {
$count = count($files);
}
return $count;
}
}
/**
* Implementation of MigrateItem, for retrieving a file from the file system
* based on source directory and an ID provided by a MigrateList class.
*/
class MigrateItemFile extends MigrateItem {
protected $baseDir;
protected $getContents;
/**
* Constructor.
*
* @param $base_dir
* The base directory from which all file paths are calculated.
* @param $get_contents
* TRUE if we should try load the contents of each file (in the case
* of a text file), or FALSE if we just want to confirm it exists (binary).
*/
public function __construct($base_dir, $get_contents = TRUE) {
parent::__construct();
$this->baseDir = $base_dir;
$this->getContents = $get_contents;
}
/**
* Return an object representing a file.
*
* @param $id
* The file id, which is the file URI.
*
* @return object
* The item object for migration.
*/
public function getItem($id) {
$item_uri = $this->baseDir . $id;
// Get the file data at the specified URI
$data = $this->loadFile($item_uri);
if (is_string($data)) {
$return = new stdClass;
$return->filedata = $data;
return $return;
}
elseif ($data === TRUE) {
$return = new stdClass;
return $return;
}
else {
$migration = Migration::currentMigration();
$message = t('Loading of !objecturi failed:', array('!objecturi' => $item_uri));
$migration->getMap()->saveMessage(
array($id), $message, MigrationBase::MESSAGE_ERROR);
return NULL;
}
}
/**
* Default file loader.
*/
protected function loadFile($item_uri) {
// Only try load the contents if we have this flag set.
if ($this->getContents) {
$data = file_get_contents($item_uri);
}
else {
$data = file_exists($item_uri);
}
return $data;
}
}

172
sites/all/modules/migrate/plugins/sources/json.inc Normal file
View File

@@ -0,0 +1,172 @@
<?php
/**
* @file
* Support for migration from JSON sources.
*/
/**
* Implementation of MigrateList, for retrieving a list of IDs to be migrated
* from a JSON object.
*/
class MigrateListJSON extends MigrateList {
/**
* A URL pointing to an JSON object containing a list of IDs to be processed.
*
* @var string
*/
protected $listUrl;
protected $httpOptions;
public function __construct($list_url, $http_options = array()) {
parent::__construct();
$this->listUrl = $list_url;
$this->httpOptions = $http_options;
}
/**
* Our public face is the URL we're getting items from
*
* @return string
*/
public function __toString() {
return $this->listUrl;
}
/**
* Load the JSON at the given URL, and return an array of the IDs found within it.
*
* @return array
*/
public function getIdList() {
migrate_instrument_start("Retrieve $this->listUrl");
if (empty($this->httpOptions)) {
$json = file_get_contents($this->listUrl);
}
else {
$response = drupal_http_request($this->listUrl, $this->httpOptions);
$json = $response->data;
}
migrate_instrument_stop("Retrieve $this->listUrl");
if ($json) {
$data = drupal_json_decode($json);
if ($data) {
return $this->getIDsFromJSON($data);
}
}
Migration::displayMessage(t('Loading of !listurl failed:',
array('!listurl' => $this->listUrl)));
return NULL;
}
/**
* Given an array generated from JSON, parse out the IDs for processing
* and return them as an array. The default implementation assumes the IDs are
* simply the values of the top-level elements - in most cases, you will need
* to override this to reflect your particular JSON structure.
*
* @param array $data
*
* @return array
*/
protected function getIDsFromJSON(array $data) {
return $data;
}
/**
* Return a count of all available IDs from the source listing. The default
* implementation assumes the count of top-level elements reflects the number
* of IDs available - in many cases, you will need to override this to reflect
* your particular JSON structure.
*/
public function computeCount() {
$count = 0;
if (empty($this->httpOptions)) {
$json = file_get_contents($this->listUrl);
}
else {
$response = drupal_http_request($this->listUrl, $this->httpOptions);
$json = $response->data;
}
if ($json) {
$data = drupal_json_decode($json);
if ($data) {
$count = count($data);
}
}
return $count;
}
}
/**
* Implementation of MigrateItem, for retrieving a parsed JSON object given
* an ID provided by a MigrateList class.
*/
class MigrateItemJSON extends MigrateItem {
/**
* A URL pointing to a JSON object containing the data for one item to be
* migrated.
*
* @var string
*/
protected $itemUrl;
protected $httpOptions;
public function __construct($item_url, $http_options) {
parent::__construct();
$this->itemUrl = $item_url;
$this->httpOptions = $http_options;
}
/**
* Implementors are expected to return an object representing a source item.
*
* @param mixed $id
*
* @return stdClass
*/
public function getItem($id) {
$item_url = $this->constructItemUrl($id);
// Get the JSON object at the specified URL
$json = $this->loadJSONUrl($item_url);
if ($json) {
return $json;
}
else {
$migration = Migration::currentMigration();
$message = t('Loading of !objecturl failed:', array('!objecturl' => $item_url));
$migration->getMap()->saveMessage(
array($id), $message, MigrationBase::MESSAGE_ERROR);
return NULL;
}
}
/**
* The default implementation simply replaces the :id token in the URL with
* the ID obtained from MigrateListJSON. Override if the item URL is not
* so easily expressed from the ID.
*
* @param mixed $id
*/
protected function constructItemUrl($id) {
return str_replace(':id', $id, $this->itemUrl);
}
/**
* Default JSON loader - just pull and decode. This can be overridden for
* preprocessing of JSON (removal of unwanted elements, caching of JSON if the
* source service is slow, etc.)
*/
protected function loadJSONUrl($item_url) {
if (empty($this->httpOptions)) {
$json = file_get_contents($item_url);
}
else {
$response = drupal_http_request($item_url, $this->httpOptions);
$json = $response->data;
}
return json_decode($json);
}
}

195
sites/all/modules/migrate/plugins/sources/list.inc Normal file
View File

@@ -0,0 +1,195 @@
<?php
/**
* @file
* Support for migration from sources with distinct means of listing items to
* import and obtaining the items themselves.
*
* TODO: multiple-field source keys
*/
/**
* Extend the MigrateList class to provide a means to obtain a list of IDs to
* be migrated from a given source (e.g., MigrateListXML extends MigrateList to
* obtain a list of IDs from an XML document).
*/
abstract class MigrateList {
public function __construct() {}
/**
* Implementors are expected to return a string representing where the listing
* is obtained from (a URL, file directory, etc.)
*
* @return string
*/
abstract public function __toString();
/**
* Implementors are expected to return an array of unique IDs, suitable for
* passing to the MigrateItem class to retrieve the data for a single item.
*
* @return Mixed, iterator or array
*/
abstract public function getIdList();
/**
* Implementors are expected to return a count of IDs available to be migrated.
*
* @return int
*/
abstract public function computeCount();
}
/**
* Extend the MigrateItem class to provide a means to obtain the data for a
* given migratable item given its ID as provided by the MigrateList class.
*/
abstract class MigrateItem {
public function __construct() {}
/**
* Implementors are expected to return an object representing a source item.
*
* @param mixed $id
*
* @return stdClass
*/
abstract public function getItem($id);
}
/**
* Implementation of MigrateSource, providing the semantics of iterating over
* IDs provided by a MigrateList and retrieving data from a MigrateItem.
*/
class MigrateSourceList extends MigrateSource {
/**
* MigrateList object used to obtain ID lists.
*
* @var MigrateList
*/
protected $listClass;
/**
* MigrateItem object used to obtain the source object for a given ID.
*
* @var MigrateItem
*/
protected $itemClass;
/**
* Iterator of IDs from the listing class.
*
* @var Iterator
*/
protected $idIterator;
/**
* List of available source fields.
*
* @var array
*/
protected $fields = array();
/**
* Simple initialization.
*/
public function __construct(MigrateList $list_class, MigrateItem $item_class, $fields = array(),
$options = array()) {
parent::__construct($options);
$this->listClass = $list_class;
$this->itemClass = $item_class;
$this->fields = $fields;
}
/**
* Return a string representing the source.
*
* @return string
*/
public function __toString() {
return (string) $this->listClass;
}
/**
* Returns a list of fields available to be mapped from the source query.
* Since we can't reliably figure out what "fields" are in the source,
* it's up to the implementing Migration constructor to fill them in.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
return $this->fields;
}
/**
* It's the list class that knows how many records are available, so ask it.
*
* @return int
*/
public function computeCount() {
// @API: Support old count method for now.
if (method_exists($this->listClass, 'computeCount')) {
return $this->listClass->computeCount();
}
else {
return $this->listClass->count();
}
}
/**
* Implementation of MigrateSource::performRewind().
*
* @return void
*/
public function performRewind() {
// If there isn't a specific ID list passed in, get it from the list class.
if ($this->idList) {
$this->idsToProcess = $this->idList;
}
else {
$this->idsToProcess = $this->listClass->getIdList();
}
$this->idIterator = ($this->idsToProcess instanceof Iterator) ?
$this->idsToProcess : new ArrayIterator($this->idsToProcess);
$this->idIterator->rewind();
}
/**
* Implementation of MigrateSource::getNextRow().
*
* @return null|stdClass
*/
public function getNextRow() {
$row = NULL;
while ($this->idIterator->valid()) {
$ids = $this->idIterator->current();
$this->idIterator->next();
// Skip empty IDs
if (empty($ids)) {
continue;
}
// Got a good ID, get the data and get out.
$row = $this->itemClass->getItem($ids);
if ($row) {
// No matter what $ids is, be it a string, integer, object, or array, we
// cast it to an array so that it can be properly mapped to the source
// keys as specified by the map. This is done after getItem is called so
// that the ItemClass doesn't have to care about this requirement.
$ids = (array) $ids;
foreach (array_keys($this->activeMap->getSourceKey()) as $key_name) {
// Grab the first id and advance the array cursor. Then save the ID
// using the map source key - it will be used for mapping.
list(, $id) = each($ids);
$row->$key_name = $id;
}
}
break;
}
return $row;
}
}

206
sites/all/modules/migrate/plugins/sources/mssql.inc Normal file
View File

@@ -0,0 +1,206 @@
<?php
/**
* @file
* Define a MigrateSource for importing from Microsoft SQL Server databases.
*/
/**
* Implementation of MigrateSource, to handle imports from remote MS SQL Server db servers.
*/
class MigrateSourceMSSQL extends MigrateSource {
/**
* Array containing information for connecting to SQL Server:
* servername - Hostname of the SQL Server
* username - Username to connect as
* password - Password for logging in
* database (optional) - Database to select after connecting
*
* @var array
*/
protected $configuration;
/**
* The active MS SQL Server connection for this source.
*
* @var resource
*/
protected $connection;
/**
* The SQL query from which to obtain data. Is a string.
*/
protected $query;
/**
* The result object from executing the query - traversed to process the
* incoming data.
*/
protected $result;
/**
* By default, mssql_query fetches all results - severe memory problems with
* big tables. So, we will fetch a batch at a time.
*
* @var int
*/
protected $batchSize;
/**
* Return an options array for MS SQL sources.
*
* @param int $batch_size
* Number of rows to pull at once (defaults to 500).
* @param boolean $cache_counts
* Indicates whether to cache counts of source records.
*/
static public function options($batch_size, $cache_counts) {
return compact('batch_size', 'cache_counts');
}
/**
* Simple initialization.
*/
public function __construct(array $configuration, $query, $count_query,
array $fields, array $options = array()) {
parent::__construct($options);
$this->query = $query;
$this->countQuery = $count_query;
$this->configuration = $configuration;
$this->fields = $fields;
$this->batchSize = isset($options['batch_size']) ? $options['batch_size'] : 500;
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
return $this->query;
}
/**
* Connect lazily to the DB server.
*/
protected function connect() {
if (!isset($this->connection)) {
if (!extension_loaded('mssql')) {
throw new Exception(t('You must configure the mssql extension in PHP.'));
}
if (isset($this->configuration['port'])) {
$host = $this->configuration['servername'] . ':' . $this->configuration['port'];
}
else {
$host = $this->configuration['servername'];
}
$this->connection = mssql_connect(
$host,
$this->configuration['username'],
$this->configuration['password'],
TRUE);
if (isset($this->configuration['database'])) {
return mssql_select_db($this->configuration['database'], $this->connection);
}
}
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
// The fields are passed to the constructor for this plugin.
return $this->fields;
}
/**
* Return a count of all available source records.
*/
public function computeCount() {
migrate_instrument_start('MigrateSourceMSSQL count');
if ($this->connect()) {
$result = mssql_query($this->countQuery);
$count = reset(mssql_fetch_object($result));
}
else {
// Do something else?
$count = FALSE;
}
migrate_instrument_stop('MigrateSourceMSSQL count');
return $count;
}
/**
* Implementation of MigrateSource::performRewind().
*/
public function performRewind() {
/*
* Replace :criteria placeholder with idlist or highwater clauses. We
* considered supporting both but it is not worth the complexity. Run twice
* instead.
*/
if (!empty($this->idList)) {
$keys = array();
foreach ($this->activeMap->getSourceKey() as $field_name => $field_schema) {
// Allow caller to provide an alias to table containing the primary key.
if (!empty($field_schema['alias'])) {
$field_name = $field_schema['alias'] . '.' . $field_name;
}
$keys[] = $field_name;
}
// TODO: Sanitize. not critical as this is admin supplied data in drush.
$this->query = str_replace(':criteria',
$keys[0] . ' IN (' . implode(',', $this->idList) . ')', $this->query);
}
else {
if (isset($this->highwaterField['name']) && $highwater = $this->activeMigration->getHighwater()) {
if (empty($this->highwaterField['alias'])) {
$highwater_name = $this->highwaterField['name'];
}
else {
$highwater_name = $this->highwaterField['alias'] . '.' . $this->highwaterField['name'];
}
$this->query = str_replace(':criteria', "$highwater_name > '$highwater'", $this->query);
}
else {
// No idlist or highwater. Replace :criteria placeholder with harmless WHERE
// clause instead of empty since we don't know if an AND follows.
$this->query = str_replace(':criteria', '1=1', $this->query);
}
}
migrate_instrument_start('mssql_query');
$this->connect();
$this->result = mssql_query($this->query, $this->connection, $this->batchSize);
migrate_instrument_stop('mssql_query');
}
/**
* Implementation of MigrateSource::getNextRow().
*
* Returns the next row of the result set as an object, dealing with the
* difference between the end of the batch and the end of all data.
*/
public function getNextRow() {
$row = mssql_fetch_object($this->result);
// Might be totally out of data, or just out of this batch - request another
// batch and see
if (!is_object($row)) {
mssql_fetch_batch($this->result);
$row = mssql_fetch_object($this->result);
}
if (is_object($row)) {
return $row;
}
else {
return NULL;
}
}
}

186
sites/all/modules/migrate/plugins/sources/multiitems.inc Normal file
View File

@@ -0,0 +1,186 @@
<?php
/**
* @file
* Support for migration from sources where data spans multiple lines
* (ex. xml, json) and IDs for the items are part of each item and multiple
* items reside in a single file.
*/
/**
* Extend the MigrateItems class to provide a means to obtain a list of IDs to
* be migrated from a given source (e.g., MigrateItemsXML extends MigrateItem to
* obtain a list of IDs from an XML document). This class also provides a means
* to obtain the data for a given migratable item given its ID.
*/
abstract class MigrateItems {
public function __construct() {}
/**
* Implementors are expected to return a string representing where the listing
* is obtained from (a URL, file directory, etc.)
*
* @return string
*/
abstract public function __toString();
/**
* Implementors are expected to return an array of unique IDs, suitable for
* passing to the MigrateItem class to retrieve the data for a single item.
*
* @return Mixed, iterator or array
*/
abstract public function getIdList();
/**
* Implementors are expected to return a count of IDs available to be migrated.
*
* @return int
*/
abstract public function computeCount();
/**
* Implementors are expected to return an object representing a source item.
*
* @param mixed $id
*
* @return stdClass
*/
abstract public function getItem($id);
}
/**
* Implementation of MigrateItems, for providing a list of IDs and for
* retrieving a parsed XML document given an ID from this list.
*/
/**
* Implementation of MigrateSource, providing the semantics of iterating over
* IDs provided by a MigrateItems and retrieving data from a MigrateItems.
*/
class MigrateSourceMultiItems extends MigrateSource {
/**
* MigrateItems object used to obtain the list of IDs and source for
* all objects.
*
* @var MigrateItems
*/
protected $itemsClass;
/**
* List of available source fields.
*
* @var array
*/
protected $fields = array();
/**
* Iterator of IDs from the listing class.
*
* @var Iterator
*/
protected $idIterator;
/**
* List of item IDs to iterate.
*
* @var array
*/
protected $idsToProcess = array();
/**
* Simple initialization.
*/
public function __construct(MigrateItems $items_class, $fields = array(), $options = array()) {
parent::__construct($options);
$this->itemsClass = $items_class;
$this->fields = $fields;
}
/**
* Return a string representing the source.
*
* @return string
*/
public function __toString() {
return (string) $this->itemsClass;
}
/**
* Returns a list of fields available to be mapped from the source query.
* Since we can't reliably figure out what "fields" are in the source,
* it's up to the implementing Migration constructor to fill them in.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
return $this->fields;
}
/**
* It's the list class that knows how many records are available, so ask it.
*
* @return int
*/
public function computeCount() {
// @API: Support old count method for now.
if (method_exists($this->itemsClass, 'computeCount')) {
return $this->itemsClass->computeCount();
}
else {
return $this->itemsClass->count();
}
}
/**
* Implementation of MigrateSource::performRewind().
*
* @return void
*/
public function performRewind() {
// If there isn't a specific ID list passed in, get it from the list class.
if ($this->idList) {
$this->idsToProcess = $this->idList;
}
else {
$this->idsToProcess = $this->itemsClass->getIdList();
}
$this->idIterator = ($this->idsToProcess instanceof Iterator) ?
$this->idsToProcess : new ArrayIterator($this->idsToProcess);
$this->idIterator->rewind();
}
/**
* Implementation of MigrateSource::getNextRow().
*
* @return null|stdClass
*/
public function getNextRow() {
$row = NULL;
while ($this->idIterator->valid()) {
$id = $this->idIterator->current();
$this->idIterator->next();
// Skip empty IDs
if (empty($id)) {
continue;
}
// Got a good ID, get the data and get out.
$row = $this->itemsClass->getItem($id);
if ($row) {
// Save the ID using the map source key - it will be used for mapping
$sourceKey = $this->activeMap->getSourceKey();
$key_name = key($sourceKey);
$row->$key_name = $id;
}
break;
}
return $row;
}
}

221
sites/all/modules/migrate/plugins/sources/oracle.inc Normal file
View File

@@ -0,0 +1,221 @@
<?php
/**
* @file
* Define a MigrateSource class for importing from Oracle databases.
*/
/**
* Implementation of MigrateSource, to handle imports from remote Oracle servers.
*/
class MigrateSourceOracle extends MigrateSource {
/**
* Array containing information for connecting to Oracle:
* username - Username to connect as
* password - Password for logging in
* connection_string - See http://us.php.net/manual/en/function.oci-connect.php.
*
* @var array
*/
protected $configuration;
/**
* The active Oracle connection for this source.
*
* @var resource
*/
protected $connection;
public function getConnection() {
return $this->connection;
}
/**
* The SQL query from which to obtain data. Is a string.
*/
protected $query;
/**
* The result object from executing the query - traversed to process the
* incoming data.
*/
protected $result;
/**
* Character set to use in retrieving data.
*
* @var string
*/
protected $characterSet;
/**
* Return an options array for Oracle sources.
*
* @param string $character_set
* Character set to use in retrieving data. Defaults to 'UTF8'.
* @param boolean $cache_counts
* Indicates whether to cache counts of source records.
*/
static public function options($character_set = 'UTF8', $cache_counts = FALSE) {
return compact('character_set', 'cache_counts');
}
/**
* Simple initialization.
*/
public function __construct(array $configuration, $query, $count_query,
array $fields, array $options = array()) {
parent::__construct($options);
$this->query = $query;
$this->countQuery = $count_query;
$this->configuration = $configuration;
$this->fields = $fields;
if (empty($options['character_set'])) {
$this->characterSet = 'UTF8';
}
else {
$this->characterSet = $options['character_set'];
}
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
return $this->query;
}
/**
* Connect lazily to the DB server.
*/
protected function connect() {
if (!isset($this->connection)) {
if (!extension_loaded('oci8')) {
throw new Exception(t('You must configure the oci8 extension in PHP.'));
}
$this->connection = oci_connect($this->configuration['username'],
$this->configuration['password'], $this->configuration['connection_string'],
$this->characterSet);
}
if ($this->connection) {
return TRUE;
}
else {
$e = oci_error();
throw new Exception($e['message']);
return FALSE;
}
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
// The fields are passed to the constructor for this plugin.
return $this->fields;
}
/**
* Return a count of all available source records.
*/
public function computeCount() {
migrate_instrument_start('MigrateSourceOracle count');
if ($this->connect()) {
$statement = oci_parse($this->connection, $this->countQuery);
if (!$statement) {
$e = oci_error($this->connection);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
$result = oci_execute($statement);
if (!$result) {
$e = oci_error($statement);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
$count_array = oci_fetch_array($statement);
$count = reset($count_array);
}
else {
// Do something else?
$count = FALSE;
}
migrate_instrument_stop('MigrateSourceOracle count');
return $count;
}
/**
* Implementation of MigrateSource::performRewind().
*/
public function performRewind() {
$keys = array();
foreach ($this->activeMap->getSourceKey() as $field_name => $field_schema) {
// Allow caller to provide an alias to table containing the primary key.
if (!empty($field_schema['alias'])) {
$field_name = $field_schema['alias'] . '.' . $field_name;
}
$keys[] = $field_name;
}
/*
* Replace :criteria placeholder with idlist or highwater clauses. We
* considered supporting both but it is not worth the complexity. Run twice
* instead.
*/
if (!empty($this->idList)) {
// TODO: Sanitize. not critical as this is admin supplied data in drush.
$this->query = str_replace(':criteria',
$keys[0] . ' IN (' . implode(',', $this->idList) . ')', $this->query);
}
else {
if (isset($this->highwaterField['name']) && $highwater = $this->activeMigration->getHighwater()) {
if (empty($this->highwaterField['alias'])) {
$highwater_name = $this->highwaterField['name'];
}
else {
$highwater_name = $this->highwaterField['alias'] . '.' . $this->highwaterField['name'];
}
$this->query = str_replace(':criteria', "$highwater_name > '$highwater'", $this->query);
}
else {
// No idlist or highwater. Replace :criteria placeholder with harmless WHERE
// clause instead of empty since we don't know if an AND follows.
$this->query = str_replace(':criteria', '1=1', $this->query);
}
}
migrate_instrument_start('oracle_query');
$this->connect();
$this->result = oci_parse($this->connection, $this->query);
if (!$this->result) {
$e = oci_error($this->connection);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
$status = oci_execute($this->result);
if (!$status) {
$e = oci_error($this->result);
throw new Exception($e['message'] . "\n" . $e['sqltext']);
}
migrate_instrument_stop('oracle_query');
}
/**
* Implementation of MigrateSource::getNextRow().
*
* Returns the next row of the result set as an object, making sure NULLs are
* represented as PHP NULLs and that LOBs are returned directly without special
* handling.
*/
public function getNextRow() {
$row = oci_fetch_array($this->result, OCI_ASSOC | OCI_RETURN_NULLS | OCI_RETURN_LOBS);
if (!empty($row)) {
return (object)$row;
}
else {
return FALSE;
}
}
}

337
sites/all/modules/migrate/plugins/sources/sql.inc Normal file
View File

@@ -0,0 +1,337 @@
<?php
/**
* @file
* Define a MigrateSource for importing from Drupal connections
*/
/**
* Implementation of MigrateSource, to handle imports from Drupal connections.
*/
class MigrateSourceSQL extends MigrateSource {
/**
* The SQL query objects from which to obtain data, and counts of data
*
* @var SelectQueryInterface
*/
protected $originalQuery, $query, $countQuery;
/**
* The result object from executing the query - traversed to process the
* incoming data.
*
* @var DatabaseStatementInterface
*/
protected $result;
/**
* Number of eligible rows processed so far (used for itemlimit checking)
*
* @var int
*/
protected $numProcessed = 0;
/**
* List of available source fields.
*
* @var array
*/
protected $fields = array();
/**
* If the map is a MigrateSQLMap, and the table is compatible with the
* source query, we can join directly to the map and make things much faster
* and simpler.
*
* @var boolean
*/
protected $mapJoinable = FALSE;
// Dynamically set whether the map is joinable - not really for production use,
// this is primarily to support simpletests
public function setMapJoinable($map_joinable) {
$this->mapJoinable = $map_joinable;
}
/**
* Whether this source is configured to use a highwater mark, and there is
* a highwater mark present to use.
*
* @var boolean
*/
protected $usingHighwater = FALSE;
/**
* Whether, in the current iteration, we have reached the highwater mark.
*
* @var boolen
*/
protected $highwaterSeen = FALSE;
/**
* Return an options array for PDO sources.
*
* @param boolean $map_joinable
* Indicates whether the map table can be joined directly to the source query.
* @param boolean $cache_counts
* Indicates whether to cache counts of source records.
*/
static public function options($map_joinable, $cache_counts) {
return compact('map_joinable', 'cache_counts');
}
/**
* Simple initialization.
*
* @param SelectQueryInterface $query
* The query we are iterating over.
* @param array $fields
* Optional - keys are field names, values are descriptions. Use to override
* the default descriptions, or to add additional source fields which the
* migration will add via other means (e.g., prepareRow()).
* @param SelectQueryInterface $count_query
* Optional - an explicit count query, primarily used when counting the
* primary query is slow.
* @param boolean $options
* Options applied to this source.
*/
public function __construct(SelectQueryInterface $query, array $fields = array(),
SelectQueryInterface $count_query = NULL, array $options = array()) {
parent::__construct($options);
$this->originalQuery = $query;
$this->query = clone $query;
$this->fields = $fields;
if (is_null($count_query)) {
$this->countQuery = clone $query->countQuery();
}
else {
$this->countQuery = $count_query;
}
if (isset($options['map_joinable'])) {
$this->mapJoinable = $options['map_joinable'];
}
else {
// TODO: We want to automatically determine if the map table can be joined
// directly to the query, but this won't work unless/until
// http://drupal.org/node/802514 is committed, assume joinable for now
$this->mapJoinable = TRUE;
/* // To be able to join the map directly, it must be a PDO map on the same
// connection, or a compatible connection
$map = $migration->getMap();
if (is_a($map, 'MigrateSQLMap')) {
$map_options = $map->getConnection()->getConnectionOptions();
$query_options = $this->query->connection()->getConnectionOptions();
// Identical options means it will work
if ($map_options == $query_options) {
$this->mapJoinable = TRUE;
}
else {
// Otherwise, the one scenario we know will work is if it's MySQL and
// the credentials match (SQLite too?)
if ($map_options['driver'] == 'mysql' && $query_options['driver'] == 'mysql') {
if ($map_options['host'] == $query_options['host'] &&
$map_options['port'] == $query_options['port'] &&
$map_options['username'] == $query_options['username'] &&
$map_options['password'] == $query_options['password']) {
$this->mapJoinable = TRUE;
}
}
}
}*/
}
}
/**
* Return a string representing the source query.
*
* @return string
*/
public function __toString() {
return (string) $this->query;
}
/**
* Returns a list of fields available to be mapped from the source query.
*
* @return array
* Keys: machine names of the fields (to be passed to addFieldMapping)
* Values: Human-friendly descriptions of the fields.
*/
public function fields() {
$fields = array();
$queryFields = $this->query->getFields();
if ($queryFields) {
// Not much we can do in terms of describing the fields without manual intervention
foreach ($queryFields as $field_name => $field_info) {
// Lower case, because Drupal forces lowercase on fetch
$fields[drupal_strtolower($field_name)] = drupal_strtolower(
$field_info['table'] . '.' . $field_info['field']);
}
}
else {
// Detect available fields
$detection_query = clone $this->query;
$result = $detection_query->range(0, 1)->execute();
$row = $result->fetchAssoc();
if (is_array($row)) {
foreach ($row as $field_name => $field_value) {
// Lower case, because Drupal forces lowercase on fetch
$fields[drupal_strtolower($field_name)] = t('Example Content: !value',
array('!value' => $field_value));
}
}
}
/*
* Handle queries without explicit field lists
* TODO: Waiting on http://drupal.org/node/814312
$info = Database::getConnectionInfo($query->getConnection());
$database = $info['default']['database'];
foreach ($this->query->getTables() as $table) {
if (isset($table['all_fields']) && $table['all_fields']) {
$database = 'plants';
$table = $table['table'];
$sql = 'SELECT column_name
FROM information_schema.columns
WHERE table_schema=:database AND table_name = :table
ORDER BY ordinal_position';
$result = dbtng_query($sql, array(':database' => $database, ':table' => $table));
foreach ($result as $row) {
$fields[drupal_strtolower($row->column_name)] = drupal_strtolower(
$table . '.' . $row->column_name);
}
}
}*/
$expressionFields = $this->query->getExpressions();
foreach ($expressionFields as $field_name => $field_info) {
// Lower case, because Drupal forces lowercase on fetch
$fields[drupal_strtolower($field_name)] = drupal_strtolower($field_info['alias']);
}
// Any caller-specified fields with the same names as extracted fields will
// override them; any others will be added
if ($this->fields) {
$fields = $this->fields + $fields;
}
return $fields;
}
/**
* Return a count of all available source records.
*/
public function computeCount() {
$count = $this->countQuery->execute()->fetchField();
return $count;
}
/**
* Implementation of MigrateSource::performRewind().
*
* We could simply execute the query and be functionally correct, but
* we will take advantage of the PDO-based API to optimize the query up-front.
*/
public function performRewind() {
$this->result = NULL;
$this->query = clone $this->originalQuery;
// Get the key values, for potential use in joining to the map table, or
// enforcing idlist.
$keys = array();
foreach ($this->activeMap->getSourceKey() as $field_name => $field_schema) {
if (isset($field_schema['alias'])) {
$field_name = $field_schema['alias'] . '.' . $field_name;
}
$keys[] = $field_name;
}
// The rules for determining what conditions to add to the query are as
// follows (applying first applicable rule)
// 1. If idlist is provided, then only process items in that list (AND key
// IN (idlist)). Only applicable with single-value keys.
if ($this->idList) {
$this->query->condition($keys[0], $this->idList, 'IN');
}
else {
// 2. If the map is joinable, join it. We will want to accept all rows
// which are either not in the map, or marked in the map as NEEDS_UPDATE.
// Note that if highwater fields are in play, we want to accept all rows
// above the highwater mark in addition to those selected by the map
// conditions, so we need to OR them together (but AND with any existing
// conditions in the query). So, ultimately the SQL condition will look
// like (original conditions) AND (map IS NULL OR map needs update
// OR above highwater).
$conditions = db_or();
$condition_added = FALSE;
if ($this->mapJoinable) {
// Build the join to the map table. Because the source key could have
// multiple fields, we need to build things up.
$count = 1;
foreach ($this->activeMap->getSourceKey() as $field_name => $field_schema) {
if (isset($field_schema['alias'])) {
$field_name = $field_schema['alias'] . '.' . $field_name;
}
$map_key = 'sourceid' . $count++;
if (!isset($map_join)) {
$map_join = '';
}
else {
$map_join .= ' AND ';
}
$map_join .= "$field_name = map.$map_key";
}
$alias = $this->query->leftJoin($this->activeMap->getQualifiedMapTable(),
'map', $map_join);
$conditions->isNull($alias . '.sourceid1');
$conditions->condition($alias . '.needs_update', MigrateMap::STATUS_NEEDS_UPDATE);
$condition_added = TRUE;
// And as long as we have the map table, add its data to the row.
$count = 1;
foreach ($this->activeMap->getSourceKey() as $field_name => $field_schema) {
$map_key = 'sourceid' . $count++;
$this->query->addField($alias, $map_key, "migrate_map_$map_key");
}
$count = 1;
foreach ($this->activeMap->getDestinationKey() as $field_name => $field_schema) {
$map_key = 'destid' . $count++;
$this->query->addField($alias, $map_key, "migrate_map_$map_key");
}
$this->query->addField($alias, 'needs_update', 'migrate_map_needs_update');
}
// 3. If we are using highwater marks, also include rows above the mark.
if (isset($this->highwaterField['name'])) {
if (isset($this->highwaterField['alias'])) {
$highwater = $this->highwaterField['alias'] . '.' . $this->highwaterField['name'];
}
else {
$highwater = $this->highwaterField['name'];
}
$conditions->condition($highwater, $this->activeMigration->getHighwater(), '>');
$condition_added = TRUE;
}
if ($condition_added) {
$this->query->condition($conditions);
}
}
migrate_instrument_start('MigrateSourceSQL execute');
$this->result = $this->query->execute();
migrate_instrument_stop('MigrateSourceSQL execute');
}
/**
* Implementation of MigrateSource::getNextRow().
*
* @return object
*/
public function getNextRow() {
return $this->result->fetchObject();
}
}

622
sites/all/modules/migrate/plugins/sources/sqlmap.inc Normal file
View File

@@ -0,0 +1,622 @@
<?php
/**
* @file
* Defines a Drupal db-based implementation of MigrateMap.
*/
class MigrateSQLMap extends MigrateMap {
/**
* Names of tables created for tracking the migration.
*
* @var string
*/
protected $mapTable, $messageTable;
public function getMapTable() {
return $this->mapTable;
}
public function getMessageTable() {
return $this->messageTable;
}
/**
* Qualifying the map table name with the database name makes cross-db joins
* possible. Note that, because prefixes are applied after we do this (i.e.,
* it will prefix the string we return), we do not qualify the table if it has
* a prefix. This will work fine when the source data is in the default
* (prefixed) database (in particular, for simpletest), but not if the primary
* query is in an external database.
*
* @return string
*/
public function getQualifiedMapTable() {
$options = $this->connection->getConnectionOptions();
$prefix = $this->connection->tablePrefix($this->mapTable);
if ($prefix) {
return $this->mapTable;
}
else {
return $options['database'] . '.' . $this->mapTable;
}
}
/**
* sourceKey and destinationKey arrays are keyed by the field names; values
* are the Drupal schema definition for the field.
*
* @var array
*/
public function getSourceKey() {
return $this->sourceKey;
}
public function getDestinationKey() {
return $this->destinationKey;
}
/**
* Drupal connection object on which to create the map/message tables
* @var DatabaseConnection
*/
protected $connection;
public function getConnection() {
return $this->connection;
}
/**
* We don't need to check the tables more than once per request.
*
* @var boolean
*/
protected $ensured;
public function __construct($machine_name, array $source_key,
array $destination_key, $connection_key = 'default') {
// Default generated table names, limited to 63 characters
$this->mapTable = 'migrate_map_' . drupal_strtolower($machine_name);
$this->mapTable = drupal_substr($this->mapTable, 0, 63);
$this->messageTable = 'migrate_message_' . drupal_strtolower($machine_name);
$this->messageTable = drupal_substr($this->messageTable, 0, 63);
$this->sourceKey = $source_key;
$this->destinationKey = $destination_key;
$this->connection = Database::getConnection('default', $connection_key);
// Build the source and destination key maps
$this->sourceKeyMap = array();
$count = 1;
foreach ($source_key as $field => $schema) {
$this->sourceKeyMap[$field] = 'sourceid' . $count++;
}
$this->destinationKeyMap = array();
$count = 1;
foreach ($destination_key as $field => $schema) {
$this->destinationKeyMap[$field] = 'destid' . $count++;
}
$this->ensureTables();
}
/**
* Create the map and message tables if they don't already exist.
*/
protected function ensureTables() {
if (!$this->ensured) {
if (!$this->connection->schema()->tableExists($this->mapTable)) {
// Generate appropriate schema info for the map and message tables,
// and map from the source field names to the map/msg field names
$count = 1;
$source_key_schema = array();
$pks = array();
foreach ($this->sourceKey as $field_schema) {
$mapkey = 'sourceid' . $count++;
$source_key_schema[$mapkey] = $field_schema;
$pks[] = $mapkey;
}
$fields = $source_key_schema;
// Add destination keys to map table
// TODO: How do we discover the destination schema?
$count = 1;
foreach ($this->destinationKey as $field_schema) {
// Allow dest key fields to be NULL (for IGNORED/FAILED cases)
$field_schema['not null'] = FALSE;
$mapkey = 'destid' . $count++;
$fields[$mapkey] = $field_schema;
}
$fields['needs_update'] = array(
'type' => 'int',
'size' => 'tiny',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => MigrateMap::STATUS_IMPORTED,
'description' => 'Indicates current status of the source row',
);
$fields['last_imported'] = array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => 0,
'description' => 'UNIX timestamp of the last time this row was imported',
);
$schema = array(
'description' => t('Mappings from source key to destination key'),
'fields' => $fields,
'primary key' => $pks,
);
$this->connection->schema()->createTable($this->mapTable, $schema);
// Now for the message table
$fields = array();
$fields['msgid'] = array(
'type' => 'serial',
'unsigned' => TRUE,
'not null' => TRUE,
);
$fields += $source_key_schema;
$fields['level'] = array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => 1,
);
$fields['message'] = array(
'type' => 'text',
'size' => 'medium',
'not null' => TRUE,
);
$schema = array(
'description' => t('Messages generated during a migration process'),
'fields' => $fields,
'primary key' => array('msgid'),
'indexes' => array('sourcekey' => $pks),
);
$this->connection->schema()->createTable($this->messageTable, $schema);
}
$this->ensured = TRUE;
}
}
/**
* Retrieve a row from the map table, given a source ID
*
* @param array $source_id
*/
public function getRowBySource(array $source_id) {
migrate_instrument_start('mapRowBySource');
$query = $this->connection->select($this->mapTable, 'map')
->fields('map');
foreach ($this->sourceKeyMap as $key_name) {
$query = $query->condition("map.$key_name", array_shift($source_id), '=');
}
$result = $query->execute();
migrate_instrument_stop('mapRowBySource');
return $result->fetchAssoc();
}
/**
* Retrieve a row from the map table, given a destination ID
*
* @param array $source_id
*/
public function getRowByDestination(array $destination_id) {
migrate_instrument_start('mapRowByDestination');
$query = $this->connection->select($this->mapTable, 'map')
->fields('map');
foreach ($this->destinationKeyMap as $key_name) {
$query = $query->condition("map.$key_name", array_shift($destination_id), '=');
}
$result = $query->execute();
migrate_instrument_stop('mapRowByDestination');
return $result->fetchAssoc();
}
/**
* Retrieve an array of map rows marked as needing update.
*
* @param int $count
* Maximum rows to return; defaults to 10,000
* @return array
* Array of map row objects with needs_update==1.
*/
public function getRowsNeedingUpdate($count) {
$rows = array();
$result = db_select($this->mapTable, 'map')
->fields('map')
->condition('needs_update', MigrateMap::STATUS_NEEDS_UPDATE)
->range(0, $count)
->execute();
foreach ($result as $row) {
$rows[] = $row;
}
return $rows;
}
/**
* Given a (possibly multi-field) destination key, return the (possibly multi-field)
* source key mapped to it.
*
* @param array $destination_id
* Array of destination key values.
* @return array
* Array of source key values, or NULL on failure.
*/
public function lookupSourceID(array $destination_id) {
migrate_instrument_start('lookupSourceID');
$query = $this->connection->select($this->mapTable, 'map')
->fields('map', $this->sourceKeyMap);
foreach ($this->destinationKeyMap as $key_name) {
$query = $query->condition("map.$key_name", array_shift($destination_id), '=');
}
$result = $query->execute();
$source_id = $result->fetchAssoc();
migrate_instrument_stop('lookupSourceID');
return $source_id;
}
/**
* Given a (possibly multi-field) source key, return the (possibly multi-field)
* destination key it is mapped to.
*
* @param array $source_id
* Array of source key values.
* @return array
* Array of destination key values, or NULL on failure.
*/
public function lookupDestinationID(array $source_id) {
migrate_instrument_start('lookupDestinationID');
$query = $this->connection->select($this->mapTable, 'map')
->fields('map', $this->destinationKeyMap);
foreach ($this->sourceKeyMap as $key_name) {
$query = $query->condition("map.$key_name", array_shift($source_id), '=');
}
$result = $query->execute();
$destination_id = $result->fetchAssoc();
migrate_instrument_stop('lookupDestinationID');
return $destination_id;
}
/**
* Called upon successful import of one record, we record a mapping from
* the source key to the destination key. Also may be called, setting the
* third parameter to NEEDS_UPDATE, to signal an existing record should be remigrated.
*
* @param stdClass $source_row
* The raw source data. We use the key map derived from the source object
* to get the source key values.
* @param array $dest_ids
* The destination key values.
* @param int $needs_update
* Status of the source row in the map. Defaults to STATUS_IMPORTED.
*/
public function saveIDMapping(stdClass $source_row, array $dest_ids, $needs_update = MigrateMap::STATUS_IMPORTED) {
migrate_instrument_start('saveIDMapping');
// Construct the source key
$keys = array();
foreach ($this->sourceKeyMap as $field_name => $key_name) {
// A NULL key value will fail.
if (is_null($source_row->$field_name)) {
Migration::displayMessage(t(
'Could not save to map table due to NULL value for key field !field',
array('!field' => $field_name)));
migrate_instrument_stop('saveIDMapping');
return;
}
$keys[$key_name] = $source_row->$field_name;
}
$fields = array('needs_update' => (int)$needs_update);
$count = 1;
foreach ($dest_ids as $dest_id) {
$fields['destid' . $count++] = $dest_id;
}
if ($this->trackLastImported) {
$fields['last_imported'] = time();
}
$this->connection->merge($this->mapTable)
->key($keys)
->fields($fields)
->execute();
migrate_instrument_stop('saveIDMapping');
}
/**
* Record a message in the migration's message table.
*
* @param array $source_key
* Source ID of the record in error
* @param string $message
* The message to record.
* @param int $level
* Optional message severity (defaults to MESSAGE_ERROR).
*/
public function saveMessage($source_key, $message, $level = Migration::MESSAGE_ERROR) {
// Source IDs as arguments
$count = 1;
if (is_array($source_key)) {
foreach ($source_key as $key_value) {
$fields['sourceid' . $count++] = $key_value;
// If any key value is empty, we can't save - print out and abort
if (empty($key_value)) {
print($message);
return;
}
}
$fields['level'] = $level;
$fields['message'] = $message;
$this->connection->insert($this->messageTable)
->fields($fields)
->execute();
}
else {
// TODO: What else can we do?
Migration::displayMessage($message);
}
}
/**
* Prepares this migration to run as an update - that is, in addition to
* unmigrated content (source records not in the map table) being imported,
* previously-migrated content will also be updated in place.
*/
public function prepareUpdate() {
$this->connection->update($this->mapTable)
->fields(array('needs_update' => MigrateMap::STATUS_NEEDS_UPDATE))
->execute();
}
/**
* Returns a count of records in the map table (i.e., the number of
* source records which have been processed for this migration).
*
* @return int
*/
public function processedCount() {
$query = $this->connection->select($this->mapTable);
$query->addExpression('COUNT(*)', 'count');
$count = $query->execute()->fetchField();
return $count;
}
/**
* Returns a count of imported records in the map table.
*
* @return int
*/
public function importedCount() {
$query = $this->connection->select($this->mapTable);
$query->addExpression('COUNT(*)', 'count');
$query->condition('needs_update', array(MigrateMap::STATUS_IMPORTED, MigrateMap::STATUS_NEEDS_UPDATE), 'IN');
$count = $query->execute()->fetchField();
return $count;
}
/**
* Returns a count of records which are marked as needing update.
*
* @return int
*/
public function updateCount() {
$query = $this->connection->select($this->mapTable);
$query->addExpression('COUNT(*)', 'count');
$query->condition('needs_update', MigrateMap::STATUS_NEEDS_UPDATE);
$count = $query->execute()->fetchField();
return $count;
}
/**
* Get the number of source records which failed to import.
*
* @return int
* Number of records errored out.
*/
public function errorCount() {
$query = $this->connection->select($this->mapTable);
$query->addExpression('COUNT(*)', 'count');
$query->condition('needs_update', MigrateMap::STATUS_FAILED);
$count = $query->execute()->fetchField();
return $count;
}
/**
* Get the number of messages saved.
*
* @return int
* Number of messages.
*/
public function messageCount() {
$query = $this->connection->select($this->messageTable);
$query->addExpression('COUNT(*)', 'count');
$count = $query->execute()->fetchField();
return $count;
}
/**
* Delete the map entry and any message table entries for the specified source row.
*
* @param array $source_key
*/
public function delete(array $source_key, $messages_only = FALSE) {
if (!$messages_only) {
$map_query = $this->connection->delete($this->mapTable);
}
$message_query = $this->connection->delete($this->messageTable);
$count = 1;
foreach ($source_key as $key_value) {
if (!$messages_only) {
$map_query->condition('sourceid' . $count, $key_value);
}
$message_query->condition('sourceid' . $count, $key_value);
$count++;
}
if (!$messages_only) {
$map_query->execute();
}
$message_query->execute();
}
/**
* Delete the map entry and any message table entries for the specified destination row.
*
* @param array $destination_key
*/
public function deleteDestination(array $destination_key) {
$map_query = $this->connection->delete($this->mapTable);
$message_query = $this->connection->delete($this->messageTable);
$source_key = $this->lookupSourceID($destination_key);
if (!empty($source_key)) {
$count = 1;
foreach ($destination_key as $key_value) {
$map_query->condition('destid' . $count, $key_value);
$count++;
}
$map_query->execute();
$count = 1;
foreach ($source_key as $key_value) {
$message_query->condition('sourceid' . $count, $key_value);
$count++;
}
$message_query->execute();
}
}
/**
* Set the specified row to be updated, if it exists.
*/
public function setUpdate(array $source_key) {
$query = $this->connection->update($this->mapTable)
->fields(array('needs_update' => MigrateMap::STATUS_NEEDS_UPDATE));
$count = 1;
foreach ($source_key as $key_value) {
$query->condition('sourceid' . $count++, $key_value);
}
$query->execute();
}
/**
* Delete all map and message table entries specified.
*
* @param array $source_keys
* Each array member is an array of key fields for one source row.
*/
public function deleteBulk(array $source_keys) {
// If we have a single-column key, we can shortcut it
if (count($this->sourceKey) == 1) {
$sourceids = array();
foreach ($source_keys as $source_key) {
$sourceids[] = $source_key;
}
$this->connection->delete($this->mapTable)
->condition('sourceid1', $sourceids, 'IN')
->execute();
$this->connection->delete($this->messageTable)
->condition('sourceid1', $sourceids, 'IN')
->execute();
}
else {
foreach ($source_keys as $source_key) {
$map_query = $this->connection->delete($this->mapTable);
$message_query = $this->connection->delete($this->messageTable);
$count = 1;
foreach ($source_key as $key_value) {
$map_query->condition('sourceid' . $count, $key_value);
$message_query->condition('sourceid' . $count++, $key_value);
}
$map_query->execute();
$message_query->execute();
}
}
}
/**
* Clear all messages from the message table.
*/
public function clearMessages() {
$this->connection->truncate($this->messageTable)
->execute();
}
/**
* Remove the associated map and message tables.
*/
public function destroy() {
$this->connection->schema()->dropTable($this->mapTable);
$this->connection->schema()->dropTable($this->messageTable);
}
protected $result = NULL;
protected $currentRow = NULL;
protected $currentKey = array();
public function getCurrentKey() {
return $this->currentKey;
}
/**
* Implementation of Iterator::rewind() - called before beginning a foreach loop.
* TODO: Support idlist, itemlimit
*/
public function rewind() {
$this->currentRow = NULL;
$fields = array();
foreach ($this->sourceKeyMap as $field) {
$fields[] = $field;
}
foreach ($this->destinationKeyMap as $field) {
$fields[] = $field;
}
/* TODO
if (isset($this->options['itemlimit'])) {
$query = $query->range(0, $this->options['itemlimit']);
}
*/
$this->result = $this->connection->select($this->mapTable, 'map')
->fields('map', $fields)
->execute();
$this->next();
}
/**
* Implementation of Iterator::current() - called when entering a loop
* iteration, returning the current row
*/
public function current() {
return $this->currentRow;
}
/**
* Implementation of Iterator::key - called when entering a loop iteration, returning
* the key of the current row. It must be a scalar - we will serialize
* to fulfill the requirement, but using getCurrentKey() is preferable.
*/
public function key() {
return serialize($this->currentKey);
}
/**
* Implementation of Iterator::next() - called at the bottom of the loop implicitly,
* as well as explicitly from rewind().
*/
public function next() {
$this->currentRow = $this->result->fetchObject();
$this->currentKey = array();
if (!is_object($this->currentRow)) {
$this->currentRow = NULL;
}
else {
foreach ($this->sourceKeyMap as $map_field) {
$this->currentKey[$map_field] = $this->currentRow->$map_field;
// Leave only destination fields
unset($this->currentRow->$map_field);
}
}
}
/**
* Implementation of Iterator::valid() - called at the top of the loop, returning
* TRUE to process the loop and FALSE to terminate it
*/
public function valid() {
// TODO: Check numProcessed against itemlimit
return !is_null($this->currentRow);
}
}

1067
sites/all/modules/migrate/plugins/sources/xml.inc Normal file
View File

File diff suppressed because it is too large Load Diff

29
sites/all/modules/migrate/tests/UNTESTED.txt Normal file
View File

@@ -0,0 +1,29 @@
The following test cases remain to be written:
Handlers:
All handlers
Rollback:
Various options
Import:
Various options
Various update scenarios
Sources:
Highwater support
Destinations:
All term fields - presence/absence, default values, boundary conditions, error
conditions
All user fields - ditto
All node fields - ditto
All comment fields - ditto
Path module support
TableCopy destination
Infrastructure:
Team stuff
The following are features we would like to test, but are (or seem) impractical:
Drush commands
SQL Server support

55
sites/all/modules/migrate/tests/import/options.test Normal file
View File

@@ -0,0 +1,55 @@
<?php
/**
* @file
* Tests for import options.
*/
/**
* Test node migration.
*/
class MigrateImportOptionsTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Import options',
'description' => 'Test the import options',
'group' => 'Migrate',
);
}
function setUp() {
parent::setUp('migrate_example');
}
function testItemLimitOption() {
$migration = Migration::getInstance('BeerTerm');
$limit = 1;
$options = array(
'limit' => array(
'unit' => 'item',
'value' => $limit,
),
);
// We use the timers to track how many times prepareRow() is called.
global $timers, $_migrate_track_timer;
$_migrate_track_timer = TRUE;
$result = $migration->processImport($options);
$this->verbose(print_r($timers, 1));
$successes = $migration->importedCount();
$this->verbose("Total successes: {$successes}");
$assertion = format_plural($limit, 'The migration successfully processed 1 item.',
'The migration successfully processed @count items.');
$this->assertEqual($limit, $successes, $assertion);
$prepare_row_count = $timers['BeerTermMigration prepareRow']['count'];
$this->verbose("prepareRow() count: {$prepare_row_count}");
$processed = $migration->processedCount();
$this->verbose("Total processed count: {$processed}");
$assertion = format_plural($processed, 'The migration executed processRow() on 1 item.',
'The migration executed processRow() on @count items.');
$this->assertEqual($prepare_row_count, $processed, $assertion);
}
}

157
sites/all/modules/migrate/tests/plugins/destinations/comment.test Normal file
View File

@@ -0,0 +1,157 @@
<?php
/**
* @file
* Tests for the comment destination plugin.
*/
/**
* Test comment migration.
*/
class MigrateCommentUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Comment migration',
'description' => 'Test migration of comment data',
'group' => 'Migrate',
);
}
function setUp() {
parent::setUp('taxonomy', 'image', 'comment', 'migrate', 'migrate_example');
}
function testCommentImport() {
$migration = Migration::getInstance('WineVariety');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Variety term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineRegion');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Region term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineBestWith');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('"Best With" term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineFileCopy');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('File import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineRole');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Role import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineProducer');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineWine');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineComment');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Comment import returned RESULT_COMPLETED'));
$result = db_select('migrate_message_winecomment', 'w')
->fields('w', array('message'))
->execute();
foreach ($result as $row) {
$this->error($row->message);
}
$result = db_select('migrate_example_wine_comment', 'wc')
->fields('wc', array('commentid', 'comment_parent', 'name', 'mail',
'accountid', 'body', 'wineid', 'subject', 'commenthost', 'userpage',
'posted', 'lastchanged'))
->orderBy('comment_parent')
->execute();
$rawcomments = comment_load_multiple(FALSE);
// Index by subject
$comments = array();
foreach ($rawcomments as $comment) {
$comments[$comment->subject] = $comment;
}
$rows = array();
foreach ($result as $row) {
$rows[$row->subject] = $row;
}
if (!$this->assertEqual(count($comments), count($rows), t('Counts of comments and input rows match'))) {
$this->error(t('!comments comments, should be !rows',
array('!comments' => count($comments), '!rows' => count($rows))));
}
$comment = $comments['im second'];
$row = $rows['im second'];
$this->assertEqual($comment->mail, $row->mail, t('Mail matches'));
$this->assertEqual($comment->name, $row->name, t('Name matches'));
$this->assertEqual($comment->status, COMMENT_PUBLISHED, t('Status matches'));
$wine_migration = MigrationBase::getInstance('WineWine');
$destid = $wine_migration->getMap()->lookupDestinationID(array($row->wineid));
$this->assertEqual($comment->nid, reset($destid), t('Nid matches'));
$body = field_get_items('comment', $comment, 'comment_body');
$this->assertEqual($body[0]['value'], $row->body, t('Body matches'));
$this->assertEqual($comment->hostname, $row->commenthost, t('Hostname matches'));
$this->assertEqual($comment->homepage, $row->userpage, t('Homepage matches'));
$this->assertEqual($comment->created, $row->posted, t('Created matches'));
$this->assertEqual($comment->changed, $row->lastchanged, t('Changed matches'));
$comment = $comments['im child'];
$row = $rows['im child'];
$user_migration = MigrationBase::getInstance('WineUser');
$destid = $user_migration->getMap()->lookupDestinationID(array($row->accountid));
$this->assertEqual($comment->uid, reset($destid), t('Uid matches'));
$this->assertEqual($comment->pid, $comments['im parent']->cid, t('Parent matches'));
// Test updates
// Capture original comments
$original_comments = comment_load_multiple(FALSE);
$update_migration = Migration::getInstance('WineCommentUpdates');
$result = $update_migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine comment updates import returned RESULT_COMPLETED'));
$final_comments = comment_load_multiple(FALSE);
foreach ($original_comments as $cid => $original_comment) {
foreach ($original_comment as $field => $value) {
if ($field == 'subject') {
if ($value == $final_comments[$cid]->$field) {
$this->error(t('Field !field should have changed but did not, value=!value',
array('!field' => $field, '!value' => print_r($value, TRUE))));
}
}
else {
if ($value != $final_comments[$cid]->$field) {
$this->error(t('Field !field mismatch: original !value1, result !value2',
array('!field' => $field, '!value1' => print_r($value, TRUE),
'!value2' => print_r($final_comments[$cid]->$field, TRUE))));
}
}
}
}
// Test rollback
$result = $migration->processRollback(array('force' => TRUE));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Comment rollback returned RESULT_COMPLETED'));
$rawcomments = comment_load_multiple(FALSE);
$this->assertEqual(count($rawcomments), 0, t('All comments deleted'));
$count = db_select('migrate_map_winecomment', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_winecomment', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
}
}

393
sites/all/modules/migrate/tests/plugins/destinations/node.test Normal file
View File

@@ -0,0 +1,393 @@
<?php
/**
* @file
* Tests for the node destination plugin.
*/
/**
* Test node migration.
*/
class MigrateNodeUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Node migration',
'description' => 'Test migration of node data',
'group' => 'Migrate',
);
}
function setUp() {
parent::setUp('list', 'number', 'taxonomy', 'image', 'migrate', 'migrate_example');
}
function testNodeImport() {
$migration = Migration::getInstance('WineVariety');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Variety term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineRegion');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Region term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineBestWith');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('"Best With" term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineFileCopy');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('File import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineRole');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Role import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineProducer');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineWine');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import returned RESULT_COMPLETED'));
// Gather wine and producer nodes, and their corresponding input data
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_producer'), TRUE);
// Index by title
$producer_nodes = array();
foreach ($rawnodes as $node) {
$producer_nodes[$node->title] = $node;
}
$query = db_select('migrate_example_wine_producer', 'p')
->fields('p', array('producerid', 'name', 'body', 'excerpt', 'accountid'));
// Region term is singletons, handled straighforwardly
$query->leftJoin('migrate_example_wine_category_producer', 'reg',
"p.producerid = reg.producerid");
$query->addField('reg', 'categoryid', 'region');
$result = $query->execute();
$producer_rows = array();
foreach ($result as $row) {
$producer_rows[$row->name] = $row;
}
$this->assertEqual(count($producer_nodes), count($producer_rows),
t('Counts of producer nodes and input rows match'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
// Index by title
$wine_nodes = array();
foreach ($rawnodes as $node) {
$wine_nodes[$node->title] = $node;
}
$query = db_select('migrate_example_wine', 'w')
->fields('w', array('wineid', 'name', 'body', 'excerpt', 'accountid',
'posted', 'last_changed', 'variety', 'region'));
$query->leftJoin('migrate_example_wine_category_wine', 'cwbw',
"w.wineid = cwbw.wineid");
$query->leftJoin('migrate_example_wine_categories', 'bw',
"cwbw.categoryid = bw.categoryid AND bw.type = 'best_with'");
// Gives a single comma-separated list of related terms
$query->groupBy('cwbw.wineid');
$query->addExpression('GROUP_CONCAT(bw.categoryid)', 'best_with');
$result = $query->execute();
$wine_rows = array();
foreach ($result as $row) {
$wine_rows[$row->name] = $row;
}
$this->assertEqual(count($wine_nodes), count($wine_rows),
t('Counts of wine nodes and input rows match'));
// Test each base node field
$producer_node = $producer_nodes['Montes'];
$producer_row = $producer_rows['Montes'];
$wine_node = $wine_nodes['Montes Classic Cabernet Sauvignon'];
$wine_row = $wine_rows['Montes Classic Cabernet Sauvignon'];
$user_migration = MigrationBase::getInstance('WineUser');
$mapped_uid = $user_migration->getMap()->lookupDestinationID(array($producer_row->accountid));
if (is_array($mapped_uid)) {
$this->assertEqual($producer_node->uid, reset($mapped_uid),
t('uid properly migrated'));
}
else {
$this->error(t('Account ID !id not migrated', array('!id' => $producer_row->accountid)));
}
$this->assertEqual($wine_node->created, $wine_row->posted,
t('created properly migrated'));
$this->assertEqual($wine_node->changed, $wine_row->last_changed,
t('changed properly migrated'));
// Test Field API fields of all types
// body_with_summary
$body = field_get_items('node', $wine_node, 'body');
$this->assertEqual($body[0]['value'], 'REVIEW: ' . drupal_strtoupper($wine_row->body),
t('body properly migrated'));
$this->assertEqual($body[0]['summary'], $wine_row->excerpt,
t('summary properly migrated'));
// taxonomy_term_reference - single and multiple
$variety = field_get_items('node', $wine_node, 'migrate_example_wine_varieties');
$variety_migration = MigrationBase::getInstance('WineVariety');
$mapped_tid = $variety_migration->getMap()->lookupDestinationID(array($wine_row->variety));
if (is_array($mapped_tid)) {
$this->assertEqual($variety[0]['tid'], reset($mapped_tid),
t('Single taxonomy_term_reference properly migrated'));
}
else {
$this->error(t('Variety !var not migrated', array('!var' => $wine_row->variety)));
}
$best_with = field_get_items('node', $wine_node, 'migrate_example_wine_best_with');
$best_with_migration = MigrationBase::getInstance('WineBestWith');
$source_ids = explode(',', $wine_row->best_with);
$mapped_tids = array();
foreach ($source_ids as $source_id) {
$tid = $best_with_migration->getMap()->lookupDestinationID(array($source_id));
if ($tid) {
$mapped_tids[reset($tid)] = reset($tid);
}
}
$this->assertEqual(count($best_with), count($mapped_tids),
t('Counts of Best With match'));
foreach ($best_with as $current) {
$this->assertNotNull($mapped_tids[$current['tid']],
t('Multiple value taxonomy_term_reference properly migrated'));
}
// Test the vintages field (demonstrating prepareRow() works) - we know
// the valid vintages for this node are 2006 and 2007
$expected = array(array('value' => 2006), array('value' => 2007));
$this->assertEqual($wine_node->field_migrate_example_top_vintag[LANGUAGE_NONE], $expected,
t('Vintages match (prepareRow works)'));
// Test updates
// Capture original nodes
$original_nodes = node_load_multiple(array(), array('type' => 'migrate_example_wine'));
$update_migration = Migration::getInstance('WineUpdates');
$result = $update_migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine updates import returned RESULT_COMPLETED'));
$final_nodes = node_load_multiple(array(), array('type' => 'migrate_example_wine'), TRUE);
foreach ($original_nodes as $nid => $original_node) {
foreach ($original_node as $field => $value) {
if ($field == 'field_migrate_example_wine_ratin' || $field == 'changed' || $field == 'revision_timestamp') {
if ($value == $final_nodes[$nid]->$field) {
$this->error(t('Field !field should have changed but did not, value=!value',
array('!field' => $field, '!value' => print_r($value, TRUE))));
}
}
else {
if ($value != $final_nodes[$nid]->$field) {
$this->error(t('Field !field mismatch: original !value1, result !value2',
array('!field' => $field, '!value1' => print_r($value, TRUE),
'!value2' => print_r($final_nodes[$nid]->$field, TRUE))));
}
}
}
}
// Test highwater marks - add new wines, modify an old one, and see what changes
$fields = array('wineid', 'name', 'body', 'excerpt', 'accountid',
'posted', 'last_changed', 'variety', 'region', 'rating');
$query = db_insert('migrate_example_wine')
->fields($fields);
$data = array(
// Timestamps 1284008523, 1284120550
array(3, 'Schloss Muhlenhof Dornfelder', 'Juicy black & red berry fruits', 'Pretty good', 9,
strtotime('2010-09-09 01:02:03'), strtotime('2010-09-10 08:09:10'), 25, 17, 95),
// Timestamps 1286122209, 1286122209
array(4, 'Gachot-Monot Bourgogne Rge 06', 'Funky', 'Pair with white sauced dishes', 3,
strtotime('2010-10-03 12:10:09'), strtotime('2010-10-03 12:10:09'), 26, 2, 85),
);
foreach ($data as $row) {
$query->values(array_combine($fields, $row));
}
$query->execute();
db_update('migrate_example_wine')
->fields(array(
'body' => 'Not so much berry character',
// Timestamp 1285058521
'last_changed' => strtotime('2010-10-21 04:42:01'),
))
->condition('wineid', 2)
->execute();
$migration = Migration::getInstance('WineWine');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
if (!$this->assertEqual(count($rawnodes), 4, t('Now 4 wine nodes exist'))) {
$this->error(t('There are now !count nodes', array('!count' => count($rawnodes))));
}
$nodes = node_load_multiple(FALSE, array('title' => 'Archeo Ruggero di Tasso Nero d\'Avola'), TRUE);
$node = reset($nodes);
$body = $node->body[LANGUAGE_NONE][0]['value'];
if (!$this->assertEqual($body, 'REVIEW: NOT SO MUCH BERRY CHARACTER', t('Body updated'))) {
$this->error(t('Actual body !body', array('!body' => $body)));
}
// Test rollback
$result = $migration->processRollback(array('force' => TRUE));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node rollback returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
$this->assertEqual(count($rawnodes), 0, t('All nodes deleted'));
$count = db_select('migrate_map_winewine', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_winewine', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
// Now test highwater with unjoined map table
$migration->getSource()->setMapJoinable(FALSE);
$result = $migration->processImport(array('limit' =>
array('value' => 2, 'unit' => 'items')));
db_update('migrate_example_wine')
->fields(array(
'body' => 'Very berry',
// Timestamp 1286008921
'last_changed' => strtotime('2010-10-02 04:42:01'),
))
->condition('wineid', 1)
->execute();
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
if (!$this->assertEqual(count($rawnodes), 4, t('Now 4 wine nodes exist'))) {
$this->error(t('There are now !count nodes', array('!count' => count($rawnodes))));
}
$nodes = node_load_multiple(FALSE, array('title' => 'Montes Classic Cabernet Sauvignon'), TRUE);
$node = reset($nodes);
$body = $node->body[LANGUAGE_NONE][0]['value'];
if (!$this->assertEqual($body, 'REVIEW: VERY BERRY', t('Body updated'))) {
$this->error(t('Actual body !body', array('!body' => $body)));
}
// Test itemlimit (joined map table)
$result = $migration->processRollback(array('force' => TRUE));
$migration->getSource()->setMapJoinable(TRUE);
$result = $migration->processImport(array('limit' =>
array('value' => 1, 'unit' => 'item')));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import with itemlimit returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
$this->assertEqual(count($rawnodes), 1, t('One node imported'));
// Test idlist (joined map table)
$result = $migration->processRollback(array('force' => TRUE));
$result = $migration->processImport(array('idlist' => 2));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import with idlist returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
$this->assertEqual(count($rawnodes), 1, t('One node imported'));
$node = reset($rawnodes);
$this->assertEqual($node->title, 'Archeo Ruggero di Tasso Nero d\'Avola',
t('Single specified node imported'));
// Test itemlimit (unjoined map table)
$result = $migration->processRollback(array('force' => TRUE));
$migration->getSource()->setMapJoinable(FALSE);
$result = $migration->processImport(array('limit' =>
array('value' => 1, 'unit' => 'item')));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import with itemlimit returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
$this->assertEqual(count($rawnodes), 1, t('One node imported'));
// Test idlist (unjoined map table)
$result = $migration->processRollback(array('force' => TRUE));
$result = $migration->processImport(array('idlist' => 2));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine node import with idlist returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_wine'), TRUE);
$this->assertEqual(count($rawnodes), 1, t('One node imported'));
$node = reset($rawnodes);
$this->assertEqual($node->title, 'Archeo Ruggero di Tasso Nero d\'Avola',
t('Single specified node imported'));
// Test integer highwater marks (http://drupal.org/node/1161612)
$result = $migration->processRollback(array('force' => TRUE));
db_update('migrate_example_wine')
->fields(array('last_changed' => 100000000))
->condition('wineid', 1)
->execute();
db_update('migrate_example_wine')
->fields(array('last_changed' => 200000000))
->condition('wineid', 2)
->execute();
db_update('migrate_example_wine')
->fields(array('last_changed' => 300000000))
->condition('wineid', 3)
->execute();
db_update('migrate_example_wine')
->fields(array('last_changed' => 400000000))
->condition('wineid', 4)
->execute();
$result = $migration->processImport();
// Just a quick check to make sure we got four nodes with the right changed values
$count = db_query("SELECT COUNT(nid)
FROM {node} n
INNER JOIN {migrate_map_winewine} map ON n.nid=map.destid1
WHERE n.changed = map.sourceid1*100000000")->fetchField();
$this->assertEqual($count, 4, t('Four nodes with updated changed values imported'));
// We mark two nodes with higher updated values. If these end up being treated
// as strings in saveHighwater(), the saved highwater mark will end up as
// 500000000 instead of 1000000000.
db_update('migrate_example_wine')
->fields(array('last_changed' => 1000000000))
->condition('wineid', 2)
->execute();
db_update('migrate_example_wine')
->fields(array('last_changed' => 500000000))
->condition('wineid', 3)
->execute();
$result = $migration->processImport();
$newHighwater = db_select('migrate_status', 'ms')
->fields('ms', array('highwater'))
->condition('machine_name', 'WineWine')
->execute()
->fetchField();
if (!$this->assertEqual($newHighwater, 1000000000, t('Correct highwater mark set'))) {
$this->error(t('Unexpected highwater mark !highwater', array('!highwater' => $newHighwater)));
}
// Test for http://drupal.org/node/1037872 - updating with nid mapped and idlist
$migration = Migration::getInstance('BeerTerm');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Beer term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('BeerUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Beer user import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('BeerNode');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Beer node import returned RESULT_COMPLETED'));
db_update('migrate_map_beernode')
->fields(array('needs_update' => 1))
->execute();
$result = $migration->processImport(array('idlist' => 99999999));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Beer node update import returned RESULT_COMPLETED'));
$result = db_select('migrate_message_beernode', 'msg')
->fields('msg', array('message'))
->execute();
foreach ($result as $row) {
$this->error($row->message);
}
}
}

49
sites/all/modules/migrate/tests/plugins/destinations/table.test Normal file
View File

@@ -0,0 +1,49 @@
<?php
/**
* @file
* Tests for the table destination plugin.
*/
/**
* Test table migration.
*/
class MigrateTableUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Table migration',
'description' => 'Test migration of table data',
'group' => 'Migrate',
);
}
function setUp() {
parent::setUp('migrate', 'migrate_example');
}
function testTableImport() {
$migration = Migration::getInstance('WineTable');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Table import returned RESULT_COMPLETED'));
$result = db_query(
"SELECT COUNT(*)
FROM {migrate_example_wine_table_source} s
INNER JOIN {migrate_map_winetable} map ON s.fooid=map.sourceid1
INNER JOIN {migrate_example_wine_table_dest} d ON map.destid1=d.recordid"
);
$this->assertEqual($result->fetchField(), 3,
t('Count of imported records is correct'));
// Test rollback
$result = $migration->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Variety term rollback returned RESULT_COMPLETED'));
$result = db_query("SELECT COUNT(*) FROM {migrate_example_wine_table_dest}");
$this->assertEqual($result->fetchField(), 0, t('All migrated rows removed'));
$result = db_query("SELECT COUNT(*) FROM {migrate_map_winetable}");
$this->assertEqual($result->fetchField(), 0, t('All map rows removed'));
}
}

113
sites/all/modules/migrate/tests/plugins/destinations/term.test Normal file
View File

@@ -0,0 +1,113 @@
<?php
/**
* @file
* Tests for the taxonomy term destination plugin.
*/
/**
* Test taxonomy migration.
*/
class MigrateTaxonomyUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Taxonomy migration',
'description' => 'Test migration of taxonomy data',
'group' => 'Migrate',
);
}
function setUp() {
parent::setUp('taxonomy', 'migrate', 'migrate_example');
}
function testTermImport() {
$migration = Migration::getInstance('WineVariety');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Variety term import returned RESULT_COMPLETED'));
$vocab = taxonomy_vocabulary_machine_name_load('migrate_example_wine_varieties');
$rawterms = taxonomy_term_load_multiple(array(), array('vid' => $vocab->vid));
$terms = array();
foreach ($rawterms as $term) {
$terms[$term->name] = $term;
}
$query = db_select('migrate_example_wine_categories', 'wc')
->fields('wc', array('categoryid', 'name', 'details', 'category_parent', 'ordering'))
->condition('wc.type', 'variety');
$query->leftJoin('migrate_example_wine_categories', 'wcpar',
'wc.category_parent=wcpar.categoryid');
$query->addField('wcpar', 'name', 'parent_name');
$result = $query->execute();
$rows = array();
foreach ($result as $row) {
$rows[$row->name] = $row;
}
$this->assertEqual(count($terms), count($rows), t('Counts of variety terms and input rows match'));
// Test each base term field
$this->assert(isset($terms['Merlot']) && isset($rows['Merlot']),
t("Name 'Merlot' migrated correctly"));
$this->assertEqual($terms['Merlot']->description, $rows['Merlot']->details,
t('Descriptions match'));
$this->assertEqual($terms['Merlot']->weight, $rows['Merlot']->ordering,
t('Weights match'));
$this->assertEqual($terms['Merlot']->format, $migration->basicFormat->format,
t('Formats match'));
$parents = taxonomy_get_parents($terms['White wine']->tid);
$this->assertEqual(count($parents), 0, t('Term without parent properly migrated'));
$parents = taxonomy_get_parents($terms['Merlot']->tid);
$parent = array_pop($parents);
$this->assertEqual($parent->name, 'Red wine', t('Parents match'));
// Test updates
// Capture original terms
$tempterms = taxonomy_term_load_multiple(array(), array('vid' => $vocab->vid));
foreach ($tempterms as $tid => $term) {
$original_terms[$tid] = clone $term;
}
$update_migration = Migration::getInstance('WineVarietyUpdates');
$result = $update_migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine variety term updates import returned RESULT_COMPLETED'));
$final_terms = taxonomy_term_load_multiple(array(), array('vid' => $vocab->vid));
foreach ($original_terms as $tid => $original_term) {
foreach ($original_term as $field => $value) {
if ($field == 'description') {
if ($value == $final_terms[$tid]->$field) {
$this->error(t('Field !field should have changed but did not, value=!value',
array('!field' => $field, '!value' => print_r($value, TRUE))));
}
}
else {
if ($value != $final_terms[$tid]->$field) {
$this->error(t('Field !field mismatch: original !value1, result !value2',
array('!field' => $field, '!value1' => print_r($value, TRUE),
'!value2' => print_r($final_terms[$tid]->$field, TRUE))));
}
}
}
}
// Test rollback
$result = $migration->processRollback(array('force' => TRUE));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Variety term rollback returned RESULT_COMPLETED'));
$rawterms = taxonomy_term_load_multiple(array(), array('vid' => $vocab->vid));
$this->assertEqual(count($rawterms), 0, t('All terms deleted'));
$count = db_select('migrate_map_winevariety', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_winevariety', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
}
}

208
sites/all/modules/migrate/tests/plugins/destinations/user.test Normal file
View File

@@ -0,0 +1,208 @@
<?php
/**
* @file
* Tests for the user destination plugin.
*/
/**
* Test user migration.
*/
class MigrateUserUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'User migration',
'description' => 'Test migration of user data',
'group' => 'Migrate',
);
}
function setUp() {
parent::setUp('migrate', 'migrate_example');
// To test timestamps
date_default_timezone_set('US/Mountain');
}
function testUserImport() {
$migration = Migration::getInstance('WineFileCopy');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('File import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineRole');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Role import returned RESULT_COMPLETED'));
// Confirm both roles were successfully imported
$result = db_select('role', 'r')
->fields('r', array('rid', 'name'))
->condition('name', array('Taster', 'Vintner'), 'IN')
->execute();
$roles = array();
foreach ($result as $row) {
$roles[$row->name] = $row->rid;
}
$this->assertEqual(count($roles), 2, t('Both roles imported'));
$migration = Migration::getInstance('WineUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$result = db_select('migrate_example_wine_account', 'mea')
->fields('mea', array('accountid', 'status', 'posted', 'name',
'sex', 'password', 'mail', 'last_access', 'last_login',
'sig', 'original_mail'))
->execute();
$uids = db_select('users', 'u')
->fields('u', array('uid'))
->execute()
->fetchCol();
// Index by name
$users = array();
foreach ($uids as $uid) {
// Skip anon/admin users
if ($uid > 1) {
$account = user_load($uid);
$users[$account->name] = $account;
}
}
$rows = array();
foreach ($result as $row) {
$rows[$row->name] = $row;
}
$this->assertEqual(count($users), count($rows),
t('Counts of users and input rows match'));
// Test each base user field
$this->assert(isset($users['darren']) && isset($rows['darren']),
t("Username 'darren' migrated correctly"));
$this->assertEqual($users['darren']->mail, $rows['darren']->mail,
t('Email addresses match'));
$this->assertEqual($users['darren']->status, $rows['darren']->status,
t('Statuses match'));
$this->assertNotNull($users['darren']->roles[2], t('Authenticated role'));
$this->assertNotNull($users['darren']->roles[$roles['Taster']], t('Taster role'));
$this->assertFalse(isset($users['darren']->roles[$roles['Vintner']]), t('No Vintner role'));
$this->assertEqual($users['darren']->created, strtotime($rows['darren']->posted),
t('Created times match'));
$this->assertEqual($users['darren']->access, strtotime($rows['darren']->last_access),
t('Access times match'));
$this->assertEqual($users['darren']->login, strtotime($rows['darren']->last_login),
t('Login times match'));
$this->assertTrue(user_check_password($rows['darren']->password, $users['darren']),
t('Passwords match'));
$this->assertEqual($users['darren']->init, $rows['darren']->original_mail,
t('Init mails match'));
$this->assertEqual($users['darren']->signature, $rows['darren']->sig,
t('Signatures match'));
$this->assertEqual($users['darren']->signature_format, $migration->basicFormat->format,
t('Signature formats match'));
$this->assertEqual($users['darren']->field_migrate_example_gender[LANGUAGE_NONE][0]['value'],
0, t('Male gender migrated'));
$this->assertEqual($users['emily']->field_migrate_example_gender[LANGUAGE_NONE][0]['value'],
1, t('Female gender migrated'));
$this->assert(!isset($users['fonzie']->field_migrate_example_gender[LANGUAGE_NONE][0]['value']),
t('Missing gender left unmigrated'));
$this->assert(is_object($users['fonzie']->picture) &&
$users['fonzie']->picture->filename == 'association-individual.png',
t('Picture migrated'));
$this->assertNotNull($users['fonzie']->roles[$roles['Taster']], t('Taster role'));
$this->assertNotNull($users['fonzie']->roles[$roles['Vintner']], t('Vintner role'));
// TODO: Theme, timezone, language
// Test updates
// Capture original users
$query = new EntityFieldQuery;
$result = $query
->entityCondition('entity_type', 'user')
->propertyCondition('uid', 1, '>')
->execute();
$uids = array_keys($result['user']);
$original_users = user_load_multiple($uids, array(), TRUE);
$update_migration = Migration::getInstance('WineUserUpdates');
$result = $update_migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Wine user updates import returned RESULT_COMPLETED'));
$final_users = user_load_multiple($uids, array(), TRUE);
foreach ($original_users as $uid => $original_user) {
foreach ($original_user as $field => $value) {
if ($field == 'field_migrate_example_gender') {
if ($value == $final_users[$uid]->$field) {
$this->error(t('For user !name, field !field should have changed but did not, value=!value',
array('!name' => $final_users[$uid]->name, '!field' => $field,
'!value' => print_r($value, TRUE))));
}
}
else {
if ($value != $final_users[$uid]->$field) {
// Core bug http://drupal.org/node/935592 causes picture mismatches, ignore until it's fixed
if ($field != 'picture') {
$this->error(t('For user !name, field !field mismatch: original !value1, result !value2',
array('!name' => $final_users[$uid]->name, '!field' => $field,
'!value1' => print_r($value, TRUE),
'!value2' => print_r($final_users[$uid]->$field, TRUE))));
}
}
}
}
}
// Test rollback
$result = $migration->processRollback(array('force' => TRUE));
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User rollback returned RESULT_COMPLETED'));
$count = db_select('users', 'u')
->fields('u', array('uid'))
->countQuery()
->execute()
->fetchField();
// 2 users left - anon and admin
$this->assertEqual($count, 2, t('All imported users deleted'));
$count = db_select('migrate_map_wineuser', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_wineuser', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
// Test deduping
// First, do the original import
$migration = Migration::getInstance('BeerUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$accounts = db_select('users', 'u')
->fields('u', array('mail', 'name'))
->execute()
->fetchAllKeyed();
if (!$this->assertEqual($accounts['alice@example.com'], 'alice', t('alice found'))) {
$this->error(t('Expected alice, found !name', array('!name' => $accounts['alice@example.com'])));
}
if (!$this->assertEqual($accounts['alice2@example.com'], 'alice_2', t('alice_2 found'))) {
$this->error(t('Expected alice_2, found !name', array('!name' => $accounts['alice2@example.com'])));
}
// Then, update in place and make sure the usernames did not change
$migration->prepareUpdate();
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$accounts = db_select('users', 'u')
->fields('u', array('mail', 'name'))
->execute()
->fetchAllKeyed();
if (!$this->assertEqual($accounts['alice@example.com'], 'alice', t('alice found'))) {
$this->error(t('Expected alice, found !name', array('!name' => $accounts['alice@example.com'])));
}
if (!$this->assertEqual($accounts['alice2@example.com'], 'alice_2', t('alice_2 found'))) {
$this->error(t('Expected alice_2, found !name', array('!name' => $accounts['alice2@example.com'])));
}
}
}

99
sites/all/modules/migrate/tests/plugins/sources/oracle.test Normal file
View File

@@ -0,0 +1,99 @@
<?php
/**
* @file
* Tests for the Oracle source plugin.
*/
/**
* Test migration from Oracle.
*
* NOTE: Test won't run correctly due to http://drupal.org/node/362373, enable
* when that is fixed.
*/
class MigrateOracleUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'Oracle migration',
'description' => 'Test migration from an Oracle source',
'group' => 'Migrate',
);
}
function setUp() {
global $conf;
if (empty($conf['oracle_db']) || empty($conf['oracle_db']['username']) ||
empty($conf['oracle_db']['password']) || empty($conf['oracle_db']['connection_string'])) {
parent::setUp();
}
else {
parent::setUp('features', 'migrate', 'migrate_example_oracle');
}
}
function testOracleImport() {
global $conf;
if (empty($conf['oracle_db'])) {
$this->pass(t('To run the Oracle test, you need to defined $conf[\'oracle_db\']
in settings.php - see migrate_example_oracle.migrate.inc.'));
}
$migration = Migration::getInstance('MigrateExampleOracle');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Region term import returned RESULT_COMPLETED'));
// Gather destination nodes, and their corresponding input data
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_oracle'), TRUE);
$data = migrate_example_oracle_sample_data();
$this->assertEqual(count($rawnodes), count($data), t('Counts of nodes and input rows match'));
// Index nodes by title
$nodes = array();
foreach ($rawnodes as $node) {
$nodes[$node->title] = $node;
}
// Test each value
foreach ($data as $row) {
$node = $nodes[$row['title']];
if (!$this->assertEqual($node->title, $row['title'], 'Titles match')) {
$this->error(t('Source title !source does not match node title !destination',
array('!source' => $row['title'], '!destination' => $node->title)));
}
if (!$this->assertEqual($node->body[LANGUAGE_NONE][0]['value'], $row['body'], 'Bodies match')) {
$this->error(t('Source body !source does not match node body !destination',
array('!source' => $row['body'], '!destination' => $node->body)));
}
$created = format_date($node->created, 'custom', 'Y/m/d H:i:s');
if (!$this->assertEqual($created, $row['created'], 'Created timestamps match')) {
$this->error(t('Source created !source does not match node created !destination',
array('!source' => $row['created'], '!destination' => $created)));
}
$updated = format_date($node->changed, 'custom', 'Y/m/d H:i:s');
if (!$this->assertEqual($updated, $row['updated'], 'Updated timestamps match')) {
$this->error(t('Source updated !source does not match node changed !destination',
array('!source' => $row['updated'], '!destination' => $updated)));
}
}
// Test rollback
$result = $migration->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Node rollback returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_oracle'), TRUE);
$this->assertEqual(count($rawnodes), 0, t('All nodes deleted'));
$count = db_select('migrate_map_migrateexampleoracle', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_migrateexampleoracle', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
}
}

100
sites/all/modules/migrate/tests/plugins/sources/xml.test Normal file
View File

@@ -0,0 +1,100 @@
<?php
/**
* @file
* Tests for the XML source plugins.
*/
/**
* Test node migration.
*/
class MigrateXMLUnitTest extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'XML migration',
'description' => 'Test migration from XML source',
'group' => 'Migrate',
);
}
function setUp() {
parent::setUp('taxonomy', 'migrate', 'migrate_example');
}
function testNodeImport() {
$migration = Migration::getInstance('WineRegion');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Region term import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineFileCopy');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('File import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineRole');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Role import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineUser');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('User import returned RESULT_COMPLETED'));
$migration = Migration::getInstance('WineProducerXML');
$result = $migration->processImport();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node import returned RESULT_COMPLETED'));
// Gather producer nodes, and their corresponding input data
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_producer'), TRUE);
// Index by title
$producer_nodes = array();
foreach ($rawnodes as $node) {
$producer_nodes[$node->title] = $node;
}
$this->assertEqual(count($producer_nodes), 1,
t('Counts of producer nodes and input rows match'));
// Test each base node field
$producer_node = $producer_nodes['Lolonis Winery'];
$user_migration = MigrationBase::getInstance('WineUser');
$mapped_uid = $user_migration->getMap()->lookupDestinationID(array(3));
if (is_array($mapped_uid)) {
$this->assertEqual($producer_node->uid, reset($mapped_uid),
t('uid properly migrated'));
}
else {
$this->error(t('Account ID !id not migrated', array('!id' => 3)));
}
// Test Field API fields of all types
// body_with_summary
$body = field_get_items('node', $producer_node, 'body');
$this->assertEqual($body[0]['value'], 'Makers of Ladybug Red',
t('body properly migrated'));
$region = field_get_items('node', $producer_node, 'migrate_example_wine_regions');
$term = taxonomy_get_term_by_name('Redwood Valley');
$term = reset($term);
$this->assertEqual($region[0]['tid'], $term->tid,
t('region properly migrated'));
// Test rollback
$result = $migration->processRollback();
$this->assertEqual($result, Migration::RESULT_COMPLETED,
t('Producer node rollback returned RESULT_COMPLETED'));
$rawnodes = node_load_multiple(FALSE, array('type' => 'migrate_example_producer'), TRUE);
$this->assertEqual(count($rawnodes), 0, t('All nodes deleted'));
$count = db_select('migrate_map_wineproducerxml', 'map')
->fields('map', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Map cleared'));
$count = db_select('migrate_message_wineproducerxml', 'msg')
->fields('msg', array('sourceid1'))
->countQuery()
->execute()
->fetchField();
$this->assertEqual($count, 0, t('Messages cleared'));
}
}

83
sites/all/modules/migrate/uri_map_redirect.php Normal file
View File

@@ -0,0 +1,83 @@
<?php
/**
* @file
* Sample file for handling redirection from old to new URIs. Use an Apache
* rewrite rule (or equivalent) to map legacy requests to this file. To use, copy
* or symlink this file to the root of your drupal site. Customize
* this file to your needs.
*
* CREATE TABLE `migrate_source_uri_map` (
* `source_uri` varchar(255) NOT NULL DEFAULT '',
* `migration_name` varchar(255) NOT NULL,
* `source_id` int(11) NOT NULL, -- can be varchar for some migrations
* PRIMARY KEY (`source_uri`)
* )
*
*/
// For security, this script is disabled by default.
die('Comment out this line when you are ready to use this script');
// Based on custom patterns, build the destination_uri for given source_uri
function migrate_build_url($destid1, $migration_name) {
global $base_url;
// TODO: Add an entry for each migration that we need to redirect.
$patterns = variable_get('migrate_patterns', array(
'BeerTerm' => 'taxonomy/term/:source_id',
'BlogEntries' => 'node/:source_id',
'Slideshows' => 'node/:source_id',
'TagTerm' => 'taxonomy/term/:source_id',
));
$pattern = $patterns[$migration_name];
// Swap in the destination ID.
$destination_uri = str_replace(':source_id', $destid1, $pattern);
// For speed, we go right to aliases table rather than more bootstrapping.
if ($uri_clean = db_query("SELECT alias FROM {url_alias} WHERE source = :destination_uri", array(':destination_uri' => $destination_uri))->fetchField()) {
$destination_uri = $uri_clean;
}
// Build absolute url for 301 redirect.
return $base_url . '/' . $destination_uri;
}
define('DRUPAL_ROOT', getcwd());
require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
// Only bootstrap to DB so we are as fast as possible. Much of the Drupal API
// is not available to us.
drupal_bootstrap(DRUPAL_BOOTSTRAP_DATABASE);
// You must populate this querystring param from a rewrite rule or $_SERVER
// On Apache, we could likely use _SERVER['REDIRECT_URL']. nginx?
if (!$source_uri = $_GET['migrate_source_uri']) {
print '$_GET[migrate_source_uri] was not found on the request.';
exit();
}
// This is a tall table mapping legacy URLs to source_id and migration_name.
// If you can already know the migration name and source_id based on the URI,
// then the first lookup is not needed.
$uri_table = variable_get('migrate_source_uri_table', 'migrate_source_uri_map');
if ($uri_map = db_query("SELECT migration_name, source_id FROM $uri_table WHERE source_uri = :source_uri", array(':source_uri' => $source_uri))->fetchObject()) {
// Hurray, we do recognize this URI.
// Consult migrate_map_x table to determine corresponding Drupal nid/tid/cid/etc.
$map_table = 'migrate_map_' . drupal_strtolower($uri_map->migration_name);
$sql = "SELECT destid1 FROM $map_table WHERE sourceid1 = :source_id";
if ($destid1 = $migrate_map = db_query($sql, array(':source_id' => $uri_map->source_id))->fetchField()) {
// Hurray. We already migrated this content. Go there.
header('Location: ' . migrate_build_url($destid1, $uri_map->migration_name), TRUE, 301);
}
else {
// We recognize URI but don't have the content in Drupal. Very unlikely.
}
}
else {
// Can't find the source URI. TODO: Make nice 404 page.
header('Status=Not Found', TRUE, 404);
print 'Sorry folks. Park is closed.';
}