Magento2 Data migration getting stuck at Step Customer Attribute
We are trying to migrate Magento 1.9.2.0 to Magento 2.0.4 EE. We have added the missing tables and columns in M2 Database and then started running the below command.
php bin/magento migrate:data vendor/magento/data-migration-tool/etc/ce-to-ee/1.9.2.0/config.xml.dist
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
EAV Step]: started [2016-04-13 13:30:05][INFO][mode: data][stage:
integrity check][step: Customer Attributes Step]: started [2016-04-13
13:30:05][INFO][mode: data][stage: integrity check][step: Map Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: integrity
check][step: Url Rewrite Step]: started [2016-04-13
13:30:05][INFO][mode: data][stage: integrity check][step: Log Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: integrity
check][step: Ratings Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: integrity check][step: ConfigurablePrices step]: started
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
OrderGrids Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: integrity check][step: Tier Price Step]: started
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
SalesIncrement Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: setup triggers][step: Stage]: started [2016-04-13
13:30:05][INFO][mode: data][stage: data migration][step: EAV Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: volume
check][step: EAV Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: data migration][step: Customer Attributes Step]: started
0% [>---------------------------] Remaining Time:
Please anyone can help.
magento2 magento2-migration-tool
bumped to the homepage by Community♦ yesterday
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
We are trying to migrate Magento 1.9.2.0 to Magento 2.0.4 EE. We have added the missing tables and columns in M2 Database and then started running the below command.
php bin/magento migrate:data vendor/magento/data-migration-tool/etc/ce-to-ee/1.9.2.0/config.xml.dist
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
EAV Step]: started [2016-04-13 13:30:05][INFO][mode: data][stage:
integrity check][step: Customer Attributes Step]: started [2016-04-13
13:30:05][INFO][mode: data][stage: integrity check][step: Map Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: integrity
check][step: Url Rewrite Step]: started [2016-04-13
13:30:05][INFO][mode: data][stage: integrity check][step: Log Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: integrity
check][step: Ratings Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: integrity check][step: ConfigurablePrices step]: started
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
OrderGrids Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: integrity check][step: Tier Price Step]: started
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
SalesIncrement Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: setup triggers][step: Stage]: started [2016-04-13
13:30:05][INFO][mode: data][stage: data migration][step: EAV Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: volume
check][step: EAV Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: data migration][step: Customer Attributes Step]: started
0% [>---------------------------] Remaining Time:
Please anyone can help.
magento2 magento2-migration-tool
bumped to the homepage by Community♦ yesterday
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
plz show error display in cmd so we can figure out it and also migrate log are generated those file errors are show to solve issue.
– Rakesh Jesadiya
Apr 14 '16 at 5:46
@Rakesh where can I see the logs generated..
– amit
Apr 14 '16 at 7:31
inside var/migration folder
– Rakesh Jesadiya
Apr 14 '16 at 7:35
We have add little code in Migration/Step/Customer/Data.php in perform function after this line"$destinationRecords->addRecord($destRecord);" and the code is if ($count >= 5) { $count = 0; break; }. After this we executed migration command it started migrating the customer.
– amit
Apr 14 '16 at 7:43
add a comment |
We are trying to migrate Magento 1.9.2.0 to Magento 2.0.4 EE. We have added the missing tables and columns in M2 Database and then started running the below command.
php bin/magento migrate:data vendor/magento/data-migration-tool/etc/ce-to-ee/1.9.2.0/config.xml.dist
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
EAV Step]: started [2016-04-13 13:30:05][INFO][mode: data][stage:
integrity check][step: Customer Attributes Step]: started [2016-04-13
13:30:05][INFO][mode: data][stage: integrity check][step: Map Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: integrity
check][step: Url Rewrite Step]: started [2016-04-13
13:30:05][INFO][mode: data][stage: integrity check][step: Log Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: integrity
check][step: Ratings Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: integrity check][step: ConfigurablePrices step]: started
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
OrderGrids Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: integrity check][step: Tier Price Step]: started
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
SalesIncrement Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: setup triggers][step: Stage]: started [2016-04-13
13:30:05][INFO][mode: data][stage: data migration][step: EAV Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: volume
check][step: EAV Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: data migration][step: Customer Attributes Step]: started
0% [>---------------------------] Remaining Time:
Please anyone can help.
magento2 magento2-migration-tool
We are trying to migrate Magento 1.9.2.0 to Magento 2.0.4 EE. We have added the missing tables and columns in M2 Database and then started running the below command.
php bin/magento migrate:data vendor/magento/data-migration-tool/etc/ce-to-ee/1.9.2.0/config.xml.dist
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
EAV Step]: started [2016-04-13 13:30:05][INFO][mode: data][stage:
integrity check][step: Customer Attributes Step]: started [2016-04-13
13:30:05][INFO][mode: data][stage: integrity check][step: Map Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: integrity
check][step: Url Rewrite Step]: started [2016-04-13
13:30:05][INFO][mode: data][stage: integrity check][step: Log Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: integrity
check][step: Ratings Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: integrity check][step: ConfigurablePrices step]: started
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
OrderGrids Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: integrity check][step: Tier Price Step]: started
[2016-04-13 13:30:05][INFO][mode: data][stage: integrity check][step:
SalesIncrement Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: setup triggers][step: Stage]: started [2016-04-13
13:30:05][INFO][mode: data][stage: data migration][step: EAV Step]:
started [2016-04-13 13:30:05][INFO][mode: data][stage: volume
check][step: EAV Step]: started [2016-04-13 13:30:05][INFO][mode:
data][stage: data migration][step: Customer Attributes Step]: started
0% [>---------------------------] Remaining Time:
Please anyone can help.
magento2 magento2-migration-tool
magento2 magento2-migration-tool
edited Apr 13 '16 at 13:32
Raphael at Digital Pianism
53.3k19110267
53.3k19110267
asked Apr 13 '16 at 13:32
amit
7017
7017
bumped to the homepage by Community♦ yesterday
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ yesterday
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
plz show error display in cmd so we can figure out it and also migrate log are generated those file errors are show to solve issue.
– Rakesh Jesadiya
Apr 14 '16 at 5:46
@Rakesh where can I see the logs generated..
– amit
Apr 14 '16 at 7:31
inside var/migration folder
– Rakesh Jesadiya
Apr 14 '16 at 7:35
We have add little code in Migration/Step/Customer/Data.php in perform function after this line"$destinationRecords->addRecord($destRecord);" and the code is if ($count >= 5) { $count = 0; break; }. After this we executed migration command it started migrating the customer.
– amit
Apr 14 '16 at 7:43
add a comment |
plz show error display in cmd so we can figure out it and also migrate log are generated those file errors are show to solve issue.
– Rakesh Jesadiya
Apr 14 '16 at 5:46
@Rakesh where can I see the logs generated..
– amit
Apr 14 '16 at 7:31
inside var/migration folder
– Rakesh Jesadiya
Apr 14 '16 at 7:35
We have add little code in Migration/Step/Customer/Data.php in perform function after this line"$destinationRecords->addRecord($destRecord);" and the code is if ($count >= 5) { $count = 0; break; }. After this we executed migration command it started migrating the customer.
– amit
Apr 14 '16 at 7:43
plz show error display in cmd so we can figure out it and also migrate log are generated those file errors are show to solve issue.
– Rakesh Jesadiya
Apr 14 '16 at 5:46
plz show error display in cmd so we can figure out it and also migrate log are generated those file errors are show to solve issue.
– Rakesh Jesadiya
Apr 14 '16 at 5:46
@Rakesh where can I see the logs generated..
– amit
Apr 14 '16 at 7:31
@Rakesh where can I see the logs generated..
– amit
Apr 14 '16 at 7:31
inside var/migration folder
– Rakesh Jesadiya
Apr 14 '16 at 7:35
inside var/migration folder
– Rakesh Jesadiya
Apr 14 '16 at 7:35
We have add little code in Migration/Step/Customer/Data.php in perform function after this line"$destinationRecords->addRecord($destRecord);" and the code is if ($count >= 5) { $count = 0; break; }. After this we executed migration command it started migrating the customer.
– amit
Apr 14 '16 at 7:43
We have add little code in Migration/Step/Customer/Data.php in perform function after this line"$destinationRecords->addRecord($destRecord);" and the code is if ($count >= 5) { $count = 0; break; }. After this we executed migration command it started migrating the customer.
– amit
Apr 14 '16 at 7:43
add a comment |
2 Answers
2
active
oldest
votes
Try disabling the customer hash in the config.xml file.
<upgrade_customer_password_hash>0</upgrade_customer_password_hash>
add a comment |
I know this is an old question, but if you're still having issues with Customer Attributes getting stuck at 0 or 16% (common numbers I've seen and had issues with myself), I've made a quick fix for it. The problem is if you have 25k+ customers in your Magento 1 instance. In our case, we have over 300,000, so it's just too much data for php and mysql to process all at once as it attempts to save. It seems to cap out for me around 50k and then stop itself from processing anything.
Version: 2.0.7 is what we are using currently of the migration tool, 2.0.8 of Mage2 (confirmed working with 2.0.7 by Enterprise Support)
EDIT - Official fix in pull request but not in new release yet here: https://github.com/magento-nord/data-migration-tool/pull/1/commits/6949dfe5ec425c78da162ca1003f0d377939c407
Edit: vendor/magento/data-migration-tool/src/Migration/Step/Customer/Data.php
~ Line 142
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
foreach ($bulk as $recordData) {
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
Make changes to this, and add in a count for blocks that your server/machine can handle. I did blocks of 1000 just for timeout sake.
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
$count = 0; // ++ Added within the while loop to continue processing
foreach ($bulk as $recordData) {
if($count > 1000) // ++
{ // ++
break; // ++
} // ++
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
$count++; // ++
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
What about the page number. Wouldn't this approach result in loss of data and implicit corrupted database? You can see that the page number counts when loading the records. Sure u break the cycle when your count reaches 1000 but the page increments even though not all records from that page were processed. (Hint the break). I believe that the page count effects which records are loaded and thus this will result in lost records.
– vitoriodachef
Jan 25 '18 at 13:52
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "479"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmagento.stackexchange.com%2fquestions%2f110752%2fmagento2-data-migration-getting-stuck-at-step-customer-attribute%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Try disabling the customer hash in the config.xml file.
<upgrade_customer_password_hash>0</upgrade_customer_password_hash>
add a comment |
Try disabling the customer hash in the config.xml file.
<upgrade_customer_password_hash>0</upgrade_customer_password_hash>
add a comment |
Try disabling the customer hash in the config.xml file.
<upgrade_customer_password_hash>0</upgrade_customer_password_hash>
Try disabling the customer hash in the config.xml file.
<upgrade_customer_password_hash>0</upgrade_customer_password_hash>
answered Aug 3 '16 at 20:21
MagentoMan
15819
15819
add a comment |
add a comment |
I know this is an old question, but if you're still having issues with Customer Attributes getting stuck at 0 or 16% (common numbers I've seen and had issues with myself), I've made a quick fix for it. The problem is if you have 25k+ customers in your Magento 1 instance. In our case, we have over 300,000, so it's just too much data for php and mysql to process all at once as it attempts to save. It seems to cap out for me around 50k and then stop itself from processing anything.
Version: 2.0.7 is what we are using currently of the migration tool, 2.0.8 of Mage2 (confirmed working with 2.0.7 by Enterprise Support)
EDIT - Official fix in pull request but not in new release yet here: https://github.com/magento-nord/data-migration-tool/pull/1/commits/6949dfe5ec425c78da162ca1003f0d377939c407
Edit: vendor/magento/data-migration-tool/src/Migration/Step/Customer/Data.php
~ Line 142
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
foreach ($bulk as $recordData) {
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
Make changes to this, and add in a count for blocks that your server/machine can handle. I did blocks of 1000 just for timeout sake.
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
$count = 0; // ++ Added within the while loop to continue processing
foreach ($bulk as $recordData) {
if($count > 1000) // ++
{ // ++
break; // ++
} // ++
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
$count++; // ++
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
What about the page number. Wouldn't this approach result in loss of data and implicit corrupted database? You can see that the page number counts when loading the records. Sure u break the cycle when your count reaches 1000 but the page increments even though not all records from that page were processed. (Hint the break). I believe that the page count effects which records are loaded and thus this will result in lost records.
– vitoriodachef
Jan 25 '18 at 13:52
add a comment |
I know this is an old question, but if you're still having issues with Customer Attributes getting stuck at 0 or 16% (common numbers I've seen and had issues with myself), I've made a quick fix for it. The problem is if you have 25k+ customers in your Magento 1 instance. In our case, we have over 300,000, so it's just too much data for php and mysql to process all at once as it attempts to save. It seems to cap out for me around 50k and then stop itself from processing anything.
Version: 2.0.7 is what we are using currently of the migration tool, 2.0.8 of Mage2 (confirmed working with 2.0.7 by Enterprise Support)
EDIT - Official fix in pull request but not in new release yet here: https://github.com/magento-nord/data-migration-tool/pull/1/commits/6949dfe5ec425c78da162ca1003f0d377939c407
Edit: vendor/magento/data-migration-tool/src/Migration/Step/Customer/Data.php
~ Line 142
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
foreach ($bulk as $recordData) {
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
Make changes to this, and add in a count for blocks that your server/machine can handle. I did blocks of 1000 just for timeout sake.
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
$count = 0; // ++ Added within the while loop to continue processing
foreach ($bulk as $recordData) {
if($count > 1000) // ++
{ // ++
break; // ++
} // ++
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
$count++; // ++
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
What about the page number. Wouldn't this approach result in loss of data and implicit corrupted database? You can see that the page number counts when loading the records. Sure u break the cycle when your count reaches 1000 but the page increments even though not all records from that page were processed. (Hint the break). I believe that the page count effects which records are loaded and thus this will result in lost records.
– vitoriodachef
Jan 25 '18 at 13:52
add a comment |
I know this is an old question, but if you're still having issues with Customer Attributes getting stuck at 0 or 16% (common numbers I've seen and had issues with myself), I've made a quick fix for it. The problem is if you have 25k+ customers in your Magento 1 instance. In our case, we have over 300,000, so it's just too much data for php and mysql to process all at once as it attempts to save. It seems to cap out for me around 50k and then stop itself from processing anything.
Version: 2.0.7 is what we are using currently of the migration tool, 2.0.8 of Mage2 (confirmed working with 2.0.7 by Enterprise Support)
EDIT - Official fix in pull request but not in new release yet here: https://github.com/magento-nord/data-migration-tool/pull/1/commits/6949dfe5ec425c78da162ca1003f0d377939c407
Edit: vendor/magento/data-migration-tool/src/Migration/Step/Customer/Data.php
~ Line 142
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
foreach ($bulk as $recordData) {
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
Make changes to this, and add in a count for blocks that your server/machine can handle. I did blocks of 1000 just for timeout sake.
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
$count = 0; // ++ Added within the while loop to continue processing
foreach ($bulk as $recordData) {
if($count > 1000) // ++
{ // ++
break; // ++
} // ++
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
$count++; // ++
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
I know this is an old question, but if you're still having issues with Customer Attributes getting stuck at 0 or 16% (common numbers I've seen and had issues with myself), I've made a quick fix for it. The problem is if you have 25k+ customers in your Magento 1 instance. In our case, we have over 300,000, so it's just too much data for php and mysql to process all at once as it attempts to save. It seems to cap out for me around 50k and then stop itself from processing anything.
Version: 2.0.7 is what we are using currently of the migration tool, 2.0.8 of Mage2 (confirmed working with 2.0.7 by Enterprise Support)
EDIT - Official fix in pull request but not in new release yet here: https://github.com/magento-nord/data-migration-tool/pull/1/commits/6949dfe5ec425c78da162ca1003f0d377939c407
Edit: vendor/magento/data-migration-tool/src/Migration/Step/Customer/Data.php
~ Line 142
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
foreach ($bulk as $recordData) {
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
Make changes to this, and add in a count for blocks that your server/machine can handle. I did blocks of 1000 just for timeout sake.
while (!empty($bulk = $this->source->getRecords($sourceDocName, $pageNumber))) {
$pageNumber++;
$destinationRecords = $destDocument->getRecords();
$count = 0; // ++ Added within the while loop to continue processing
foreach ($bulk as $recordData) {
if($count > 1000) // ++
{ // ++
break; // ++
} // ++
$this->source->setLastLoadedRecord($sourceDocName, $recordData);
if ($this->helper->isSkipRecord($attributeType, $sourceDocName, $recordData)) {
continue;
}
/** @var Record $record */
$record = $this->recordFactory->create(['document' => $sourceDocument, 'data' => $recordData]);
/** @var Record $destRecord */
$destRecord = $this->recordFactory->create(['document' => $destDocument]);
$recordTransformer->transform($record, $destRecord);
$destinationRecords->addRecord($destRecord);
$count++; // ++
}
$this->progress->advance(LogManager::LOG_LEVEL_INFO);
$this->progress->advance(LogManager::LOG_LEVEL_DEBUG);
$this->helper->updateAttributeData($attributeType, $sourceDocName, $destinationRecords);
$this->destination->saveRecords($destinationName, $destinationRecords);
}
edited Dec 21 '16 at 13:35
Manashvi Birla
6,09441840
6,09441840
answered Aug 10 '16 at 2:13
BtotheT
12
12
What about the page number. Wouldn't this approach result in loss of data and implicit corrupted database? You can see that the page number counts when loading the records. Sure u break the cycle when your count reaches 1000 but the page increments even though not all records from that page were processed. (Hint the break). I believe that the page count effects which records are loaded and thus this will result in lost records.
– vitoriodachef
Jan 25 '18 at 13:52
add a comment |
What about the page number. Wouldn't this approach result in loss of data and implicit corrupted database? You can see that the page number counts when loading the records. Sure u break the cycle when your count reaches 1000 but the page increments even though not all records from that page were processed. (Hint the break). I believe that the page count effects which records are loaded and thus this will result in lost records.
– vitoriodachef
Jan 25 '18 at 13:52
What about the page number. Wouldn't this approach result in loss of data and implicit corrupted database? You can see that the page number counts when loading the records. Sure u break the cycle when your count reaches 1000 but the page increments even though not all records from that page were processed. (Hint the break). I believe that the page count effects which records are loaded and thus this will result in lost records.
– vitoriodachef
Jan 25 '18 at 13:52
What about the page number. Wouldn't this approach result in loss of data and implicit corrupted database? You can see that the page number counts when loading the records. Sure u break the cycle when your count reaches 1000 but the page increments even though not all records from that page were processed. (Hint the break). I believe that the page count effects which records are loaded and thus this will result in lost records.
– vitoriodachef
Jan 25 '18 at 13:52
add a comment |
Thanks for contributing an answer to Magento Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmagento.stackexchange.com%2fquestions%2f110752%2fmagento2-data-migration-getting-stuck-at-step-customer-attribute%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
plz show error display in cmd so we can figure out it and also migrate log are generated those file errors are show to solve issue.
– Rakesh Jesadiya
Apr 14 '16 at 5:46
@Rakesh where can I see the logs generated..
– amit
Apr 14 '16 at 7:31
inside var/migration folder
– Rakesh Jesadiya
Apr 14 '16 at 7:35
We have add little code in Migration/Step/Customer/Data.php in perform function after this line"$destinationRecords->addRecord($destRecord);" and the code is if ($count >= 5) { $count = 0; break; }. After this we executed migration command it started migrating the customer.
– amit
Apr 14 '16 at 7:43