Linux installed on USB disk, can data be lost during shutdown?












4














I was observing the use of the sync command and I have read that this command makes sure that anything in memory is written to disk.



As I was thinking of installing Ubuntu on a removable disk (USB) I would like to know if it is necessary to execute this command.



It worries me that when turning off the computer, if data are still being written, they end up losing because of the system shutdown.



Is it necessary to run sync before shutting down or restart the system?



Does anyone explain more about this?










share|improve this question






















  • I have been running Ubuntu from Flash drive for over 12 years and do not recall ever loosing any data with a Full install and letting the machine shut down before yanking the USB. ext4 is a journaling file system. Never tried sync.
    – C.S.Cameron
    9 hours ago


















4














I was observing the use of the sync command and I have read that this command makes sure that anything in memory is written to disk.



As I was thinking of installing Ubuntu on a removable disk (USB) I would like to know if it is necessary to execute this command.



It worries me that when turning off the computer, if data are still being written, they end up losing because of the system shutdown.



Is it necessary to run sync before shutting down or restart the system?



Does anyone explain more about this?










share|improve this question






















  • I have been running Ubuntu from Flash drive for over 12 years and do not recall ever loosing any data with a Full install and letting the machine shut down before yanking the USB. ext4 is a journaling file system. Never tried sync.
    – C.S.Cameron
    9 hours ago
















4












4








4


1





I was observing the use of the sync command and I have read that this command makes sure that anything in memory is written to disk.



As I was thinking of installing Ubuntu on a removable disk (USB) I would like to know if it is necessary to execute this command.



It worries me that when turning off the computer, if data are still being written, they end up losing because of the system shutdown.



Is it necessary to run sync before shutting down or restart the system?



Does anyone explain more about this?










share|improve this question













I was observing the use of the sync command and I have read that this command makes sure that anything in memory is written to disk.



As I was thinking of installing Ubuntu on a removable disk (USB) I would like to know if it is necessary to execute this command.



It worries me that when turning off the computer, if data are still being written, they end up losing because of the system shutdown.



Is it necessary to run sync before shutting down or restart the system?



Does anyone explain more about this?







usb shutdown sync






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked yesterday









MarianoMMarianoM

6410




6410












  • I have been running Ubuntu from Flash drive for over 12 years and do not recall ever loosing any data with a Full install and letting the machine shut down before yanking the USB. ext4 is a journaling file system. Never tried sync.
    – C.S.Cameron
    9 hours ago




















  • I have been running Ubuntu from Flash drive for over 12 years and do not recall ever loosing any data with a Full install and letting the machine shut down before yanking the USB. ext4 is a journaling file system. Never tried sync.
    – C.S.Cameron
    9 hours ago


















I have been running Ubuntu from Flash drive for over 12 years and do not recall ever loosing any data with a Full install and letting the machine shut down before yanking the USB. ext4 is a journaling file system. Never tried sync.
– C.S.Cameron
9 hours ago






I have been running Ubuntu from Flash drive for over 12 years and do not recall ever loosing any data with a Full install and letting the machine shut down before yanking the USB. ext4 is a journaling file system. Never tried sync.
– C.S.Cameron
9 hours ago












2 Answers
2






active

oldest

votes


















6














Data can be lost during unexpected shutdowns or system crashes - when the operating system cannot actually flush the data from memory to disk. During normal shutdown or restart the operating system is still in control, so it will write data to disk.



In other words, normal shutdowns/reboots are OK. There is also something know as halt state - where machine is still running, but OS has went through shutdown process and relinquished control of the hardware. In such case, powering off the machine is also OK. What's not OK is abnormal power offs and shutdowns.






share|improve this answer





















  • It would be a good idea to sync before unmounting and unplugging a USB drive.
    – heynnema
    yesterday



















0














The normal shutdown includes a sync at the end, after disks have been unmounted (or remounted read-only, in the case of the root file system). Normally, this should do nothing, as file systems are already synchronized on umount, so data should have been written during the "Unmounting file systems" stage.



Because network file systems can take arbitrarily long to unmount when the server is unreachable, certain init systems implement a timeout on umount, and because disks can not be unmounted if mountpoints inside them are still mounted, this can cause a cascading failure to unmount file systems cleanly, in which case there may still be unwritten data when the shutdown point is reached. The final sync then makes sure that no data is lost even if the file system is not cleanly unmounted. The file system check and/or journal replay (depending on file system) on the next boot should clean this up then.



The final sync runs at a time when (ideally) no read-write mounts exist, and no other program is still running, so no new write requests can be generated after this point. Zombie processes holding open file handles to deleted files should also have been cleaned up by this point.



Disks can implement their own caching, which is supposed to be fully transparent, but Linux generally sends a shutdown command to the drive nonetheless and waits for that to be reported as complete; drive firmware is generally smart enough to complete all pending writes before that.



For USB devices, SCSI commands are wrapped in USB packets, so the same sequence works for them, but because USB is a lot slower than SCSI, the kernel caches usually grow a lot bigger, so sync can take several minutes to complete here, but the kernel will dutifully wait.






share|improve this answer





















    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "89"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1107651%2flinux-installed-on-usb-disk-can-data-be-lost-during-shutdown%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    6














    Data can be lost during unexpected shutdowns or system crashes - when the operating system cannot actually flush the data from memory to disk. During normal shutdown or restart the operating system is still in control, so it will write data to disk.



    In other words, normal shutdowns/reboots are OK. There is also something know as halt state - where machine is still running, but OS has went through shutdown process and relinquished control of the hardware. In such case, powering off the machine is also OK. What's not OK is abnormal power offs and shutdowns.






    share|improve this answer





















    • It would be a good idea to sync before unmounting and unplugging a USB drive.
      – heynnema
      yesterday
















    6














    Data can be lost during unexpected shutdowns or system crashes - when the operating system cannot actually flush the data from memory to disk. During normal shutdown or restart the operating system is still in control, so it will write data to disk.



    In other words, normal shutdowns/reboots are OK. There is also something know as halt state - where machine is still running, but OS has went through shutdown process and relinquished control of the hardware. In such case, powering off the machine is also OK. What's not OK is abnormal power offs and shutdowns.






    share|improve this answer





















    • It would be a good idea to sync before unmounting and unplugging a USB drive.
      – heynnema
      yesterday














    6












    6








    6






    Data can be lost during unexpected shutdowns or system crashes - when the operating system cannot actually flush the data from memory to disk. During normal shutdown or restart the operating system is still in control, so it will write data to disk.



    In other words, normal shutdowns/reboots are OK. There is also something know as halt state - where machine is still running, but OS has went through shutdown process and relinquished control of the hardware. In such case, powering off the machine is also OK. What's not OK is abnormal power offs and shutdowns.






    share|improve this answer












    Data can be lost during unexpected shutdowns or system crashes - when the operating system cannot actually flush the data from memory to disk. During normal shutdown or restart the operating system is still in control, so it will write data to disk.



    In other words, normal shutdowns/reboots are OK. There is also something know as halt state - where machine is still running, but OS has went through shutdown process and relinquished control of the hardware. In such case, powering off the machine is also OK. What's not OK is abnormal power offs and shutdowns.







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered yesterday









    Sergiy KolodyazhnyySergiy Kolodyazhnyy

    70.1k9145307




    70.1k9145307












    • It would be a good idea to sync before unmounting and unplugging a USB drive.
      – heynnema
      yesterday


















    • It would be a good idea to sync before unmounting and unplugging a USB drive.
      – heynnema
      yesterday
















    It would be a good idea to sync before unmounting and unplugging a USB drive.
    – heynnema
    yesterday




    It would be a good idea to sync before unmounting and unplugging a USB drive.
    – heynnema
    yesterday













    0














    The normal shutdown includes a sync at the end, after disks have been unmounted (or remounted read-only, in the case of the root file system). Normally, this should do nothing, as file systems are already synchronized on umount, so data should have been written during the "Unmounting file systems" stage.



    Because network file systems can take arbitrarily long to unmount when the server is unreachable, certain init systems implement a timeout on umount, and because disks can not be unmounted if mountpoints inside them are still mounted, this can cause a cascading failure to unmount file systems cleanly, in which case there may still be unwritten data when the shutdown point is reached. The final sync then makes sure that no data is lost even if the file system is not cleanly unmounted. The file system check and/or journal replay (depending on file system) on the next boot should clean this up then.



    The final sync runs at a time when (ideally) no read-write mounts exist, and no other program is still running, so no new write requests can be generated after this point. Zombie processes holding open file handles to deleted files should also have been cleaned up by this point.



    Disks can implement their own caching, which is supposed to be fully transparent, but Linux generally sends a shutdown command to the drive nonetheless and waits for that to be reported as complete; drive firmware is generally smart enough to complete all pending writes before that.



    For USB devices, SCSI commands are wrapped in USB packets, so the same sequence works for them, but because USB is a lot slower than SCSI, the kernel caches usually grow a lot bigger, so sync can take several minutes to complete here, but the kernel will dutifully wait.






    share|improve this answer


























      0














      The normal shutdown includes a sync at the end, after disks have been unmounted (or remounted read-only, in the case of the root file system). Normally, this should do nothing, as file systems are already synchronized on umount, so data should have been written during the "Unmounting file systems" stage.



      Because network file systems can take arbitrarily long to unmount when the server is unreachable, certain init systems implement a timeout on umount, and because disks can not be unmounted if mountpoints inside them are still mounted, this can cause a cascading failure to unmount file systems cleanly, in which case there may still be unwritten data when the shutdown point is reached. The final sync then makes sure that no data is lost even if the file system is not cleanly unmounted. The file system check and/or journal replay (depending on file system) on the next boot should clean this up then.



      The final sync runs at a time when (ideally) no read-write mounts exist, and no other program is still running, so no new write requests can be generated after this point. Zombie processes holding open file handles to deleted files should also have been cleaned up by this point.



      Disks can implement their own caching, which is supposed to be fully transparent, but Linux generally sends a shutdown command to the drive nonetheless and waits for that to be reported as complete; drive firmware is generally smart enough to complete all pending writes before that.



      For USB devices, SCSI commands are wrapped in USB packets, so the same sequence works for them, but because USB is a lot slower than SCSI, the kernel caches usually grow a lot bigger, so sync can take several minutes to complete here, but the kernel will dutifully wait.






      share|improve this answer
























        0












        0








        0






        The normal shutdown includes a sync at the end, after disks have been unmounted (or remounted read-only, in the case of the root file system). Normally, this should do nothing, as file systems are already synchronized on umount, so data should have been written during the "Unmounting file systems" stage.



        Because network file systems can take arbitrarily long to unmount when the server is unreachable, certain init systems implement a timeout on umount, and because disks can not be unmounted if mountpoints inside them are still mounted, this can cause a cascading failure to unmount file systems cleanly, in which case there may still be unwritten data when the shutdown point is reached. The final sync then makes sure that no data is lost even if the file system is not cleanly unmounted. The file system check and/or journal replay (depending on file system) on the next boot should clean this up then.



        The final sync runs at a time when (ideally) no read-write mounts exist, and no other program is still running, so no new write requests can be generated after this point. Zombie processes holding open file handles to deleted files should also have been cleaned up by this point.



        Disks can implement their own caching, which is supposed to be fully transparent, but Linux generally sends a shutdown command to the drive nonetheless and waits for that to be reported as complete; drive firmware is generally smart enough to complete all pending writes before that.



        For USB devices, SCSI commands are wrapped in USB packets, so the same sequence works for them, but because USB is a lot slower than SCSI, the kernel caches usually grow a lot bigger, so sync can take several minutes to complete here, but the kernel will dutifully wait.






        share|improve this answer












        The normal shutdown includes a sync at the end, after disks have been unmounted (or remounted read-only, in the case of the root file system). Normally, this should do nothing, as file systems are already synchronized on umount, so data should have been written during the "Unmounting file systems" stage.



        Because network file systems can take arbitrarily long to unmount when the server is unreachable, certain init systems implement a timeout on umount, and because disks can not be unmounted if mountpoints inside them are still mounted, this can cause a cascading failure to unmount file systems cleanly, in which case there may still be unwritten data when the shutdown point is reached. The final sync then makes sure that no data is lost even if the file system is not cleanly unmounted. The file system check and/or journal replay (depending on file system) on the next boot should clean this up then.



        The final sync runs at a time when (ideally) no read-write mounts exist, and no other program is still running, so no new write requests can be generated after this point. Zombie processes holding open file handles to deleted files should also have been cleaned up by this point.



        Disks can implement their own caching, which is supposed to be fully transparent, but Linux generally sends a shutdown command to the drive nonetheless and waits for that to be reported as complete; drive firmware is generally smart enough to complete all pending writes before that.



        For USB devices, SCSI commands are wrapped in USB packets, so the same sequence works for them, but because USB is a lot slower than SCSI, the kernel caches usually grow a lot bigger, so sync can take several minutes to complete here, but the kernel will dutifully wait.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered yesterday









        Simon RichterSimon Richter

        1,956109




        1,956109






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Ask Ubuntu!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1107651%2flinux-installed-on-usb-disk-can-data-be-lost-during-shutdown%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            An IMO inspired problem

            Management

            Has there ever been an instance of an active nuclear power plant within or near a war zone?