19 September 2010

Mount and Automount Ext3 USB Drives on Western Digital My Book World 2ND Edition (White Light)

It seems somewhat silly that the WD My Book World 2ND Edition (MBWE II)supports USB drives that have NTFS and FAT filesystems but doesn't support the Linux EXT3 considering it runs Linux.

These instructions will let you mount and use USB drives with EXT3 partitions and I expect drives with other supported filesystems.

Firstly be careful! As with every tutorial I post only do this if you know what you are doing I take no responsibility you do it all at your own risk.

I actually expect that this will work for other filesystems that the NAS knows about but I don't have any drives with these formats to test it with, if you do try please post your results for others.


Manual Mount
Firstly you will need to SSH into the drive (you may need to turn it on using the admin web interface). There are lots of other sites that will tell you how to do this if you aren't familiar with this process.

If you aren't familiar using SSH you don't want to type the # it just indicates it is a command to enter, and everything is case sensitive. If you needed this help then you really need to be careful.

If you just want to mount the EXT3 drive as a one off it's easiest to do it manually, I've been doing this for some time now.

To mount the drive we need to find the device file associated with the drive, we can do this by running fdisk.
# fdisk -l
...
Disk /dev/sdb: 1024 MB, 1024966656 bytes
32 heads, 62 sectors/track, 1009 cylinders
Units = cylinders of 1984 * 512 = 1015808 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdb1               1         159      157697    b  Win95 FAT32
/dev/sdb2   *         160        1009      843200   83  Linux

The first set of entries (/dev/sda*) which I have omitted here are the WD's internal drive (if you have a dual drive version I'm assuming you will have a second drive listed here as well).

You should have a set of entries for your EXT3 drive. The above example is for a 1GB drive with two partitions, the first is a ~160MB fat drive and the second is a ~840MB EXT3 partition. So what I'm interested in is /dev/sdb2.

Next we create a folder to mount to, this can be anything but I've found the following nice because you can log into the copy manager on the web interface of the drive and use it to copy files between the drives (if you don't care about the copy manager /DataVolume/usb is probably a better place to make it).

# mkdir /shares/usb1-1share1

Then we mount the drive (replacing /dev/sdb2 with whatever device file fdisk showed your EXT3 partition to be listed as):

# mount -n -t ext3 /dev/sdb2 /shares/usb1-1share1

for other filesystems try:

# mount -n /dev/sdb2 /shares/usb1-1share1

You can then copy files to the mounted drive either using the web interface copy manager or from the console using either of the following commands:

# cp /DataVolume/sharename/file /shares/usb1-1share1/
# cp -r /DataVolume/sharename/folder /shares/usb1-1share1/



Note however that this method does not share the mounted filesystem though the standard mechanisms like CIFS. While it is possible to do this with this method it is just easier to automount the drive so if you want this read on.

Automount

The above worked fine for me for some time but I really wanted the drive to automount, and when I did a search around the web to find out how to do it, it seemed like I wasn't the only one to want the feature:
http://mybookworld.wikidot.com/forum/t-24171/
http://mybookworld.wikidot.com/forum/t-166644/need-help-sharing-ext3-usb-drive
http://mybookworld.wikidot.com/forum/t-247589
http://community.wdc.com/t5/My-Book-World-Edition/usb-ext3-drive-not-mounting/td-p/19414
http://community.wdc.com/t5/My-Book-World-Edition/MBWE-II-ext3-formatted-USB-storage/td-p/18643

I am currently not a member of all of these communities, and haven't bothered to list this on all these old threads, but if you have come here from one of these communities or another post about this it might be helpful if you would link this post.

This code patch is for Firmware Version 01.02.04 (currently the most recent firmware version) and should work with (Firmware Version 01.01.16).

First things first lets backup the file incase we make a mistake:
# cp /sbin/usb_auto_share_add.php /sbin/usb_auto_share_add.php.bkp

Should you need it you can restore the file using:
# cp /sbin/usb_auto_share_add.php.bkp /sbin/usb_auto_share_add.php

The only text editor on the NAS is currently vi, I like to recommend nano or pico to inexperienced users but unfortunately they aren't installed on this system. So I have developed an alternative for those who aren't familiar with vi.

Copy the file to a share you have, open the file over the network, edit it and then copy the file back. Here's how to do that, if your happy with vi just skip on a bit.

Find the share you want to copy to:
# ls /DataVolume/

Copy the file to the share using (substituting sharename with the case sesitive name you found using ls):
# cp /sbin/usb_auto_share_add.php /DataVolume/sharename/

Now open the file over the network using the method you normally would, make the edits outlined below and then save the file.

Once you've made the changes copy the file back to the folder using
# cp /DataVolume/sharename/usb_auto_share_add.php /sbin/

Here are the changes to make to the file. If your going to post this elsewhere a little credit that you found it here first might be nice :)

Open the file in vi:
# vi /sbin/usb_auto_share_add.php

Sorry for the layout of the following, I suggest that you copy it into something nicer before you read it (like notepad with Word Wrap turned off).

| lines from the original used for alignment.
o optional lines
+ lines to add.
- lines to remove


/sbin/usb_auto_share_add.php
|    // mount USB share for NTFS/HFS+ filesystem using ufsd
|    @system('/bin/mount -n -t ufsd -o gid=1000,umask=002,iocharset=utf8,force /dev/' . $USBDevInfo['devname'] . ' ' . $mountPoint, $retval);
|    if($retval != 0) {
|      // mount USB share with UTF8 option for FAT filesystem
|      @system('/bin/mount -n -o gid=1000,umask=002,iocharset=utf8 /dev/' . $USBDevInfo['devname'] . ' ' . $mountPoint, $retval);
|      $filesystem = 'fat';
|    }
+  
+    if($retval != 0) {
+      // Try mounting  USB share as ext3.
+      @system('/bin/mount -n -t ext3 /dev/' . $USBDevInfo['devname'] . ' ' . $mountPoint, $retval);
+      if($retval == 0) {
+        $filesystem = 'ext3';
+      }
+    }
+  
+    if($retval != 0) {
+      // Try mounting  USB share as any other file system known to the system.
+      @system('/bin/mount -n /dev/' . $USBDevInfo['devname'] . ' ' . $mountPoint, $retval);
+      if($retval == 0) {
+        $filesystem = ''; // or could use [= 'unknown';] // Note: Could find file system type with [mount] but see no reason to at this stage.
+      }
+    }
O  
O    // Debug Log
O    @system("/usr/bin/logger \"Mounted File System: $filesystem\" ");
O  
|    // mount USB share without UTF8 option for HFS+ filesystem
|    //if($retval != 0)
|    //  @system('/bin/mount -n -o gid=1000,umask=002 /dev/' . $USBDevInfo['devname'] . ' ' . $mountPoint, $retval);

=-=-=-=-=


|      // mount USB share for NTFS/HFS+ filesystem using ufsd
|      @system('/bin/mount -n -t ufsd -o gid=1000,umask=002,iocharset=utf8,force /dev/' . $USBDevInfo['partition'][$i] . ' ' . $mountPoint, $retval);
|      if($retval != 0) {
|        // mount USB share with UTF8 option for FAT filesystem
|        @system('/bin/mount -n -o gid=1000,umask=002,iocharset=utf8 /dev/' . $USBDevInfo['partition'][$i] . ' ' . $mountPoint, $retval);
|        $filesystem = 'fat';
|      }
+     
+      if($retval != 0) {
+        // Try mounting  USB share as ext3.
+        @system('/bin/mount -n -t ext3 /dev/' . $USBDevInfo['partition'][$i] . ' ' . $mountPoint, $retval);
+        if($retval == 0) {
+          $filesystem = 'ext3';
+        }
+      }
+     
+      if($retval != 0) {
+      // Try mounting  USB share as any other file system known to the system.
+        @system('/bin/mount -n /dev/' . $USBDevInfo['partition'][$i] . ' ' . $mountPoint, $retval);
+        if($retval == 0) {
+          $filesystem = ''; // or could use [= 'unknown';] // Note: Could find file system type with [mount] but see no reason to at this stage.
+        }
+      }
+     
O      // Debug Log
O      @system("/usr/bin/logger \"Mounted File System: $filesystem\" ");
O     
|      // mount USB share without UTF8 option for HFS+ filesystem
|      //if($retval != 0)
|      //  @system('/bin/mount -n -o gid=1000,umask=002 /dev/' . $USBDevInfo['partition'][$i] . ' ' . $mountPoint, $retval);
=====

Now you should be able to automount EXT3 drives and get them to show up as a shares like other filesystems on USB drives (according to your USB permsssions, see below):


It would have been nicer to get the filesystem type from mount but I haven't got around to that yet.
Now I didn't spend to long on this patch, and I havn't tested it with other file systems, but I believe it will also work with others supported by the NAS. I believe mount will try the filesystems listed in order in the following file:
# cat /etc/filesystems

You may be able to add more file systems to the list and even build and compile modules for file systems that aren't supported yet but that is well beond the scope of this article. I might add another post in future if I have the need, ext4 anyone???

I have identified a number of potential issues using EXT3, some which I have experienced and some that I hypothesised will create problems, if you find any more let me know and I'll list them here.
  • To access the share using normal methods you need to set the USB share permssions apropriately using the admin web interface (Advanced Mode->Users->USB Share Permissions).
  • Because EXT3 is a native Linux file system supporting permssions, the permissions of the files may cause problems. This will particularly be a problem if you have created the files (or permssions) on another Linux system and then connected the drive to the NAS or are doing the reverse. You should be able to fix this using commands like:
    # chmod 775 /shares/usb1-1share1/
    # chown root /shares/usb1-1share1/
    # chgrp jewab /shares/usb1-1share1/

I will try to get in contact with the developers and see if they will include this patch into the next firmware update. If I get any worthwhile feedback I'll let you know.

Update: I've tried to contact the developers (listed in the file) and the email address appears to be dead. If you know of a firmware developer that works on the MBWE II or want to contact support to have this included in future firmware updates please feel free I'm sure it will help out many others.

01 September 2010

S.M.A.R.T Western Digital My Book World 2ND Edition (White Light)

No matter how you look at it the hard drive in my computer is failing.




Fortunately I know "Data You Have Not Backed Up Is Data You Wouldn’t Mind Losing" so I'm not too worried and hopefully I'll be right when the drive does fail (I'm currently just trying to limp it through until I can replace it). But one of the drives I use is the Western Digital My Book World 2ND Edition (White Light) which due to its external nature I have no way of being warned about SMART errors. SMART warnings don't always necessarily correlate to an imminent HDD failure, but they are defiantly better then nothing.

So I set out to work out how to access the SMART information on the drive. Fortunately the developers had already included the smartmontools package on the drive so it was a quite easy process.

Firstly you will need to SSH into the drive (you may need to turn it on using the web interface).

Once your in all you need is the following command which will give you all the SMART information for the drive.

$ smartctl -a -d ata /dev/sda

Example:
# smartctl -a -d ata /dev/sda
smartctl version 5.38 [arm-unknown-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/


=== START OF INFORMATION SECTION ===
Device Model:     WDC WD10EAVS-00D7B1
Serial Number:    -- Removed --
Firmware Version: 01.01A01
User Capacity:    1,000,204,886,016 bytes
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Wed Sep  1 10:15:56 2010 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled


=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED


General SMART Values:
Offline data collection status:  (0x84)    Offline data collection activity
                    was suspended by an interrupting command from host.
                    Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever
                    been run.
Total time to complete Offline
data collection:          (22800) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      ( 255) minutes.
Conveyance self-test routine
recommended polling time:      (   5) minutes.
SCT capabilities:            (0x303f)    SCT Status supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.


SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   163   160   021    Pre-fail  Always       -       6825
  4 Start_Stop_Count        0x0032   098   098   000    Old_age   Always       -       2040
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   100   253   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   098   098   000    Old_age   Always       -       1747
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       757
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       4
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       2040
194 Temperature_Celsius     0x0022   119   096   000    Old_age   Always       -       31
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   100   253   000    Old_age   Offline      -       0


SMART Error Log Version: 1
No Errors Logged


SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]




SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.