Hardware, New Iteration
Here is my new NAS box:
- CPU: Intel Core 2 Duo 65nm E4300 Allendale 1.8GHz 800MHz FSB LGA 775 65W
- Motherboard: MSI G41M-P25 with Intel G41 video, Intel ICH7 south bridge with 4x SATA2 ports and Realtek 8111DL gigabit Ethernet controller.
- RAM: 2x4GB DDR3 1333mhz PC3-10666
- HBA: I am playing with HDs connected to motherboard’s ICH7. I also have an IBM M1015 (with cables) re-flashed to LSI9211-IT.
- HDs: mirror of 2xSeagate Barracuda Green 2TB ST2000DL003, assorted drives for tests, all in this 4-bay and this 3-bay rack. Fan controlled by this gadget.
- DVDROM: attached only for new installation, not enough external bays to have it full time.
- PSU: Antec EarthWatts Green EA-380D Green 380W
- Case: Cooler Master Gladiator 600
- UPS: APC BACK-UPS ES BE550G connected to USB port
Interestingly enough, this configuration refused to boot with both memory sticks installed. Overclocking from 200MHz to 266MHz solves the problem. I conclude that the memory is just too fast for such a slow CPU. As a result CPU runs at 2.3GHz. Not a big deal, given that SpeedStep throttles it down most of the time anyway. Setup ssh.
One of my clients is a Windows 7 HTPC:
- CPU: Intel Core 2 Duo 45nm E8500 Wolfdale 3.16Ghz 1333MHZ 6MB Cache 65W
- Motherboard: ASUS P5N-D LGA 775 NVIDIA nForce 750i SLI ATX with gigabit Ethernet controller
- RAM: 2x2GB, DDR2 800, PC2 6400, OCZ2RPR800C44GK
- Video: PowerColor ATI Radeon HD5770 512 MB DDR5 DVI/HDMI 512MD5-H
- HDs: Hitachi Deskstar 7200rpm, 3.5” 1TB SATA 3.0Gb/s, Seagate Barracuda XT 2 TB ST32000641AS
- BDROM: LG Black LG Blu-ray/HD DVD-ROM & 16X DVD±R DVD Burner SATA Model GGC-H20L - OEM
Another Windows 7 client is
- CPU: Intel Core i3-530 Clarkdale 2.93GHz 4MB L3 Cache LGA 1156 73W Dual-Core Desktop Processor BX80616I3530
- Motherboard: ASUS P7H55-M/CSM LGA 1156 Intel H55 HDMI Micro ATX with Realtek 8111E gigabit Ethernet controller.
- RAM: 2x2GB G.SKILL DDR3 1333 (PC3 10666) Model F3-10666CL9D-4GBNQ
- Video: PowerColor ATI Radeon HD5770 512 MB DDR5 DVI/HDMI 512MD5-H
- DVD RW/ROM: LITE-ON
- HD: Seagate Barracuda 7200.12 1TB ST31000528AS
I also have Intel PCI-E Gigabit NIC to play with. All computers are connected to this Gigabit switch.
Network Performance
My first attempts to figure out why performance is lacking were very confusing. Therefore I am testing separately performance of the network, disks, and file system.
For network performance testing I use iperf which is available for all OSs I am running. From HTPC:
C:\utils\iperf-2.0.5-cygwin>iperf.exe -c nas -f M
------------------------------------------------------------
Client connecting to nas, TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.104 port 49575 connected with 192.168.1.10 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 13.4 MBytes 1.33 MBytes/sec
C:\utils\iperf-2.0.5-cygwin>iperf.exe -c nas -f M -w 512k
------------------------------------------------------------
Client connecting to nas, TCP port 5001
TCP window size: 0.50 MByte
------------------------------------------------------------
[ 3] local 192.168.1.104 port 49579 connected with 192.168.1.10 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1052 MBytes 105 MBytes/sec
Notice a dramatic impact of the window size on the measured bandwidth. For UDP however larger window size does not help at all:
C:\utils\iperf-2.0.5-cygwin>iperf.exe -c nas -f M -w 256k -u
------------------------------------------------------------
Client connecting to nas, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 0.25 MByte
------------------------------------------------------------
[ 3] local 192.168.1.104 port 62782 connected with 192.168.1.10 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1.25 MBytes 0.13 MBytes/sec
[ 3] Sent 893 datagrams
[ 3] Server Report:
[ 3] 0.0-10.0 sec 1.25 MBytes 0.13 MBytes/sec 0.338 ms 0/ 892 (0%)
[ 3] 0.0-10.0 sec 1 datagrams received out-of-order
I then will use FTP and only then Samba for throughput measurements. On NAS side I will use
netstat
and
sysctl dev.re.0.stats=1
dev.re.0.stats: -1 -> -1
command, where re0 is the Realtek NIC. Start with direct cable connection, to avoid bad cable/switch issues. Techniques to consider: network polling, Windows-side optimization. More on Enabling High Performance Data Transfers, and FreeBSD Network Performance Tuning.
Disk Performance
Disk performance testing
using diskinfo -t.
nas:/mnt# camcontrol devlist
<SanDisk SDCFH-004G HDX 6.02> at scbus0 target 0 lun 0 (ada0,pass0)
<ST3320620AS 3.AAK> at scbus1 target 0 lun 0 (ada1,pass1)
<ST2000DL003-9VT166 CC3C> at scbus1 target 1 lun 0 (ada2,pass2)
<WDC WD5000AAKS-00V1A0 05.01D05> at scbus2 target 0 lun 0 (ada3,pass3)
<ST2000DL003-9VT166 CC3C> at scbus2 target 1 lun 0 (ada4,pass4)
nas:/mnt# diskinfo -t ada1
ada1
512 # sectorsize
320072933376 # mediasize in bytes (298G)
625142448 # mediasize in sectors
0 # stripesize
0 # stripeoffset
620181 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
9QF3BS42 # Disk ident.
Seek times:
Full stroke: 250 iter in 5.238228 sec = 20.953 msec
Half stroke: 250 iter in 3.867091 sec = 15.468 msec
Quarter stroke: 500 iter in 6.215048 sec = 12.430 msec
Short forward: 400 iter in 1.674251 sec = 4.186 msec
Short backward: 400 iter in 2.023333 sec = 5.058 msec
Seq outer: 2048 iter in 0.989333 sec = 0.483 msec
Seq inner: 2048 iter in 1.033952 sec = 0.505 msec
Transfer rates:
outside: 102400 kbytes in 1.774573 sec = 57704 kbytes/sec
middle: 102400 kbytes in 1.783387 sec = 57419 kbytes/sec
inside: 102400 kbytes in 2.684722 sec = 38142 kbytes/sec
nas:/mnt# diskinfo -t ada2
ada2
512 # sectorsize
2000398934016 # mediasize in bytes (1.8T)
3907029168 # mediasize in sectors
4096 # stripesize
0 # stripeoffset
3876021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
6YD20H52 # Disk ident.
Seek times:
Full stroke: 250 iter in 6.660599 sec = 26.642 msec
Half stroke: 250 iter in 5.055938 sec = 20.224 msec
Quarter stroke: 500 iter in 8.201362 sec = 16.403 msec
Short forward: 400 iter in 2.848986 sec = 7.122 msec
Short backward: 400 iter in 3.272430 sec = 8.181 msec
Seq outer: 2048 iter in 0.578140 sec = 0.282 msec
Seq inner: 2048 iter in 0.583157 sec = 0.285 msec
Transfer rates:
outside: 102400 kbytes in 1.093656 sec = 93631 kbytes/sec
middle: 102400 kbytes in 1.081835 sec = 94654 kbytes/sec
inside: 102400 kbytes in 1.656841 sec = 61804 kbytes/sec
nas:/mnt# diskinfo -t ada3
ada3
512 # sectorsize
500107862016 # mediasize in bytes (465G)
976773168 # mediasize in sectors
0 # stripesize
0 # stripeoffset
969021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
WD-WCAWF0558229 # Disk ident.
Seek times:
Full stroke: 250 iter in 6.727210 sec = 26.909 msec
Half stroke: 250 iter in 4.525683 sec = 18.103 msec
Quarter stroke: 500 iter in 6.967313 sec = 13.935 msec
Short forward: 400 iter in 2.097940 sec = 5.245 msec
Short backward: 400 iter in 1.366547 sec = 3.416 msec
Seq outer: 2048 iter in 0.699322 sec = 0.341 msec
Seq inner: 2048 iter in 0.693734 sec = 0.339 msec
Transfer rates:
outside: 102400 kbytes in 1.193988 sec = 85763 kbytes/sec
middle: 102400 kbytes in 1.071763 sec = 95544 kbytes/sec
inside: 102400 kbytes in 1.639349 sec = 62464 kbytes/sec
nas:/mnt# diskinfo -t ada4
ada4
512 # sectorsize
2000398934016 # mediasize in bytes (1.8T)
3907029168 # mediasize in sectors
4096 # stripesize
0 # stripeoffset
3876021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
6YD1Z7CA # Disk ident.
Seek times:
Full stroke: 250 iter in 6.561036 sec = 26.244 msec
Half stroke: 250 iter in 4.939224 sec = 19.757 msec
Quarter stroke: 500 iter in 8.066986 sec = 16.134 msec
Short forward: 400 iter in 3.188722 sec = 7.972 msec
Short backward: 400 iter in 3.768447 sec = 9.421 msec
Seq outer: 2048 iter in 0.565191 sec = 0.276 msec
Seq inner: 2048 iter in 0.601144 sec = 0.294 msec
Transfer rates:
outside: 102400 kbytes in 1.079444 sec = 94864 kbytes/sec
middle: 102400 kbytes in 1.081266 sec = 94704 kbytes/sec
inside: 102400 kbytes in 1.409226 sec = 72664 kbytes/sec
Alternatively create load
dd bs=1m of=/dev/null if=/dev/da6
and then use gstat
tags: hardware