RitualClarity Posted January 19, 2014 Posted January 19, 2014 I decided months ago that next time I switched, I'd go with Intel. AMD has really dropped the ball in terms of performance over the past few years. This PC was built in 2009, so it is a little on the 'old' side. PCIe 2.0, USB 2.0, etc. The one thing that almost made me stick with AMD is the fact that many of their boards support up to 64G of memory, while the intel consumer chipset tops out at 32. Anyway, ordered: Intel Z87 Xeon E3-1270V3 4x8G kit I'm not a crazy (PC) modder/overclocker/etc anymore, and my requirements for a motherboard are a bit different than most gamers/enthusiasts; e.g. I want an intel NIC onboard, or at least not a realtek. Some have a qualcomm which I could give a chance, their wireless chips are great. If I could get one with no onboard raid, sound, or video I'd be all over it. LINK - manufacture Nice choice are you going to use the msata and cache features? A 256 msata isn't that expensive and can provide some boost or perhaps a scratch disk. AMD might support 64gig of memory but one of the AMD vendors told me that it would "downclock" the memory bandwidth with large blocks of memory. Intel doesn't do that as far as I know. Agree with the intel NIC. Nice and stable. this one comes with duel lan to boot. Even has a raid controller for software raid 10 if you so chose to do so. I know you have a very good raid card coming in the mail. Thunderbolt when you are ready. The external enclosures are way to expensive for what the give in my opinion.
prideslayer Posted January 19, 2014 Author Posted January 19, 2014 LINK - manufacture Nice choice are you going to use the msata and cache features? A 256 msata isn't that expensive and can provide some boost or perhaps a scratch disk. There's always a chance, but it won't be any time soon. To be honest, with 6 or 8 drives in the array, there's no point to tossing one in there for cache. As for scratch space.. the amount of disk space I have laying around at this point is mind boggling as it is. This machine alone has 4TB internal and 2.5TB external, and I have another 8TB or so available on the network. My scratch space needs are well satisfied AMD might support 64gig of memory but one of the AMD vendors told me that it would "downclock" the memory bandwidth with large blocks of memory. Intel doesn't do that as far as I know. Honestly, that would be fine. A big part of my job involves vmware. I have an ESXi server here in a rack for the 'important' stuff, and also run vmware workstation on my desktop to quickly test/prototype things. The more memory I can give to the VMs, the better. 32G will be good enough for now certainly, since I only have 16G in the machine as it is. Agree with the intel NIC. Nice and stable. this one comes with duel lan to boot. Even has a raid controller for software raid 10 if you so chose to do so. I know you have a very good raid card coming in the mail. Thunderbolt when you are ready. The external enclosures are way to expensive for what the give in my opinion. The video, dual lan, sound, firewire.. are all things I wish I could ditch from the board and save money on. Unfortunately, the market has decided otherwise. Out of all that stuff, thunderbolt is actually the one that sounds interesting, and I don't even think it will work; What I've read is that it's only available if you have a cpu with the iGPU built in, which the one I ordered does not. Hell everything might get here and I may have to send the CPU back. The feature sheet says any LGA1150 should work, but the manual download page for it is 404 and the "view all compatible intel processor numbers" link just runs some search on their site that turns up no results. The cool thing about the mini-pci isn't that it can do that mSATA thing -- but on most intel boards, they offer an addon card that can upgrade the fakeraid to realraid. That is something I might be interested in at some point.
prideslayer Posted January 20, 2014 Author Posted January 20, 2014 This one is well out of warranty, so why not. http://imgur.com/TVuC3M5,CImqy3Q,8ZwOpUE
pantherlux Posted January 20, 2014 Posted January 20, 2014 is this the manual you're looking for? http://downloadmirror.intel.com/22677/eng/DZ87KLT-75K_TechProdSpec02.pdf is this cleaning the drive taken too literally? ^^ edit: might be the compatible-list you need: http://processormatch.intel.com/Processors/CompatibleProcessors?componentName=DZ87KLT75K i hope the link works :/
Halstrom Posted January 20, 2014 Posted January 20, 2014 This one is well out of warranty, so why not. http://imgur.com/TVuC3M5,CImqy3Q,8ZwOpUE Perhaps just run the round disky bit through the dishwasher to fix it
RitualClarity Posted January 20, 2014 Posted January 20, 2014 This one is well out of warranty, so why not. http://imgur.com/TVuC3M5,CImqy3Q,8ZwOpUE is this the manual you're looking for? http://downloadmirror.intel.com/22677/eng/DZ87KLT-75K_TechProdSpec02.pdf is this cleaning the drive taken too literally? ^^ edit: might be the compatible-list you need: http://processormatch.intel.com/Processors/CompatibleProcessors?componentName=DZ87KLT75K i hope the link works :/ Perhaps just run the round disky bit through the dishwasher to fix it Here is a link while people are talking about hard drives. I found it interesting. Don't understand everything that is being talked about but it is pretty clear. Now I can imagine hundreds of HD's being destroyed across the world... muhaaaa.
prideslayer Posted January 20, 2014 Author Posted January 20, 2014 Thanks for the links panther, those look like the ones, and it seems everything should be OK. Not sure what I'm going to do with the drive parts.. make a bling necklace from the disks maybe, deploy some new superstrength kitchen magnets, and... not sure what else. I thought about stuffing an SDcard in there and making a poor mans SSD just for the hell of it, but that's probably too much work for something that would just be stupid. In other news, murphy ain't done with me yet. Up all night after my athome esxi server completely failed. Redundant 750w PSUs and both of them just sit there clicking on and off and making my basement smell like an electrical fire. Awesome. So, just finished repurposing my HTPC into a new esxi server. Only one NIC in that machine so I dug through junkbins looking for another and found three, none of which work, so.. VLAN time! I figure at this point, when I'm all squared away again, a sinkhole will open beneath the house and just swallow everything. The link goes to a video saying "sorry, this show has been removed from blip." From the intro text, it sounds *old* though. Cylinder boundaries... heh. Oh C/H/S how I miss you.
RitualClarity Posted January 20, 2014 Posted January 20, 2014 Sorry Pride about that link. Guess they dropped that video recently. Here is a youtube link page You could look through the web for the title but then click and click and search for all the sections. (reason for the above bad link it was one clip ) He also does some pod cast on SSD and solid state memory and such as well. Two power supplies failing at the same time.. Quick search for that dam gremlin that invaded your house. You are having a very bad stretch of bad luck indeed.
pantherlux Posted January 21, 2014 Posted January 21, 2014 glad i could help a little, even if its just "nothing" really big ^^ 2 psus in one time? man thats a thing, i only to managed to burn one in my whole time (wasnt even part of the pc, used to be a psu for testing fucked up tools like handdrills and so on) *placing a gremlin trap and waiting hidden in the basement* any one with me? got some tacos
prideslayer Posted January 21, 2014 Author Posted January 21, 2014 I don't think the PSUs actually both failed, that's just... so unlikely. More likely it's the PSU backplane (It's a rackmount server w/ PSU 'cartridges') or the motherboard that's toast. In any case, HTPC is doing that job fine for now, and new workstation parts are all slated to arrive today. So after work it's a PC (re-)building 'party', then on to deciding what to do about the server/htpc situation.
prideslayer Posted January 22, 2014 Author Posted January 22, 2014 All up and running at last! The intel bios doesn't like my USB keyboard so I had to find an old one to get in there with initially, but hopefully won't have to go back in any time soon. Of course, this just dropped today -- or yesterday technically. I'll just stick my fingers in my ears and scream LA LA LA LA.
Captain Cobra Posted January 22, 2014 Posted January 22, 2014 That blog post just reinforced every hard drive stereotype I believed in.
RitualClarity Posted January 22, 2014 Posted January 22, 2014 All up and running at last! The intel bios doesn't like my USB keyboard so I had to find an old one to get in there with initially, but hopefully won't have to go back in any time soon. Of course, this just dropped today -- or yesterday technically. I'll just stick my fingers in my ears and scream LA LA LA LA. I do something along the same lines as they do. I don't have a many drives of course but I use regular drives and have good backups. I don't care if a drive fails so much in regards to my data. Cost and time to create/restore the backups again (depending on if the backup or original dies) annoys me. I wouldn't worry about the drives you have to much. From what I understand from your post it is the rest of the hardware that was the main cause of your week in tech hell. My use of HD were the following: Seagate 100% (4 drives / <1year years) The LP version. They also didn't honor their warranty because it wasn't a smart failure. Because of that I doubt I will ever buy a Seagate again. Hitachi. 1 every two years or so. Acceptable because of the usage /price. I currently have several 2 tb being replaced for the 4tb versions as they die. WD (Black) none. I have had those and not one has died. I have one for more than 4 years. WD (Blue) None. I finally retired them due to size and age. (greater than 5 years old). Would have kept using them if they weren't so small (80gb) and slow.(Sata 1)
prideslayer Posted January 23, 2014 Author Posted January 23, 2014 I agree with your philosophy, which is why I run RAID-10 as well. Drive space is cheap, my time isn't. If a drive fails and I don't have a spare on hand, I can just order one and continue on, while ramping up my backup frequency from once a day to 3-4 times/day. Chances are the new drive will arrive and I'll be able to rebuild the array without ever having to restore from backup. With RAID-5 this just isn't an option; the performance hit for having to recalculate parity on every read is astonishing. I highly recommend everyone who is running RAID-5 or RAID-6 to intentionally fail a drive right now (after backing up of course, so you don't repeat my mistake!) and see if they can live with the performance. Then rebuild the array and see the performance suffer even further. You may well decide, as I did years ago, that it's got to be RAID-1 or RAID-10. My HDDs.. honestly, without thinking hard about it, I have ended up with almost all seagate drives here. I do prefer them over others, I have for a long time, but I don't exclude any brands when shopping -- they're just what I ended up with. So in house right now I have.. 6x 1T SV35.5 : One failed a few weeks ago, RMA on the way. 5x 500G 7200.4 : One failed, out of warranty, disassembled in pictures above. 4x 1T Constellation : Brand new, no failures thus far. 2x 500G barracuda 7200.12 : No failures. 1x External Seagate 2TB : I think this is just one drive inside, but have never opened it up. It has never had a problem. So that's 18 seagate drives with 15.5TB of space, with two failures. Both failures are 4-5 years old, and one is still in warranty and I should have the RMA sometime this week. I also have (had) 4x WD Black 1T's, one of which was DOA. That's the first time I've ever had a DOA drive. Otherwise I also have 4x 18GB IBM 18LZX's, and 5x 36GB Seagate Cheetahs; All 9 of these are still working as well and are closer to the 10 year old mark. Lastly my laptop, about 5 years old, has a problem free 500G Toshiba in it.
prideslayer Posted January 23, 2014 Author Posted January 23, 2014 Oh, my windows experience index is now 6.4. Anyone saying you can't get over 5.9 on a Win7 system without an SSD is lying!
RitualClarity Posted January 23, 2014 Posted January 23, 2014 I wouldn't exclude any manufacture either except after they refused to replace/repair the 4 drives that failed in less than a year because they were non-smart failures. They even told me to continue using them until they failed and they would be happy to replace them.. The report on the drive I wrote indicated that they caused massive dissruption of the computer. Freezing and stalling (likely due to a failed circurt boards, etc). How am I suppose to use them? Not to mention they took almost 3 months to respond back r/t this issue. I haven't tried other manufacture RMA yet. I am not even the least bit disturbed that one of the Hitachi's died prematurely. (I had a store warranty on them). All drives will die, They will die when it is the least convient for you.. That is the cosmic rule. I understand this and can live with it. I can't live with poor service. If they had just stopped at the fact that they would only replace smart failure drives I would have been upset but not as much as they made me when they continued on and told me to continue to use a drive that wasn't working (contaning data and corrupting anything that was placed on it and freezing and stalling the OS because the comptuer was waiting for the drive to respond.. ) I hear people praise Raid 5... dumb. It might be good for a NAS that you are running nightly backups on provided you run a UPS on is as well. Don't want corrupted data because of a power issue. But as a main data drive that is being accessed constantly .. it just sucks.. Sorry it does. It isn't as secure or stable as raid 10. (two *right* drives can die and still recover instead of only one drive) but it keeps chugging along regardless of what you are doing .. Recovery.. no problem I like raid 10. The only downside is the # drives / Space provided. That is where you *Pay* for the use of Raid 10. ZFS is probably better than raid 10 however the cost of setting it up with the equiptment, (professional level to get the best results), compatability with the hardware, and time "scrubbing" especially if you don't use "enterprise" drives. .. Makes it almost impossible to get the full features. It is said to be a "tank" and almost impossible to have corruption of data or loss of data with a properly set up ZFS server, short of the power supply blowing up and frying all the hard drives... nothing can withstand that... I was entertaining my next rig having raid 10 software set up...with this thread I think I am more confident in that decision.. Thanks Pride.
prideslayer Posted January 24, 2014 Author Posted January 24, 2014 When was that? My RMA experience with them (and WD as well) was stellar. I went to their website, punched in the model # and serial #, and *bam* -- instant RMA authorization. In raid10 half the array can fail. This generally means "two" because most consumers/endusers only use 4, but if you have 8 or 10 drives, then 4 or 5 can fail. People must be careful that they are using 10/1+0 and not 0+1. I've heard some of the newer arrays are smarter and will treat a 0+1 more like a 10, but the older ones functioned as implied by the nesting order. I'm playing around with ZFS. The biggest problem is that it needs tons of memory for good performance. Secondarily, it doesn't really give you much over enterprise (real enterprise, not just branded that way) hardware. ECC protects against occasional memory bitflips (usually) and running ZFS without it on a lot of data is unsafe (ZFS does not checksum its own internal memory structures), the disk protocols on the wire protect against bad cables, and the disks themselves checksum every read even though the RAID controller doesn't -- they have bigger sectors, and store ECC type data there "secretly". The features ZFS gives are outstanding, a lot more appeal for me comes from than than from the data protection.
RitualClarity Posted January 24, 2014 Posted January 24, 2014 When was that? About 3 years or so when 2tb hard drives were the new kid on the block. I had all 4 die almost at the same time. The wouldn't replace them because they weren't smart failures. They were board issues. I had even discussed it with a Seagate vendor and he checked and my drives weren't eligible. Like I stated if it isn't covered under the warranty due to some loophole not a problem. I understand. Perhaps they thought I handled them improperly or whatever. However commenting that I should continue to use them until the do fail.. That was just plain bad. It is the only experience I have had with hard drive vendors. I have never had a cause to use them before as either they were warrantied through a store or service or out of warranty when they failed. ECC protects against occasional memory bitflips (usually) and running ZFS without it on a lot of data is unsafe (ZFS does not checksum its own internal memory structures) I suspected as much however I couldn't find in data on that, at least any reliable data. The "fan boys" rarely write of the downsides other than the memory requirements. If I understood correctly and SSD can be used for Cache. That can help with the memory issue but with an hidden cost. (premature SSD failure) If memory serves me correctly 2 gigs for every terabyte of data storage. For large storage pools that can start to add up quickly. A single zpool with 8 drives ( I believe the recommended max) of 2tb drives would mean that you need 32 gigs of cache/memory to function correctly. Jump the drives to 4tb and you then need 64gigs of memory minimum to function. The above was my attempts to understand the requirements. Also raid cards aren't advised as ZFS likes having direct control over the drives for best results. Or so most articles mention. ZFS sounds awesome, very awesome. However many of the requirements and acceptable hardware makes it more difficult and picky than just running raid 10 on a computer (software.) as many motherboards support this. (Even if it is only 4 drives)
Halstrom Posted January 24, 2014 Posted January 24, 2014 Wow you've Levelled up in Windows Experience? What buffs & skills did you pick?
RitualClarity Posted January 24, 2014 Posted January 24, 2014 Wow you've Levelled up in Windows Experience? What buffs & skills did you pick? If I was him after going through all this .. I'd pick "Hulk Rage Level 2". FTW.
prideslayer Posted January 25, 2014 Author Posted January 25, 2014 ECC protects against occasional memory bitflips (usually) and running ZFS without it on a lot of data is unsafe (ZFS does not checksum its own internal memory structures)I suspected as much however I couldn't find in data on that, at least any reliable data. The "fan boys" rarely write of the downsides other than the memory requirements. If I understood correctly and SSD can be used for Cache. That can help with the memory issue but with an hidden cost. (premature SSD failure) If memory serves me correctly 2 gigs for every terabyte of data storage. For large storage pools that can start to add up quickly. A single zpool with 8 drives ( I believe the recommended max) of 2tb drives would mean that you need 32 gigs of cache/memory to function correctly. Jump the drives to 4tb and you then need 64gigs of memory minimum to function. They consider the risk of running ZFS without ECC no greater than running any other filesystem without ECC. It is, but the risk is still pretty small, though there is the increased exposure due simply to so much memory being used by the filesystem, and how it's used. If the memory corruption occurs after a block checksum has occured but before it's written out to disk, what do you suppose happens when that block is read up in the future and the checksum doesn't match? No matter the RAIDz level, that checksum is only calculated once before a write. So even if you have say a RAIDz2, which has 3 copies of everything, all of the copies will contain the same thing -- the correct data, but an incorrect checksum/hash. The above was my attempts to understand the requirements. Also raid cards aren't advised as ZFS likes having direct control over the drives for best results. Or so most articles mention. ZFS doesn't know if it's running on top of RAID or not. They advise against using a RAID beneath it because they expect most people to be using RAIDz, and ZFS decides how to spread data around in a RAIDz based on the disks in the pool. It ensures that mirrored data is stored on different physical disks, and it can't do this if the disks are LUNs from an array beneath it. Personally I think that two different uses cases are being conflated. In a consumer environment, you should follow their advice and just give ZFS direct access to the drives. With "real" enterprise hardware, disks store their own checksums for every sector and validate it (and error correct it) on every read, so there's no need for RAIDz. It's non-ZFS RAID and ZFS RAIDz that it doesn't make sense to mix -- not ZFS itself and RAID. ZFS sounds awesome, very awesome. However many of the requirements and acceptable hardware makes it more difficult and picky than just running raid 10 on a computer (software.) as many motherboards support this. (Even if it is only 4 drives) Be careful that hype isn't mistaken for 'awesome'. It has some great features, but tuning it is very difficult, the hardware requirements for reliable and fast operation are very high, and the protection it offers is rather small. The most commonly trotted out industry support for ZFS is the CERN data integrity paper from 2007, which didn't deal with ZFS itself, it was just a study on undetected hardware errors both from disk and memory. The result of their study was that they found 500 undetected (by disk or RAID) errors after writing and reading back 2.5 Petabytes. That's 2,500 TB. They wrote and read back 2GB of data to 3,000 different servers every 2 hours for 5 weeks, and detected 500 errors that the disks and RAID arrays did not catch. The errors were concentrated on 100 of the 3,000 nodes the test was run on. They also tested 500 RAID-5 arrays with their built in verify, which found and corrected 300 errors over the course of 4 weeks. I haven't been able to find any details on their hardware except that the drives were WD and the controllers 3ware. Not telling if they are SATA, SAS, SCSI, FC, or what. They did say that 80% (400 of the 500) of the disk errors they encountered were due to bad WD firmware.
RitualClarity Posted January 25, 2014 Posted January 25, 2014 those are pretty small errors rate on data. However it evident that if you want zfs to work properly you either do it correctly or go home. It isn't like runing a simple software raid from some 70 dollar MoBo that you picked up at Fry's. However I keep getting drawn to it because I just really like all the features that they have. The only other one that was interesting was BTFS.
prideslayer Posted January 26, 2014 Author Posted January 26, 2014 What I've heard says btrfs isn't ready for primetime yet, but that could just be disinfo from a zfs fanboy. The two "new" things I want out of any filesystem are online expansion, and elegant handling of thousands (or millions) of small files. At work we archive millions of smallish files (200K or smaller) and need to be able to quickly serve any of them, and delete any branches on demand. ZFS solves the online expansion issue, but still sucks at the other one. I started trying some distributed solutions (Moose, GridFS, etc) to spread the load around. I really like moose but unfortunately there are still some bugs in the FreeBSD FUSE system, and heavy usage leads to kernel panics. In the end no filesystem could fit the bill and databases are simply too slow, so I started storing the files in Riak which does everything we need it to. At home, I don't need any of that fancy stuff.
Recommended Posts
Archived
This topic is now archived and is closed to further replies.