Dell PowerEdge SC1430
Dual Quad Core for Folding
Yes for those of you counting out there that's 8 processors on one board!
Chapters
10/22/07
First our objective is to replace my old Xeons (3.2ghz dually at 533 fsb, a couple of 2.8's) and build a dual quad core machine that will out fold them and by so much that I can replace 3 or 4 of these dually Xeons with one quad core machine. Thus saving a ton in electric bills.
The reason I chose the above Dell machine is that it was on sale and I don't think I could build a machine with this much power for the same price myself. Although I had wanted to build a monster.. with fire coming out the sides.. lol... I decided this machine will make a very capable machine for a very reasonable price. Besides there are rumors that the 1.6 processors overclock fairly well. So investing in the more expensive faster processors may not be necessary in the beginning. Besides we all know the prices will drop in the near future and this board will reportedly support any 5000 series Xeon. Oh and the price drop is usually timed directly or within 3 days of me buying something.
First off the spec's:
Dell had a deal (don't they always? lol) where you could get the second quad core free.. so...
I got the above bare box server (fans, lights, etc)
2 - Quad Core Intel Xeon E5310; 2X4Mb Cache, 1.6Ghz, 1066MHZ FSB
1GB 667Mhz (2 x 512MB) Single Ranked FB-Dimms
160GB 7.2K Serial ATA 3Gbps 3.5in HD.
That was the package from Dell.. with shipping and handling.. a little over $800. Which I thought was a pretty good deal.
Yah I could have gotten faster CPUs.. more memory... more drive space... but for $800 I figured I did quite well.
Here is the CPU-Z detailed specs on the machine. Dell Server
Now to add to that I ordered 2 2gb Samsung 2GB FB-DIMM's from a guy on www.2cpu.com where I buy a lot of used parts.
The machine then will have a total of 5 GB of RAM and I have a spare 160 SATA drive sitting here on the shelf in case I want to mirror them or raid them for more speed. However I think the only place we might run into trouble here is the memory. Not sure if 5 gb is enough to keep an 8 core machine busy. We shall see though. I also have to think hard about what operating system to run. Linux 64 of course would be a great match but I am not much of a linux guru and not sure if I want to tackle making it work. I have an XP license here but I don't think it can handle the memory unless I go to 64 bit. That is still a problem most likely as 64 bit is the future but still not all the drivers out there are great yet. So I will contemplate all of this and decide what to do. My ideal plan would be to use XP but the memory thing bugs me.
10/23/07
Course now is the hard part. All this hardware on order and me sitting here drumming my fingers on the desk. hehe. Well not really. I have a lot of choices going on right now. Trying to decide whether to overclock or not. Whether to just sell the 1.6 processors now and buy faster ones. I am a bit of a cooling nut so will be looking at adding fans and maybe changing heatsinks. Reading about overclocking at www.2cpu.com. They have a monster overclocking thread on 5000 series xeons and those guys are pretty sharp. I will pay close attention to them. Right now they have 39 pages of messages... and I am on page 12 so far. Oh and my eyes are tired, needless to say. One other thing, I ordered a Kill-A-Watt meter today to monitor power. I really want to know exactly how much power I was using with the old systems and how much I will be using with this one. I am thinking it is going to be an eye opener. Stay tuned.
10-25-07
Good news.... the Dell was supposed to be shipped on the 26th... well they shipped it on the 23rd. Then Fed Ex tracking said it was going to be delivered on the 29th. None of that turned out to be true. It arrived today... the 25th. So I am happily sitting here looking at the new quad core behemoth that soon will be folding its heart out. Now the bad news... The memory for it and some adaptors aren't here yet. The Kill-A-Watt meter came today but not the adaptors I need. I checked tracking and it says tomorrow. So... I spent the evening taking a look at the insides. I am a bit overwhelmed I must say. I am used to a pretty basic case design with lots of room etc. Well this thing has cages for drives, cages for add ons... etc. Its really pretty well put together. A bit too much plastic for my tastes but well put together. Here are some pictures of the insides.
Notice the blue handle in the middle of the first picture. That entire metal cage holds two drives. One is installed and that is the black you see in the picture. I assume that grey aluminum thingy with fins on the motherboard is the heatsink for the north bridge chip.
Here you see the cage opened up and the fan on
the underside.
Although its hard to see this fan will blow down on the FB-DIMMS. Good
stuff there. The back side behind the FB-DIMMS has a place for two
80mm fans. I will probably put two fans there to exhaust the heat. But
first we will test this machine in a completely stock condition. Well
except for DVD burner I installed. White was all I had on the shelf.
Other than that I will leave this box completely stock for the moment.
Oh and there in the front... that big black cage
on the right
has a 120mm fan that looks like about 25mm thick. Should move a lot of
air. Right next to it is another 80mm. That should cool those puppies.
Tomorrow we will install the memory... if it gets here, and install Windows XP Professional 64x edition and get this box folding. I have not looked at how to measure temperatures and such yet but will put the watt meter on her and get some power measurements. Now one thing... for this test to be accurate we must break this unit it. the various heatsinks, on the processors, the north bridge chips, etc all need to get hot and settle in. It actually takes some time to get the grease on the heatsinks to melt and do their designed job. So... once I get it running I want it to run at least 48 hours before we start measuring anything.
10/26/07
Got the USB to PS2 adaptors. Let me back up. This machine has no PS2 connectors for the keyboard and mouse. So you must use USB stuff. My KVM switch has PS2 plugs on it so I needed the adaptors to plug it in. Plus I was kind of waiting one more day for the memory to get here. Looks like not tonight.. so am going to install Windows XP Professional X64 on her anyway and set up the folding and let her cook even without the extra memory. Can't let all this power just sit there on my computer room floor not trying to save humanity can I? lol
... later...
Hear that whirring?... tis the sound of power... computer has booted... configured the CMOS... and now getting ready to install OS. Oh the power. Well first hiccup. When I powered up it gave me a bunch of instructions on the screen for doing things. I followed them and then it finally wanted me to install a sever operating system. Well I am not installing any of the OS's listed so finally had to shut her down and reboot. This time into the WinXP X64 disk. Take that Dell. We shall see if that hurts me or not.
Well the only real glitch I have seen so far was that when all was installed it could not connect to the internet. So I went to the hardware settings and sure enough the network adaptor had the ole question mark on her. I opened her up and go to the point of installing the new driver... got the Dell disk out and put it in... and it took off and installed without me even telling it to. hmmm... But still no internet. Tried a few things... did a full power off and back on reboot... also known as a cold boot but no dice. So decided to double check the cable... sure enough it was sitting there unplugged from the machine. Sheesh.. plugged it in and away she went... full internet access. Course then I check Windows update and sure enough there are a ton of updates to install. Yah right I just downloaded this Operating system off their website... why would it be up to date? grrrr... Now installing first update... this might be a while...
Took me most of the evening to get all the updates installed. Now its about 10:30 and I am getting tired. Next step was to install the folding program. Well since I have 8 cores and the SMP folding program was designed for 4 cores I planned on installing two instances. No big deal right. So went to Stanfords site to read the install info and nowhere could I find anyone explaining how to install two instances. Plenty of posts about a single instance but no one describing how to put up two instances. Went back to 2cpu forums and looked there too. Can't find anyone posting HOW. Plenty of info on fixing problems but nothing about installing two instances. SO... I had to guess.. and here is what I did.
First I created two separate folders for the folding program, folding1 and folding2. I then downloaded the program from Stanford and made a second copy for the folding2 folder. Then I ran the client executable which basically just unpacks all the files. Then in the folding1 folder I ran the install.bat. This basically starts up some MPI services. I don't know what that is but I know smp folding needs it. Then I ran the fah.exe file. Since this is the first time it has run it wants you to answer the configuration questions. I went through them as normal and answered yes to the big work unit question then yes to change advanced option and told it to set the advanced methods flag (not sure if thats right or not) and made it machine id 1. Sure all easy stuff right. Same as the old folding program. Now comes the second instance.
I went to the folding2 folder and ran the client but this time didn't run the install.bat. I figured the MPI service was already running from the first instance so no need to start it again. So all I did was run the fah.exe. Answered the config questions the same EXCEPT made this one Machine Id 2. Thats all there was to it. Now they are both running full speed. All 8 cores show 100% utilization so I think things are set up right. We shall see.
10/27/07
Computer has been humming away all night... still have not checked any temperatures yet but it is on the list of things to do. Also we learned yesterday that the memory system the 5000V chipset is dual channel so we don't have the high bandwidth quad channel memory like the 5000X chipset has. This will of course slow down the system some in comparison but probably won't affect our folding project much. So for folding I am thinking there will be very little effect. I also found that there is a program for setting the affinity of the CPU's and this seems to speed up folding a bit. Not sure I understand it yet but here is a link to the discussion about it if your interested. Affinity here and here. Seems some russian guy developed it and it boosts the points for multiple SMP programs. More to follow. But like I said above I want to do all the testing on this machine in a completely stock setting so as to get a baseline on what works and what doesn't.
10/28/07
Ok first start of testing... I installed a program called fahmon to monitor the points that folding is making. It will report the points per day estimate via the log file (supposition on my part) and tell you how many points you are making per day. Nice little feature there. So I installed it on my main computer and then let it find the quad box via the network and VIOLA I had my points totals. Now I don't know a lot about this program but... for my main machine which is a dual 3.2ghz 533fsb with an X1950 pro AGP card I am getting ~668 points per day. With my new quad box 2 E5310's 1.6ghz I am getting 2835! So that is on the order of 4 times the output. I am still experimenting with what all these numbers mean but first impressions are pretty darn good! Course it is NFL Sunday here and I will take some time out to watch football so not much testing to be done today.
10/29/07
The first tests are in. Tonight I got the extra 4 gigs of memory for the machine. I have also finished 2 work units and think the machine is now pretty well burned in. So its time to run some tests. First test lets see how memory effects power usage.
Idle power full load
1gig memory 150 watts 210 watts
5gig memory 171 watts 233 watts
So the 2 2gig memery modules are pulling about 21 to 23 watts.
Now to see if the memory makes any effect on speed. My first wu averaged ~2850ppd per fahmon program. The second averaged 2910. Now to see how the memory affects the present work unit. Because they are all the same type of wu I am assuming the times should be directly comparable. All 6 of these work units (I am doing 2 at a time) were 2653's so they should be pretty close in ppd. So the only difference in this test is the memory. We shall see.
10/30/07
Preliminary tests show that the wu's right now are running a bit faster. I am not sure what to think here as I doubt very much the memory could be accounting for this. But at the moment this pair of Work Units are making about 2947 ppd. Not sure what to attribute this to. Checked taskmaker and only using 800+ mb of memory so its not like it was starved or anything. Might just be the heatsinks settling in... might be these particular WU's are a little faster... just not sure. Due to the fact that folding is very math orientated I doubt memory has much to do with it. I will just have to keep testing I guess.
11/16/07
Ok I have struggled to set up the testing on this machine. Seems the Dell software that monitors temps is not installed or at least I cannot get it going. Anyway I installed CoreTemp today which gives me at least the CPU temps. I really wanted to monitor cores, fb-dimms, case, etc. But I am getting tired of going around in circles looking for the software to do it. So I will stay with CoreTemp for the moment. Now the next little problem has been a little troublesome. Every day or two my folding program would just hang. No errors nothing. Just stop in mid fold and sit there. No CPU activity nothing. Someone in one of the forums thought it might be temperatures. I said heck no this is a stock box ... it probably is running very cool. Well looks like I have to remove my foot from my mouth as the temps that are being reported are on the edge of max for this processor. I am getting temps ranging from 57 to 66 C! That disturbs me no end as I thought those fans in there would be very good. Anyway tonight as soon as this work unit finishes I am going to tear the machine apart and see what's what. Will report back when I know more.
11/17/07
Ok opened up the machine and didn't see anything special. I thought maybe a fan was unplugged or something. No dice. So closed it back up. Then I thought well maybe something in the bios might control these fans... throttle them down or something. But no dice there either. Now I know something is holding the fans back because when the machine first boots up I can hear the fans rev up. Then they slow down and are pretty quiet. So something, some temperature sensor or something is telling them to throttle back. My first thought is to disconnect these fans from the motherboard and wire them direct to the power supply. Then they would run full blast all the time. Might be a little noisy though. We shall see.
I had one very big shock last night while looking at things. Remember the power readings we took on the new machine. We are turning out something on the order of 2950 points per day and using 233 watts to do it. Well I took the ole kill-a-watt meter and put it on my dual 3.2ghz 533 fsb machine that I have been folding for years now. This machine also has an X1950 Pro AGP card in it which is reputed to take a lot of power. Well this machine is turning out right around 880 points per day... BUT it is using 410 watts to do it! MY GOSH! That is amazing! Using all those watts and makes 1/3rd the points the dual quad machine is making. I am astounded. I know now that all my dual xeon machines will have to go. This just proves my earlier theory that the new quad cores are quite a bit more efficient.
Recap
Quad core V8 system ~ 2950 ppd using 235 watts
Dual Xeon w X1950 pro ~880 ppd using 410 watts
Ok so now we know the new quad core machine is much more electric friendly than the old xeons. Now its time to get the processors a bit cooler.