An Oracle server – How Fast for £1,000 July 27, 2010
Posted by mwidlake in Architecture, performance, Testing.trackback
Question? How fast an Oracle server can you create for £1,000 pounds?
{I’d really appreciate feedback and suggestions on this particular post}
The power of domestic PCs continues to grow, with four-core chips become pretty much standard and starting RAM looking more like 4GB than 2GB, with 8GB quite reasonable. So, how quick an Oracle server can you make based on a domestic PC? After all, those of us who play with Oracle in our spare time tned to use such machines and, in fact, they are often not far off what are our smaller servers at the office really are. When I worked at the Wellcome Trust Sanger Institute, we had to make our IT budget pounds go a long way. We were, after all, a charity with a limited budget but also a scientific organisation with a huge demand for data and processing. So we used a lot of cheap kit.
I’m seriously thinking of giving this a go. I need a new PC anyway and so I am willing to use it, at least initially, to see what can be done.
If I do this, I’m going to need to set some boundaries on the exercise. How about:
- The oracle licence is being ignored in the cost {and please, I don’t need to be told how the licence can be more than the hardware costs!}. OS cost is though.
- I am not aiming for enterprise-level resilience, so I am not going to consider hot-swappable components, dual redundant power supplies or things like that.
- I am going to use new kit, so no scavenging or buying second-hand. It must all be easily available and repeatable.
- I will use local storage in the server or connected to ports available on the server.
- It will support a database of 1TB in size {yet to be designed}.
- Oracle v11. Enterprise edition but nothing special like TimesTen or Exadata (unless Oracle are willing to sell me an Exadata box for a grand, then I’ll consider it).
- I’m not considering backup and recovery performance {and this would be a serious oversight if this was a real system, but most places have central backup/recovery facilities}.
I would also have a few other things to decide.
The main one is “Do I use Linux or Windows?” Yes, you are all probably shouting “Linux!!!” but I have never been a Linux sys admin (I was an incredibly poor HP-UX system admin for 3 months though) so it will take me more time to deal with issues under Linux – in work situations I have always had access to people who know all this stuff to sort out issues but in this case I will be doing this on my own. On the other hand, you can just chuck Oracle on a standard windows box and it works, and as a rule hardware just works under Windows. If I decide to use USB3 ports, for example, is it going to be a major pain getting drivers under Linux? But then if I want the fastest oracle box under a grand why would I slow it down with windows and spend money on the licence? I just want the box to run Oracle and a workload.
The second “software” decision is, how do I measure performance? I think I could be getting to grips with Dom Giles’ excellent Swingbench {BTW, nice tag line on that page, Dom :-)}. But it runs on Java and guess what boys and girls? I’ve never been a Java developer. How limited are my skills! So that would take some of my precious spare time up too.
I’d love feedback on this, I’d love to know what hardware suggestions you would make, what you think about the overall idea, what else I need to consider to make the tests valid… I have a few ideas already for the hardware architecture and the intention would be to try lots of things but I’ll save that for a second post. After all, if I get no feedback I might just spend the money on a gaming machine and a week’s walking in the Lake District instead.
And if anyone want to help with the cost, please send cheques to….
Martin Widlake's Yet Another Oracle Blog 
Linux or Windows? Both. You probably want to install VMWare or similar, so you can pull up Linux ro Windows VM’s and create a poor mans RAC system with 2 VM’s pointing to the same USB disk system (you’ll need to check the feasiblity of this of course). You need a min of 2GB per Virtual Machine, so 4GB is the skinniest you can go. 8GB more sensible. You’ll want a quad core CPU. That should set you back about £500-600 all told.
Use Oracle’s Linux distro. Almost everything you need is preloaded and Oracle just installs on top. Simples.
You need to think seriously about how you are going to get disk performance – perhaps one or more cheap USB-based RAID systems (I know nothing about the following: http://www.ebuyer.com/search?page=1&store=2&sq=raid ). Need to check vmware compatability of course. This is the hard bit. You should check out smaller faster drives (500GB @ 7200rpm are less than £40 – Raid0 for performance and forget redundancy) rather than large lardy ones, as you’ll get better IOPS ;o) And USB-2 is limited performance-wise. You need to know how the throughput is configured on the USB sockets to see if it’s shared or not.
£1000? Should be a reasonable system.
regs
Neil
Hmm, not keen on going the VMware route Neil, simply because I want the fastest system. vmware will slow it all down. It’s another layer in the stack.
Interesting about you suggesting I go for the oracle distro. I’m interested to see what version of linux people would suggest.
The storage really could be the interesting part of this. Small, fast discs indeed. I think external storage could be hampered by the IO into the box, might be better to try and create a raid via the motherboard?
If you don’t go VMWare, make it a dual boot (install Windows first) and Linux likewise. Then you can mess with both.
As for I/O, why not look at USB, motherboard and Firewire attachments to use all of those I/O channels?
I did this last fall, but I went the Oracle VM route to be able to whip up a more ‘exotic’ environment if I needed to (RAC, Dataguard, Streams etc).
I dont remember the exact specs, but I used an Intel Core i7 + 12GB RAM, and 6 500GB (SATA3, 7200rpm, 32MB Cache) disks (2 for the VM’s, and 4 for ‘VM-storage (ASM disks etc)’. The 4 ‘storage’ disks are configured as a 4-stripe volume group in Dom0 that I carve slices out of and present as paravirtualized block-devices directly to each VM. I was meaning to use the onboard raid-controller but I never got it to work properly.
I also could have gone with smaller disks but I needed space more than performance. I did some performance testing and I think it was pretty ok.
I’m running Oracle’s Linux, Windows and Solaris x86 VM’s and it works like a charm. I’ve previously used Centos, Whitebox Linux (all Redhat clones) but since OEL came out with its public yum repository I just couldn’t be bothered using anything else.
The setup costed me around 8000 SEK, which would be around £700 today.
regards
/M
Hi Mike,
Thanks for the feedback. The hardware sounds pretty close to what I am thinking of, I’m also planning to investigate the use of motherboard raid but maybe ASM would be a better way to manage that.
Regards,
Martin
To give you an idea, I have Oracle eBusiness Suite running on a VM on my latop. Core2 Duo CPU, 500G hardrive, on the Oracle Linux distro. So the hardware won’t really be an issue.
I have no Linux skills apart from what I’ve picked up over the years, and I managed it with a bit of help from the forums.
If you want external drives, you should look at eSata ports over anything else. They’re effectively Sata ports with a special cable, although you will need enclosures of some sort, but they are not expensive. For example; http://www.ebuyer.com/product/143544
Hope that helps.
Alex
That does help Alex, especially the part abot eSATA ports. Most of the base systems I am looking at have them.
Thanks,
Martin
I have built two identical servers at work. They were Fujitsu RX200 with dual quad core processors and 16 Gbytes of memory, two 750 Gbyte SATA discs RAID to a stripe across the pair. On one I loaded Windows Server 2003 R2 (32 bit) and the other Oracle LINUX 64 bit. Both systems had Oracle 11G R2, 64 bit on LINUX and 32 bit on Windows. I loaded into both the same database and performed the same operations on both. The ASH reports themselves were very enlightening. Windows only utilised 2Gbyte memory, LINUX utilised the whole 16Gbytes (no surprise there as I was comparing 32 bit against 64 bit). The LINUX database had a memory target of 6Gbytes and the whole of my database slotted into memory very nicely. In comparison, the windows server was continually paging in and out of memory. One intensive I/O operation on the Windows server took 26 minutes and the same operation on the LINUX box took just 6 minutes.
Just idling with no work going on, the disc lights on the windows box were continually flashing activity, in comparison the LINUX disc lights were very rarely active.
I know these servers are not in the £1000 range set by your requirements, but based on these experiences, architecturally I think I would head for single quad core processor (£170) and 16Gbytes memory (£278), case, motherboard, graphics card etc (£330) with as big a raid (4 x Samsung 7200 rpm 1Tb discs at £50 each to stripe and mirror or let ASM do it’s stuff) as possible in the left over cash giving a total of £978. The processors in my tests were very rarely taxed, so there should be plenty of scope for a bit of scalability with a quad core.
Hi Tony,
Looks like I am following in your footsteps here 🙂
You confirm my fears about windows – it seems to spend all of it’s time doing stuff behind the scenes that you have no idea it is doing (and I suspect most of it is pointless).
16GB memory would be nice. It will depend on how I develop my workload (so, how much I am flooding by buffer cache and needing to go to disk and how large my sorts are so larger sort areas keeping them off disk etc) as to how much impact it has, but it would be a more normal amount of memory to have if I am looking at a pretty large DB.
Thanks Tony.
Also, for some words of encouragement, I have only worked in a Windows environment and have no LINUX experience. The instructions on how to successfully install Oracle on LINUX were easily obtained from the web and worked a dream. I even set up SAMBA so that I could access the windows discs from LINUX and that worked a treat too. The total time spent on this two server activity was five working days. Three to build and install and two to test.
As for swingbench, I have now used this tool on my desktop test system and without any JAVA experience have successfully tailored my own queries to enable a kind of representative load.
So take heart, new technologies are there to be explored, and if I at my late age can learn new tricks, I’m sure you’ll soon be operational.
🙂 Thanks for that Tony.
It is strange that I worry a little about taking on new areas of knowledge – I’ve spent most of the last 20 years introducing new features of Oracle to clients. I’m usually the one pushing for upgrading and using the latest-greatest, even though I know it often involves pain!
Anything with an i7 and gobs of RAM should do the trick from a horsepower perspective.
I second others comments about VM’s – you’ll be amazed at how little overhead they introduce.
In terms of disk, Seagate have just launched a nice disk which has a little SSD in front, backed with standard spinning rust as a “hierarchical” performance solution in a “single” disk…
Or you could go SSD all the way …. OCZ make cheapish 128 and 256G SSD’s
Hi Conor,
I guess I am going to have to factor in testing VM and seeing what it’s overhead is or is not!
The hybrid SSD/spinning rust storage is something I have seen out there and I’ve been wondering if it is a good option. I can imagine that it would gave very varied performance, depending on what gets put into the SSD side. I wonder it if would end up being like Parallel, where the variation in performance gives you more grief from end customers than the (usually) extra performance gives you praise from…well, no one, people almost never praise a fast system, they just moan about what they thinnk is a slow one!
I’ll certainly check out cheap SSDs. Could break the bank though…
Hi,
sometime blogs with comments help person like me to get some knowledge or ideas before making any discussion.
Rajesh shah
Personaly, I am very comfortable with linux (i’ve got RHCT), so “win or lin” is not a question for me at all.
Linux always do what i want from it. A 32bit Windows rather ugly OS for ORCLE.
Well, h\w decision highly depends on load type to me.
If you’ll do OLTP, i’ll think about:
– a pair of SSD for RedoLog
– dual\quad core, but not “high grade” CPU,
– more disk (RAID10 or even RAID0 – just stripng)
But If you’ll do lot of DSS, i’ll think about:
– Lot of RAM, hight PGA_AGGREGATE_TARGET (8 Gb is a start number)
– SSD for TEMP tablespace (not large but wery fast)
– dual multicore CPU (once again – not top line!)
– as much spindles as i’ll can afford for data
Hi,
If you hit problem with adding disk in one box – you can think about solution like open filer which is open source iSCSI attached disk storage. I know that iSCSI has some speed limitation but with bonded interfacses it could make it.
Anyway you can user internal disk for redo, undo and temp and external for rest – ex. based on 2 ASM disk groups.
From my expirience well configured Linux is more setup and forget solution then Windows based.
regards,
Marcin
Did you ever throw SLOB at it?