Oracle 11gR2 RAC On Linux Using VMware Server 2

 

I’ve taken my first tentative steps into 11gR2 RAC and it was a big surprise.

11gR2 RAC feels very different to 11gR1 RAC. I can imagine quite a few people wanting to upgrade from 11gR1 thinking it will be trivial and getting a rude awakening…

The Grid Infrastructure (Clusterware + ASM) seems more complicated. There are more installation options, more prerequisites, more background processes and a bigger memory requirement…

I typically install 11gR1 RAC on VMware using 1G of RAM per VM. If you try that with 11gR2 you will get to the end of the Grid Infrastructure installation and have nothing left. The minimum recommendation for Grid Infrastructure alone is 1.5G, but if you want the RAC DB as well you are talking 2.5G. It actually worked fine with 2G of RAM allocated to each VM, but this is a whopping increase compared to 11gR1.

At this point I feel like I know nothing about 11gR2 RAC, but it certainly doesn’t feel like a patched version of 11gR1. If this had been released as 12g I would have still have been surprised by the level of change.

So over the next few days I’m expecting the dust to settle, my residual fear of all things new to subside and I’ll probably change my opinion completely and think it’s all the same as it was before… 🙂

Cheers

Tim…

PS. Please don’t try this installation on your 32-bit Windows laptop with 2G of RAM then write to me complaining it doesn’t work and telling me the article is rubbish… 🙂

Author: Tim...

DBA, Developer, Author, Trainer.

47 thoughts on “Oracle 11gR2 RAC On Linux Using VMware Server 2”

  1. Hi Tim,

    Yeah this was big change- had to buy a new server to build 2 images for the 11gR2 RAC and ASM test setup with VMWare.

    -Ben

  2. Tim,
    Quick question.

    11gR2 Rac requires ASM for voting disk, OCR for storage and RAW devices are not supported. Does this mean that we have to install the oracle software before the installation of Grid Infrastructure, or if there is a different work around?

    In Linux we can even use ASM Lib to create the asm disks but that option is not even available in UNIX. I am a bit confused here.

  3. Hi.

    ASM is part of the grid infrastructure, so you don’t need to install ASM first, then clusterware. It’s a single step.

    When you install grid infrastructure, you define in the installer both the ASM and clusterware configuration and it installs both.

    Cheers

    Tim…

  5. hi,

    should i have separate scan ip for all the nodes OR is it one scan ip for whole cluster ?

    Any idea how i can configure scan in a standalone laptop ?

    thx
    sunil

  6. Hi.

    It says in article. Scan should be a single name that round-robins between 3 IP addresses (configured in DNS). For my VMware play system I configure just a single SCAN address in the hosts file. It’s all in the article.

    Cheers

    Tim…

  9. Tim,

    I want to try it with Windows 7 as host system using VMware Workstation 7. My pc has 8G ram and 1TB disk space and has 8200 quadcore processor.

    Will this config work? Is any pointer to any document to follow?

    Thanks in advance.

    Regards
    Rafiq

  11. Tim,
    Do you know if in 11gr2 we can use multiple interfaces for interconnect and public interfaces?

    Thanks,
    Ali

  12. Hi.

    This is normally done using NIC bonding, making multiple cards look like a single interface, thereby giving redundancy.

    Cheers

    Tim…

  13. TIm,
    Thanks for quick reply , that is what my understanding was but I was wondering if we have two bonded interfaces (for example bond1 and bond2 with two subnets) can we specify them during clusterware installation at “identify network interfaces” screen. It seems like that screen does show you all the interfaces and allow you to select them for interconnect

    Thanks,
    Ali

  15. Tim,
    Sorry for bugging you but I could not find the answer in oracle documentation

    We have two seperate bonded interface for interconnect (i.e. bond1 and bond2)
    and three for public interface (i.e. bond3,bond4,bond5).
    I can define 2 for interconnect and 3 for public during installations
    but I am confused on how to define the VIP in /etc/hosts or DNS.

    Could you please tell me how the VIP configuration would work in this case?

    Thanks

  16. Hi.

    The two subnets won’t work. Maybe you can achieve this with something like multipath?

    I’m not really a network guy so I typically rely on other to present me with the interfaces in complex configurations.

    Cheers

    Tim…

  17. Hi Tim ,

    I am planning to install 11g RAC on AIX machine , and my requirement is ASM which will support OCR and Voting disk db datafiles and recovery files .

    My question is ” can i install grid infrastructure in shared cluster file system as i know from some doc we can’t install grid infra on shared file system . then we have only one option to install in local file system if i m correct ?” and can we need to install grid infrastructure in remaining nodes since if we are installing in local file system ,
    it will not not recongnize remote rac instace .

    I hope you can understand my requirement and dought .

    Thanks in advance .

    Santosh Rai

  18. Hi.

    On linux the recommendation is the Grid Infrastructure software is installed on the local machine. I can;t speak for other OSes as I’ve not read their install guides. Typically there has nor been a problem installing the software on cluster file systems in the past.

    Even when you don’t use a shared home, you don;t have to install the software on each node, the installation process does this for you, so don;t worry about that.

    Cheers

    Tim…

  19. Amazing that you try and do the install without regard to the recommended RAM and then complain that it does not work with less than the recommended RAM. If you are doing an exercise in determining the minimum amounts then just says so at the title level instead of making a big deal statement about 11gR2 being very much different than 11gR1 just based on the RAM requirements.

    Get over the fact of needing more RAM for Oracle; it does a lot more than MS Access or FoxPro

  20. Hi.

    You seem very angry about something.

    1) If you check the title of the post, it’s a VMware installation for practicing RAC with. As a result, RAM size is an issue because not everyone has a large server to play with.

    2) I think the title used is a very good description of the article I wrote and linked to, since it is about installing 11gR2 RAC on VMware, so I don’t really understand your point.

    3) I think I know the difference between Oracle, MS Access and Foxpro. I’ve been working with Oracle for over 15 years. A quick look at the articles I’vbe written on my website should prove that.

    4) An extra 1-1.5Gig of RAM is a big deal to some clients who are running on smaller kit. It means there is not a straight upgrade to R2 without a hardware upgrade. I think this is quite an important point.

    5) If you don;t like what I write, don;t read it. No need to have a cow dude.

    Cheers

    Tim…

  21. Hi Tim,

    i need to build a 11g R2 RAC on a virtual machine to be used in a workshop labs about 11gR2 oracle RAC .. students should use thier laptops ..
    as your notes about memory requirements .. i think of asking the students to have 8GB of ram on thier laptops and 100 GB of storage.

    i’m not sure if the it would be applicable .. so would you please till me the minimum requirements :
    1) is it enough to have 8GB of ram and 100 GB of storage ?

    2) is it mandatory to use vmware server or can we use vmware workstation?

    3) is it mandatory to use 64 bit H/W ? or can we use 32 bit hardware?

    4) What’s the minimum CPU power to run those 2 RAC VMware images in your openion?

  22. Hi.

    1) Yes. 8G of RAm and 100G storage is enough, but you aren’t seriously considering asking students to bring a laptop with 8G of memory are you? You will be lucky if they can bring a 4G laptop.

    2) I’ve never tried it on VMware Workstation so I can;t comment. VMware Server is free and works on laptops like VMware Workstation, so there is no barrier to using this. Note. I use Linux as my Host OS. I can;t speak for using VMware with a Windows host because I don’t do it.

    3) I guess you could use 32-bit hardware, but once again, I’ve not tried it so don’t blame me if this process doesn;t work for that.

    4) I use a Quad Core and it’s not that fast. If might struggle on a dual core, but once again, I’ve never tried it on that, so I can’t comment.

    Note. These RAC installations are very resource intensive. I think it is unrealistic to expect a student to bring a laptop with enough power to cope with it. When Oracle do the RAC-Attack sessions they provide workstations for people to use.

    It would be sad if someone pays to come then can’t do any of the work because their hardware is not up to it.

    Note 2: You need to be licensed to use Oracle RAC on all the systems you are teaching on. Not doing so is a breach of Oracles T&Cs and you could be in trouble. Contact Oracle to check on licensing for teaching. It is very specific.

    Cheers

    Tim…

  23. Hi Tim,

    thanks tim for you valuable answers.
    I’m already an Oracle consultant who works at oracle by the way 🙂

    i need to build that virtual environment to hold an internal training sessions.

    thanks for your concerns

  24. Hi.

    You might want to have a word with Dan Norris at Oracle. He did some of the RAC-Attack sessions at some OUG events. See what hardware they used during these events.

    Cheers

    Tim…

  25. Hi Tim,

    can i use Vmware workstation 7 instead of VNware server ? it also supports independent and persistent disks. or there’s another cause that make us have to use vmware server 2?
    thanks

  27. Hi,

    I have a highly configured DELL XPS 9000 with 12 GB RAM, Dual Quadcore 3.2GHz processor( ofcourse 64-bit). And one terra byte memory. I created two virtual machines with 2 GB RAM and 1 CPU and RedHat 5.3 linux on it. I had successfully complete the Grid installation. But the installation fails for database. Trying to set a two node cluster database. The Virtual machine which i created will always crashes during the database installaion. I tried to install single instance database, but still it crashes. Not sure why it can’t take the load. Can you tell what should i do if this is the case?

    Thanks
    Veera

  28. Hi,
    I am installing 11gR2 Grid on IBM Aix for 2 node RAC database.
    I am using EMC San Storage LUNS for ASM Diskgroups.
    How many disks (Luns of 2 GB size) are needed for creating a Cluster Disk group to be used for placing Voting Disks and OCR with external redundancy?
    thanx,
    Sooraj

  29. Hi,
    Just to let you know that VMWare Workstation doesn’t support clustering, so any RAC configuration can’t be done using Workstation. You need to use VMWare Server in that case.

  32. can anyone help me with 11g R2 grid infra installation? I am confused in scan and do not know how many public and virtual IPs are needed. Please help me

  33. Dude, read the article and it tells you.

    Public: 1 per node.
    Private: 1 per node.
    Virtrual: 1 per node.
    Scan: 3, regardless of number of nodes in RAC.

    Public, Virtual and Scan are all on the same subnet.

    Cheers

    Tim…

  34. I have followed all the steps as per your site

    Everything going fine but when i ran the root.sh on rac2 machine then root.sh show some failed messages i.e.

    ====================================================
    Preparing packages for installation…
    cvuqdisk-1.0.7-1
    Configure Oracle Grid Infrastructure for a Cluster … failed
    =====================================================

    root.sh shows same logs on rac2 as on rac1

    I have executed the below command

    ./olsnodes
    rac1

    But it shows the only one node instead of two node.

    Please let me know if i have miss anything.

  35. If you’ve followed the article, taking all the steps I suggest it should work fine. The fact it doesn’t leads me to believe you’ve made a mistake.

    Please don;t ask question in the comments here on in the comments on the article. Ask them in the forum on the main website.

    Cheers

    Tim…

  36. Hi, I am trying to install Oracle 11gR2 RAC on Linux 5 64-bit VM machine.

    As part of that i have used OCFS2 for Oracle software and OCR.

    I am getting following error message when pointed the oracle base location to shared disk (which is accessible between both nodes) created by OCFS2.

    INS-40103: The installer has detected that the software location specified is on an OCFS2 partition

    Please help me

  37. Hi.

    1) I would never use OCFS2 with RAC if at all possible. Why? They use two separate cluster infrastructures that interfere with each other. Only use them together if there is no other alternative.

    2) 11gR2 recommends a local installation of the Grid Infrastructure software, so OCFS2 shoudl not be used for this.

    3) 11gR2 allows you to place the OCR and voting disk inside ASM, so this is now the prefrred location for these.

    4) If you need a shared home for the database software (not the grid infrastructure), this can be placed in ASM using ACFS.

    I would never run RAC with OCFS2, so I have never tried what you are attempting and would not.

    Cheers

    Tim…

  38. Earlier in Oracle 11gR1 RAC+ASM installation we went with OCFS2, there is no special reason to attemp with OCFs2.

    11gR2 recommends a local installation of the Grid Infrastructure software, Means do i need to install grid Infrastructure software on individual nodes?

    My understanding is that ORACLE_HOME is shared between two nodes, Please correct me if i am wrong.

  39. Hi,

    I am getting following error while trying to provide Oracle software locatio.

    [INS-20702] Unexpected Internal driver error

    Cause – A call to cluster verification utility failed.

    Following are the steps i have followed:

    1. Using NFS mount created a share mount point between two nodes.
    2. Used OCFS2 for OCR
    3. Pointed local location at the time of installing Grid infrastructure.
    4. After executing rootInventory.sh and root.sh using root user on both nodes i could not able to create any folder in NFS mount and OCR location.
    5. Used NFS mount point /u01/app/…etc location for Oracle software location at the time of software installation.

    Observed the above error.

    Note: Got the same error when tried to give “grid” location at the time of “Grid software” location, so re-directed it to local home. But able to see the same content in the another node aswell.

    Please Help me

  40. Hi.

    I don’t understand why you are using multiple methods for shared filesystems. It doesn’t make sense.

    You can use NFS for all your shared files if you like:

    https://oracle-base.com/articles/11g/OracleDB11gR2RACInstallationOnLinuxUsingNFS.php

    If you want to do it all using ASM do this:

    https://oracle-base.com/articles/11g/OracleDB11gR2RACInstallationOnOEL5UsingVMwareServer2.php

    Your mix and match approach is prone to disaster, as you’ve already proved.

    If you have any questions, please ask them on the forum not on the comments here. It’s not really a suitable location for dealing with technical material.

    Cheers

    Tim…

  41. hello all,
    yesterday i tried to install 11gr2 rac on 4g ram machine. i give 1662 mg ram to each node. every thing works fine but got an error at the very end of grid installation.

    NFO: Check CTSS state started…INFO: CTSS is in Active state. Proceeding with check of clock time offsets on all nodes…INFO: PRVF-9661 : Time offset is NOT within the specified limits on the following nodes:INFO: “[rac2]”INFO: PRVF-9652 : Cluster Time Synchronization Services check failedINFO: Post-check for cluster services setup was unsuccessful on all the nodes.

    INFO:
    WARNING:
    INFO: Completed Plugin named: Oracle Cluster Verification Utility
    INFO: Oracle Cluster Verification Utility failed.

    also note : i got another error while trying to stop ntp
    # service ntpd stop
    Shutting down ntpd: [ OK ]
    # chkconfig ntpd off
    # mv /etc/ntp.conf /etc/ntp.conf.org
    # rm /var/run/ntpd.pid

    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    Shutting down ntpd: [FAILED]

    any idea?

    Thanks
    syed

  43. Hi Tim,
    I experienced similar issue like Veera.
    EL5 and Grind Infrastructure installations went fine. Most of RAC installation went fine except DB creation.
    The Virtual machine which i created will always crashes during the database installaion.
    The error of vmWare Server 2.0.2 for one or both instances is bugNr=41568
    Found on internet that entry of reslck.timeout=”1200″ in vmx file should solve the issue, but instead I am not able even to start VM (hangs on 95%)
    Any suggestion?
    Thanks,
    Lazar

  44. Hi Tim,

    I am a newbie and I am working for IT company where we don’t have RAC env. But now they have given me enough hardware to do testing and prepare a proposal document for 11gR2 RAC. The virtual machine which they offered me is ESX workstation. I am new and I am not quite sure about ESX. Any knowledge and does it allow me to setup RAC without any hassles.

    Hardware I recieved as below:
    Each node: 2GB RAM, 150GB space, they have Linux and Solaris-SPARCK 64 bit.

    Can you please shower your inputs so that I can start with.

    Thanks

    Regards
    Murthy

  45. Hi.

    VMware ESX and VMware Workstation are different products, so you have one of the two, not “ESX Workstation”. 🙂

    You can do RAC installations on ESX or Workstation, but the process is slightly different.

    Note 1. Oracle 11gR2 RAC specified a minimum of about 3.5G or memory is required. You current VMs with 2G RAM is not enough to try anything useful, in my opinion. When I do virtual RAC installations I used at least 4G RAM per VM.

    note 2. I’m currently not using either of these products for virtualization, so I’m not sure I can be of help here.

    Cheers

    Tim…

  46. Can you help me on this?

    i am trying to install Oracle RAC 11.2.0.3 on Oracle linux 6 (VMWARE)
    my system (laptop) with 8GB ram and 32 bit windows 7, when i run runcluvfy.sh i got the output system architeture failed expecting “s390x” but available is i686

    Check: System architecture
    Node Name Available Required Status
    ———— ———————— ———————— ———-
    mac1 i686 s390x failed
    Result: System architecture check failed

  47. @sundar:

    It looks like you’ve downloaded the wrong software. If you are running on a 32-bit platform, you will need 32-bit VMs, 32-bit Oracle Linux and 32-bit Oracle for Linux.

    Cheers

    Tim…

    PS. It’s easier for me if you post questions on my forum. I pick those up quicker.

Comments are closed.