02 January 2014

On the previous post vagrant part 1 we get the things started.

Let's continue doing more staff on our vagrant machine.

It's highly recommended to visit the vagrantfile documentation page for more options and cool things to do.

Our machine oracle65 have 4 hard didks:

    [vagrant@box1 ~]$ sudo fdisk -l | grep GB
    Disk /dev/sda: 16.8 GB, 16777216000 bytes
    Disk /dev/sdb: 33.6 GB, 33554432000 bytes
    Disk /dev/sdc: 33.6 GB, 33554432000 bytes
    Disk /dev/sdd: 33.6 GB, 33554432000 bytes

Typically we will use them for an Oracle clusterware/grid + database in the following way:

  • OS
  • u01 for binaries
  • u02 for datafiles
  • u03 for fast_recovery_area

Other option may be:

  • OS
  • u01 for binaries
  • asm disk for diskgroup data
  • asm disk for diskgroup fra

That pretty much cover a normal test bed you may need to have at hand.

I have created some scripts that will help us to automate this task:

    alvaros-mini:box1 alvarom$ ls -1 *sh
    sdb_u01_btrfs.sh
    sdc_data_asm.sh
    sdc_u02_ocfs2.sh
    sdd_fra_asm.sh
    sdd_u03_ocfs2.sh

You can check them and download them in the github page for box 1

Let's start with the first option, u02 and u03 as filesystem.

Modify our Vagrantfile:

        config.vm.provision :shell, :inline => "cd /etc/yum.repos.d ; > public-yum-ol6.repo ; wget http://mini.home.kikitux.net/stage/mini-yum-ol6.repo"
            config.vm.provision :shell, :path => "preinstall_crs_db.sh" 
        config.vm.provision :shell, :path => "sdc_u02_ocfs2.sh"
        config.vm.provision :shell, :path => "sdd_u03_ocfs2.sh"

Please note, that I replace the install of the rpm with a new script, this script does all the needed bits to ensure the installation of grid/clusterware and database will run without problem.

Let's run the provision again:

    alvaros-mini:box1 alvarom$ vagrant provision
    [default] Running provisioner: shell...
    [default] Running: inline script
    --2014-01-02 15:44:56--  http://mini.home.kikitux.net/stage/mini-yum-ol6.repo
    Resolving mini.home.kikitux.net... 192.168.1.2
    Connecting to mini.home.kikitux.net|192.168.1.2|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 4354 (4.3K) [application/octet-stream]
    [default] Running provisioner: shell...
    [default] Running: inline script
    Loaded plugins: security
    Setting up Install Process
    Resolving Dependencies
    --> Running transaction check
    ---> Package oracle-rdbms-server-12cR1-preinstall.x86_64 0:1.0-11.el6 will be installed
    --> Processing Dependency: xorg-x11-xauth for package: oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64
    ..SNIP..
    ---> Package libSM.x86_64 0:1.2.1-2.el6 will be installed
    --> Finished Dependency Resolution

    Dependencies Resolved

    ================================================================================
     Package                  Arch   Version                       Repository  Size
    ================================================================================
    Installing:
     oracle-rdbms-server-12cR1-preinstall
                      x86_64 1.0-11.el6                    ol6_latest  15 k
    Installing for dependencies:
     bc                       x86_64 1.06.95-1.el6                 ol6_latest 109 k
    ..SNIP..
     xorg-x11-xauth           x86_64 1:1.0.2-7.1.el6               ol6_latest  34 k

    Transaction Summary
    ================================================================================
    Install      41 Package(s)

    Total download size: 12 M
    Installed size: 35 M
    Downloading Packages:
    --------------------------------------------------------------------------------
    Total                                           1.6 MB/s |  12 MB     00:07
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
      Installing : libgssglue-0.1-11.el6.x86_64                                1/41
    ..SNIP..
      Verifying  : libXv-1.0.7-2.el6.x86_64                                   41/41

    Installed:
      oracle-rdbms-server-12cR1-preinstall.x86_64 0:1.0-11.el6

    Dependency Installed:
      bc.x86_64 0:1.06.95-1.el6
    ..SNIP..
      xorg-x11-xauth.x86_64 1:1.0.2-7.1.el6

    Complete!
    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-37770-k96al2
    mkfs.ocfs2 1.8.0
    Filesystem Type of mail
    Label:
    Features: sparse extended-slotmap backup-super unwritten inline-data strict-journal-super xattr indexed-dirs refcount discontig-bg
    Block size: 4096 (12 bits)
    Cluster size: 4096 (12 bits)
    Volume size: 33554432000 (8192000 clusters) (8192000 blocks)
    Cluster groups: 254 (tail covers 31232 clusters, rest cover 32256 clusters)
    Extent allocator size: 33554432 (8 groups)
    Journal size: 268435456
    Node slots: 1
    Creating bitmaps: done
    Initializing superblock: done
    Writing system files: done
    Writing superblock: done
    Writing backup superblock: 3 block(s)
    Formatting Journals: done
    Growing extent allocator: done
    Formatting slot map: done
    Formatting quota files: done
    Writing lost+found: done
    mkfs.ocfs2 successful

    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-37770-1j7j2xi
    mkfs.ocfs2 1.8.0
    Filesystem Type of mail
    Label:
    Features: sparse extended-slotmap backup-super unwritten inline-data strict-journal-super xattr indexed-dirs refcount discontig-bg
    Block size: 4096 (12 bits)
    Cluster size: 4096 (12 bits)
    Volume size: 33554432000 (8192000 clusters) (8192000 blocks)
    Cluster groups: 254 (tail covers 31232 clusters, rest cover 32256 clusters)
    Extent allocator size: 33554432 (8 groups)
    Journal size: 268435456
    Node slots: 1
    Creating bitmaps: done
    Initializing superblock: done
    Writing system files: done
    Writing superblock: done
    Writing backup superblock: 3 block(s)
    Formatting Journals: done
    Growing extent allocator: done
    Formatting slot map: done
    Formatting quota files: done
    Writing lost+found: done
    mkfs.ocfs2 successful

Let's check the mount points and filesystem for btrfs and ocfs2

    alvaros-mini:box1 alvarom$ vagrant ssh
    Last login: Thu Jan  2 15:33:30 2014 from 10.0.2.2
    [vagrant@box1 ~]$ df -h
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/sda3       9.3G  1.6G  7.0G  19% /
    tmpfs           2.9G     0  2.9G   0% /dev/shm
    /dev/sda1       477M   49M  403M  11% /boot
    /dev/sdb         32G  312K   30G   1% /u01
    /vagrant        238G   53G  185G  23% /vagrant
    /dev/sdc         32G  296M   31G   1% /u02
    /dev/sdd         32G  296M   31G   1% /u03
    [vagrant@box1 ~]$ mount -t btrfs
    /dev/sda3 on / type btrfs (rw)
    /dev/sdb on /u01 type btrfs (rw)
    [vagrant@box1 ~]$ mount -t ocfs2
    /dev/sdc on /u02 type ocfs2 (rw,heartbeat=none)
    /dev/sdd on /u03 type ocfs2 (rw,heartbeat=none)
    [vagrant@box1 ~]$

Pretty good..

Important tip

The scripts you use to provision and do tasks, should be idempotent, that is, they should be aware of they previously run, to avoid disaster to happen like format a harddrive that was already initialized, and this will save you lose time in case for any error you provision again the wrong machine.

The scripts I am sharing here, do this, if you provision again box1, you will see the dangerous part are skip:

    alvaros-mini:box1 alvarom$ vagrant provision
    [default] Running provisioner: shell...
    [default] Running: inline script
    --2014-01-02 15:59:02--  http://mini.home.kikitux.net/stage/mini-yum-ol6.repo
    Resolving mini.home.kikitux.net... 192.168.1.2
    Connecting to mini.home.kikitux.net|192.168.1.2|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 4354 (4.3K) [application/octet-stream]
    [default] Running provisioner: shell...
    [default] Running: inline script
    Loaded plugins: security
    Setting up Install Process
    Package oracle-rdbms-server-12cR1-preinstall-1.0-11.el6.x86_64 already installed and latest version
    Nothing to do

The part above is inocous, so not an issue if they run again..

This is the dangerous one..

    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-37890-1icc16o
    /dev/sdc: UUID="b5fc0def-a89e-44f0-8c34-ad2bd44d7e91" TYPE="ocfs2"
    filesystem metadata found on sdc, ignoring
    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-37890-zb3awc
    /dev/sdd: UUID="a10ebdf4-652f-48f6-a9ee-2301175a7dab" TYPE="ocfs2"
    filesystem metadata found on sdd, ignoring
    alvaros-mini:box1 alvarom$

Feel free to review the scripts and check the code, as you can see, they check for filesystem metadata in the device and will skip doing harm if the disks were already initialized.

Let's create box2 using asm instead of ocfs2 for storage:

    alvaros-mini:~ alvarom$ mkdir box2
    alvaros-mini:~ alvarom$ cd box2
    alvaros-mini:box2 alvarom$ cp ../box1/Vagrantfile ../box1/sd*sh .

Edit Vagrantfile, update hostname and ip, replace the scripts at the end.

    alvaros-mini:box2 alvarom$ tail Vagrantfile
        config.vm.provision :shell, :inline => "cd /etc/yum.repos.d ; > public-yum-ol6.repo ; wget http://mini.home.kikitux.net/stage/mini-yum-ol6.repo"
        config.vm.provision :shell, :path   => "preinstall_crs_db.sh"
        config.vm.provision :shell, :path   => "sdc_data_asm.sh"
        config.vm.provision :shell, :path   => "sdd_fra_asm.sh"
    end

These Vagrantfile and scripts are on the github page for box 2 project

We will follow oracle best practices and user separation role, so our preinstall_crs_db.sh script will do a bunch of stuff like checking and installing rpm files, groups, users, set initial password, and a long etc.

Let's start the machine:

    alvaros-mini:box2 alvarom$ vagrant up --provision
    Bringing machine 'default' up with 'virtualbox' provider...
    [default] Importing base box 'oracle65'...
    [default] Matching MAC address for NAT networking...
    [default] Setting the name of the VM...
    [default] Clearing any previously set forwarded ports...
    [default] Fixed port collision for 22 => 2222. Now on port 2200.
    [default] Clearing any previously set network interfaces...
    [default] Preparing network interfaces based on configuration...
    [default] Forwarding ports...
    [default] -- 22 => 2200 (adapter 1)
    [default] Running 'pre-boot' VM customizations...
    [default] Booting VM...
    [default] Waiting for machine to boot. This may take a few minutes...
    [default] Machine booted and ready!
    [default] Setting hostname...
    [default] Configuring and enabling network interfaces...
    [default] Mounting shared folders...
    [default] -- /vagrant
    [default] Running provisioner: shell...
    [default] Running: inline script
    --2014-01-02 16:31:52--  http://mini.home.kikitux.net/stage/mini-yum-ol6.repo
    Resolving mini.home.kikitux.net... 192.168.1.2
    Connecting to mini.home.kikitux.net|192.168.1.2|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 4354 (4.3K) [application/octet-stream]
    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-38466-d5jfr2
    installing oracle-rdbms-server-12cR1-preinstall
    warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
    Importing GPG key 0xEC551F03:
     Userid : Oracle OSS group (Open Source Software group) <build@oss.oracle.com>
     Package: 6:oraclelinux-release-6Server-5.0.2.x86_64 (@anaconda-OracleLinuxServer-201311261449.x86_64/6.5)
     From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
    oinstall:x:54321:
    dba:x:54322:oracle
    id: grid: No such user
    Changing password for user oracle.
    passwd: all authentication tokens updated successfully.
    Changing password for user grid.
    passwd: all authentication tokens updated successfully.
    nothing to unzip
    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-38466-1efniid
    Loaded plugins: fs-snapshot, security
    Setting up Install Process
    Package oracleasm-support-2.1.8-1.el6.x86_64 already installed and latest version
    Package parted-2.1-21.el6.x86_64 already installed and latest version
    Nothing to do
    Configuring the Oracle ASM library driver.

    This will configure the on-boot properties of the Oracle ASM library
    driver.  The following questions will determine whether the driver is
    loaded on boot and what permissions it will have.  The current values
    will be shown in brackets ('[]').  Hitting <ENTER> without typing an
    answer will keep that current value.  Ctrl-C will abort.

    Default user to own the driver interface []: Default group to own the driver interface []: Start Oracle ASM library driver on boot (y/n) [n]: Scan for Oracle ASM disks on boot (y/n) [y]: Writing Oracle ASM library driver configuration: done
    Initializing the Oracle ASMLib driver: [  OK  ]
    Scanning the system for Oracle ASMLib disks: [  OK  ]
    ok: no partition on /dev/sdc
    Marking disk "data" as an ASM disk: [  OK  ]
    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-38466-s2ob16
    Loaded plugins: fs-snapshot, security
    Setting up Install Process
    Package oracleasm-support-2.1.8-1.el6.x86_64 already installed and latest version
    Package parted-2.1-21.el6.x86_64 already installed and latest version
    Nothing to do
    oracleasm configured
    ok: no partition on /dev/sdd
    Marking disk "fra" as an ASM disk: [  OK  ]
    alvaros-mini:box2 alvarom$

Let's verify from the machine how this looks:

    alvaros-mini:box2 alvarom$ vagrant ssh
    Last login: Thu Jan  2 16:41:09 2014 from 10.0.2.2
    [vagrant@box2 ~]$ id grid
    uid=54320(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54318(asmdba),54319(asmoper),54320(asmadmin)
    [vagrant@box2 ~]$ id oracle
    uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54318(asmdba),54323(backupdba),54324(oper),54325(dgdba),54326(kmdba)
    [vagrant@box2 ~]$ sudo service oracleasm listdisks
    DATA
    FRA
    [vagrant@box2 ~]$ ls -al /dev/oracleasm/disks/
    total 0
    drwxr-xr-x 1 root root         0 Jan  2 16:33 .
    drwxr-xr-x 4 root root         0 Jan  2 16:33 ..
    brw-rw---- 1 grid asmadmin 8, 33 Jan  2 16:33 DATA
    brw-rw---- 1 grid asmadmin 8, 49 Jan  2 16:33 FRA
    [vagrant@box2 ~]$

If we run again all these on a provisioned machine, no harm will be done.

    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-38978-putgf6
    installing oracle-rdbms-server-12cR1-preinstall
    seems this part was already executed
    asmdba:x:54318:grid,oracle
    asmoper:x:54319:grid
    asmadmin:x:54320:grid
    oinstall:x:54321:
    dba:x:54322:oracle,grid
    backupdba:x:54323:oracle
    oper:x:54324:oracle
    dgdba:x:54325:oracle
    kmdba:x:54326:oracle
    Changing password for user oracle.
    passwd: all authentication tokens updated successfully.
    Changing password for user grid.
    passwd: all authentication tokens updated successfully.
    nothing to unzip
    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-38978-16l9kfz
    Loaded plugins: fs-snapshot, security
    Setting up Install Process
    Package oracleasm-support-2.1.8-1.el6.x86_64 already installed and latest version
    Package parted-2.1-21.el6.x86_64 already installed and latest version
    Nothing to do
    Configuring the Oracle ASM library driver.

    This will configure the on-boot properties of the Oracle ASM library
    driver.  The following questions will determine whether the driver is
    loaded on boot and what permissions it will have.  The current values
    will be shown in brackets ('[]').  Hitting <ENTER> without typing an
    answer will keep that current value.  Ctrl-C will abort.

    Default user to own the driver interface [grid]: Default group to own the driver interface [asmadmin]: Start Oracle ASM library driver on boot (y/n) [y]: Scan for Oracle ASM disks on boot (y/n) [y]: Writing Oracle ASM library driver configuration: done
    Initializing the Oracle ASMLib driver: [  OK  ]
    Scanning the system for Oracle ASMLib disks: [  OK  ]
    /dev/sdc1: LABEL="DATA" TYPE="oracleasm"
    filesystem metadata found on sdc, ignoring
    [default] Running provisioner: shell...
    [default] Running: /var/folders/_h/d14sw7c55ds3qr05nnp6ckhc0000gn/T/vagrant-shell20140102-38978-bf7aw4
    Loaded plugins: fs-snapshot, security
    Setting up Install Process
    Package oracleasm-support-2.1.8-1.el6.x86_64 already installed and latest version
    Package parted-2.1-21.el6.x86_64 already installed and latest version
    Nothing to do
    oracleasm configured
    /dev/sdd1: LABEL="FRA" TYPE="oracleasm"
    filesystem metadata found on sdd, ignoring

All setup to start installing oracle database and work. No more time wasted on building machines, or stick to hostnames and ip that are hard to use.

Alvaro.

  • oracle 6
  • linux 7
  • vagrant 2