Following steps describes how to use MySQL on EBS volume with OpenSolaris based AMI.

1. Create a new EBS Volume.

Click here for ec2-create-volume API help.

2. Create a new MySQL 5.1 AMI Instance based on OpenSolaris.

Click here for more details on this AMI.

3. Attach EBS volume to this AMI instance

In the above command -d option specifies the device number. In OpenSolaris devices are named from 0 to 23. Device number 0 and 1 are used for ephemeral storage. Therefore, we can use the devices starting from 2 until 23.

Click here for ec2-attach-volume API help.
Click here for ec2-describe-volumes API help.

4. Create a ZFS storage pool on the previously attached EBS volume.

Once the instance is up and running, login into the instance using ssh with your key-pair.
To verify new EBS volume attached, execute the format command and press Control + C to come out.

Create a ZFS storage pool on the EBS volume.

Move MySQL on ZFS storage pool based on EBS

Make sure MySQL service is disabled.

If MySQL service is running make sure to disable it using  svcadm disable svc:/application/mysql:default

Move MySQL installation directory to /ebsmysql

Create a soft link to /ebsmysql/mysql/ under /usr/local with the name ‘mysql’

Enable MySQL service

Test MySQL

5. Detach EBS volume from AMI instance

6. Re-attach EBS volume to the AMI instance

Import the pool

7. Add another EBS Volume to pool

Important Links:

MySQL on ZFS:
http://dev.mysql.com/tech-resources/articles/mysql-zfs.html

EBS With OpenSolaris AMI:
http://blogs.sun.com/ec2/entry/ebs_is_supported_on_opensolaris

Amazon EBS, OpenSolaris, and ZFS Guide to Getting Started
http://blogs.sun.com/ec2/resource/OpenSolaris-ZFS-EBS-v1.3.pdf

Let’s start with launching a latest hardened AMP stack AMI (ami-e0b05389) instance.
Once the instnace is up and running, open the terminal client and connect to the instance using ssh.

Edit the /etc/ssh/sshd_config file and set AllowTcpForwarding to yes

AllowTcpForwarding yes

Restart ssh service

svcadm restart svc:/network/ssh:default

Comment out skip-networking from /etc/mysql/my.cnf and save the file

#skip-networking

Restart MySQL service

svcadm restart svc:/application/database/mysql:version_51

exit the ssh connection and recreate ssh connection using following command.

ssh -i <keypair> -L 3307:localhost:3306 root@ec2-xxx-xxx-xxx-xx.compute-1.amazonaws.com

Keep the terminal opened, and start MySQL Query Browser / MySQL administrator from your local machine. and  connect to the MySQL server on remote EC2 instance via the port 3307, as shown in the following sanpshots.

When done close the ssh connection and it should close the ssh tunnel.

First Step is to setup xVM Hypervisor on the OpenSolaris 2009.06 machine.

Follow the steps mentioned at following link and setup Dom0 OpneSolaris 2009.06.

http://hub.opensolaris.org/bin/view/Community+Group+xen/2008_11_dom0

Create a RAW disk image for the DomU installation

mkdir /centos53-pv/
cd /centos53-pv/
dd if=/dev/zero of=CentOS-5.3-x86_64.img bs=1k seek=10000k count=1

Enable xvm/virtd service on Dom0

svcadm enable svc:/system/xvm/virtd:default

Begin the Paravirtualized CentOS DomU installation with the virt-install command

virt-install -n centos53_x86_64 \
-r 1024 \
-f /centos53-pv/CentOS-5.3-x86_64.img \
-p --nographics \
-l http://mirrors.kernel.org/centos/5.3/os/x86_64/

In the above command -n is for the name of the domu, -r is for the memory, -f is for specifying location of the raw disk on which installation will be performed, -p is for specifying paravirtualized mode and -l is for the location containing an installable distribution image.

Follow the regular CentOS installation steps and reboot the DomU when prompted.

Following snapshots walk you through CentOS DomU installation.

After reboot enable ssh login for root user on DomU

uncomment the line “PermitRootLogin yes” from “/etc/ssh/sshd_config” file and save it.

Extract kernel and ramdisk from the running DomU image

Copy “/boot/vmlinuz-2.6.18-128.el5xen” and “/boot/initrd-2.6.18-128.el5xen.img” files to Dom0 machine at “/centos53-pv/”.

scp  /boot/vmlinuz-2.6.18-128.el5xen root@dom0-ip:/centos53-pv/
scp  /boot/initrd-2.6.18-128.el5xen.img root@dom0-ip:/centos53-pv/

Halt the DomU.

Shutdown domU issuing halt command inside domU and remove a domain from Xend domain management using following command.

# xm delete centos53_x86_64

Create a domain configuration file as given below.

Create “/centos53-pv/domu.py” file as given below

# cat domu_64.py
name = "centos53_x86_64"
memory = "2058"
kernel = "/centos53-pv/vmlinuz-2.6.18-128.el5xen"
ramdisk = "/centos53-pv/initrd-2.6.18-128.el5xen.img"
disk = ['file:/centos53-pv/CentOS-5.3-x86_64.img,xvda,w']
vif = ['']

Relaunch the DomU by passing extracted kernel and ramdisk images using py script.

# xm create domu.py
Using config file "./domu.py".
Started domain centos53_x86_64

Access domU console

# xm console centos53_x86_64

Cent OS 5.3 Paravirtualized DomU VNC Session on OpenSolaris 2009.06 Dom0

I was looking for the free iPhone application to manage my ec2 account and came across this article: “Mobile Monitoring and Administration of Amazon Web Services: Developing an iPhone-Optimized Web Application”.

This article not only describes in details each steps on how to begin with developing iPhone application to manage AWS but also provides source code that you easily download from Google Code Repository and setup in-house.

In this blog entry I will walk you through steps on setting up this application on our popular OpenSolaris AMP Stack AMI.

Following Steps are also valid for the setup of “iPhone AWS Manager” on local machine with OpenSolaris and AMP Stack.

Lets begin with launching new OpenSolaris AMP Stack AMI Instance.
ec2-run-instances ami-c7cf28ae -k my-keypair

Get the public DNS address of an Instance and login into with your keypair
ec2-describe-instances i-xxxxxxxx
ssh -i <path_to_your_keypair> root@ec2-xx-xxx-xx-xxx.compute-1.amazonaws.com

Install SVN Package from the Repository
pkg refresh
pkg install SUNWsvn

Checkout iPhone AWS Manager code form the Google Code Repository
cd /var/apache2/2.2/htdocs/

svn co http://iphone-aws-manager.googlecode.com/svn/trunk/ aws-iphone
A aws-iphone/amis.php
A aws-iphone/class.s3.php
A aws-iphone/reboot.php
A aws-iphone/deleteobj.php
A aws-iphone/contents.php
A aws-iphone/getbuckets.php
A aws-iphone/delete.php
A aws-iphone/create.php
A aws-iphone/run.php
A aws-iphone/index.php
A aws-iphone/about.php
A aws-iphone/size.php
A aws-iphone/class.ec2.php
A aws-iphone/lib
A aws-iphone/lib/iui
A aws-iphone/lib/iui/backButton.png
A aws-iphone/lib/iui/grayButton.png
A aws-iphone/lib/iui/loading.gif
A aws-iphone/lib/iui/pinstripes.png
A aws-iphone/lib/iui/toolbar.png
A aws-iphone/lib/iui/iuix.js
A aws-iphone/lib/iui/toolButton.png
A aws-iphone/lib/iui/whiteButton.png
A aws-iphone/lib/iui/iui.js
A aws-iphone/lib/iui/toggleOn.png
A aws-iphone/lib/iui/thumb.png
A aws-iphone/lib/iui/toggle.png
A aws-iphone/lib/iui/iuix.css
A aws-iphone/lib/iui/blueButton.png
A aws-iphone/lib/iui/cancel.png
A aws-iphone/lib/iui/selection.png
A aws-iphone/lib/iui/listArrow.png
A aws-iphone/lib/iui/listGroup.png
A aws-iphone/lib/iui/iui.css
A aws-iphone/lib/iui/listArrowSel.png
A aws-iphone/iPhone AWS Source Code.zip
A aws-iphone/instances.php
A aws-iphone/terminate.php
Checked out revision 3.

Set the Document Root in the httpd.conf file
vi /etc/apache2/2.2/httpd.conf

Set the DocumentRoot as given below
DocumentRoot “/var/apache2/2.2/htdocs/aws-iphone”

edit /etc/php/5.2/php.ini and set following variables to On
register_globals = On
register_long_arrays = On

Create “key.php” file at “/var/apache2/2.2/htdocs/aws-iphone/keys.php” with following code

<?php
define('my_access_key','XXXXXXXXXXXXXXXXXXXX');
define('my_secret_key','XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX');
?>

Edit “/var/apache2/2.2/htdocs/aws-iphone/index.php” file and replace php code block at the top of the page with following code.

<?php
// start the session
session_start();
// include the EC2 and S3 libraries
require_once("class.ec2.php");
require_once("class.s3.php");
include_once("keys.php");
// create session variables for AWS keys
session_register("AMAZON_KEY");
session_register("AMAZON_PRIVATE_KEY");
$HTTP_SESSION_VARS["AMAZON_KEY"]=my_access_key;
$HTTP_SESSION_VARS["AMAZON_PRIVATE_KEY"]=my_secret_key;
?>

change your keypair in “/var/apache2/2.2/htdocs/aws-iphone/class.ec2.php” file
look for the function definition and replace your keyName

function runInstances($imageId, $min = 1, $max = 1, $keyName = "divyen-personal-aws-key")

Restart the Apache Server
svcadm disable svc:/network/http:apache22
svcadm enable svc:/network/http:apache22

Open Safari Browser in your iPhone and type in EC2 Instance DNS Address

Create a Shortcut icon so that you don’t need to type in URL every time

Snapshots of the basic operations supported on this application are described as given below

Describe AMIs – Both Public and Private

Create Instance, Reboot Instance, Terminate Instance

Create Bucket, Delete Bucket

Enable following desktop related services

Set xvm_vnc property to true for x11-server SMF service.

Set the root password, you may need it to get back to your session in case if screen saver starts.

Setup VNC server

Create .vnc directory under /root if it does not exist

Create /root/.vnc/xstartup file with following lines.

Note: if xstartup file is already exist, replace it with above lines to start gnome terminal session for VNC.

Provide execute permission on xtartup file.

Start the vncserver and set the password.

Authorize VNC Port
Here we are done with configuring EC2 instance for GUI Remote Access.
Finally we need to authorize the corresponding VNC port on users machine where VNC client is installed.

Screen Shots
VNC client connection configuration

Use public DNS address associated with EC2 instance as the host name.

VNC client connected to OpenSolaris 2009.06 EC2 Instance.

For more details on OpenSolaris AMIs visit http://blogs.sun.com/ec2/

I assume
- you are running OpenSolaris 2008.11 build rc2 dom0.
- ec2 ami/api tools installed at /opt/ec2
- your ec2 certificate and private key files available at /opt/ec2/keys
- your ec2 keypair file located at /opt/ec2

Login in with your keypair file

ssh -i /opt/ec2/your-keypair root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com

[ Step-2 ]
Remove EC2 specific stuff from the instance.

Open “/etc/ssh/sshd_config” and replace “PermitRootLogin without-password” with “PermitRootLogin yes” and save the file.

Set the root password

passwd
passwd: Changing password for root
New Password:
Re-enter new Password:
passwd: password successfully changed for root

Disable and remove ec2 specific services

svcadm disable svc:/ec2/sshkey:default
svcadm disable svc:/ec2/fixes:default
svcadm disable svc:/ec2/mount:default

svccfg delete svc:/ec2/sshkey:default
svccfg delete svc:/ec2/fixes:default
svccfg delete svc:/ec2/mount:default

[ Step-3 ]
Log out of the ec2 instance and transfer your keys at /mnt/keys to begin re-bundling image.

scp -i /opt/ec2/id-your-keypair -r /opt/ec2/keys/ root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com:/mnt

[ Step-4 ]
Login back to ec2 instance.

ssh -i /opt/ec2/your-keypair root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com

[ Step-5 ]
Start bundling the image.

# cd /mnt
# /opt/ec2/sbin/rebundle.sh -v opensolaris_2008_11_ami.img
Creating image file /mnt/opensolaris_2008_11_ami.img
Preparing clone pool on /dev/lofi/1
Cloning
#

rebundle.sh will take about 10-15 minutes to create raw opensolaris image.

[ Step-6 ]
Bundle image and create parts.

export BUCKET=my-bucket
export JAVA_HOME=/usr/jdk/latest
export EC2_HOME=/opt/ec2
export PATH=$PATH:$EC2_HOME/bin
export RUBYLIB=$EC2_HOME/lib
export EC2_URL=https://ec2.amazonaws.com

export EC2_PRIVATE_KEY=/mnt/keys/pk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pem
export EC2_CERT=/mnt/keys/cert-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pem

export EC2_KEYID=xxxxxxxxxxxxxxxxxxxxxxx
export EC2_KEY=”xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx”

export DIRECTORY=/mnt
export IMAGE=opensolaris_2008_11_ami.img

# ec2-bundle-image -c $EC2_CERT -k $EC2_PRIVATE_KEY \
–kernel aki-6552b60c –ramdisk ari-6452b60d \
–block-device-mapping “root=rpool/52@0,ami=0,ephemeral0=1″ \
–user <user-id> –arch i386 \
-i $DIRECTORY/$IMAGE -d $DIRECTORY/parts

[ Step-7 ]
Upload image parts on s3

# cd $DIRECTORY/parts

# ec2-upload-bundle -b $BUCKET -m opensolaris_2008_11_ami.img.manifest.xml \
–url http://s3.amazonaws.com \
–retry -a $EC2_KEYID -s $EC2_KEY

[ Step-8 ]
Log out of the instance.
Download image parts and unbundle it to re-create the raw image locally.

# mkdir /osol-ec2-image
# mkdir /osol-ec2-image/parts
# cd /osol-ec2-image/parts

# ec2-download-bundle -b my-bucket -m opensolaris_2008_11_ami.img.manifest.xml \
-a <your-access-key-id> -s <your-secret-key-id> \
–privatekey /opt/ec2/keys/pk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx .pem

# ec2-unbundle -m opensolaris_2008_11_ami.img.manifest.xml \
-k /opt/ec2/keys/pk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx .pem

# mv opensolaris_2008_11_ami.img
# cd /osol-ec2-image/
[ Step-9 ]
Create “domu.py” at /osol-ec2-image with following lines

[ Step-10 ]
Launch domu image and login with root user and the password you set during step-2

xm create domu.py

Using config file “./domu.py”.
Started domain opensolaris-ec2-image

xm console opensolaris-ec2-image
v3.1.4-xvm chgset ‘Mon Oct 13 22:14:51 2008 -0700 15904:94bd0a643efe’

SunOS Release 5.11 Version snv_101b 32-bit

Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.

Use is subject to license terms.
Hostname: dhcp-umpk16-85-23
Reading ZFS config: done.
Mounting ZFS filesystems: (5/5)

dhcp-umpk16-85-23 console login:

Create a file default.conf at /etc/vpnc/ with following lines and
fill in the required parameters replacing strings enclosed within “<” and “>” without keeping “<" and ">”.

Note: Make sure you don’t append extra space at the end of parameter value.

Connect using “vpnc-connect” command.

To disconnect type in following command.

Lets begin with the installation of xVM Hypervisor on OpenSolaris 2008.11.

You can follow exact steps mentioned on Dileep Kumar’s blog – “How to create domU in OpenSolaris 2008.05“. These steps are also valid for OpenSolaris 2008.11.

Once dom0 is up and running, walk through the below given steps to create a fully virtualized ubuntu 9.04 DomU. Fully virtualized DomU is also known as HVM Domain (Hardware-assisted Virtual Machine Domain), with that you can run unmodified guest operating system without the knowledge of underlying hypervisor.

Step-1: Download and save ubuntu 9.04 ISO file at /ubuntu-hvm

mkdir /ubuntu-hvm
cd /ubuntu-hvm
wget http://ftp.ucsb.edu/pub/mirrors/linux/ubuntu/jaunty/ubuntu-9.04-desktop-i386.iso

Step-2: Create a virtual Disk for the guest domain.

dd if=/dev/zero of=/ubuntu-hvm/ubuntu-9.04-desktop-i386.img bs=1k seek=10000k count=1

Step-3: Create guest configuration file install.py with following script at /ubuntu-hvm

Detailed explanation on each lines used in the script is given here.

disk = ['file:/ubuntu-hvm/ubuntu-9.04-desktop-i386.img,hda,w','file:/ubuntu-hvm/ubuntu-9.04-desktop-i386.iso,hdc:cdrom,r']
memory = 1024
vcpus=1
name = "ubuntu-904-hvm-install"
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
vif = [ '' ]
on_poweroff = 'destroy'
on_reboot   = 'destroy'
on_crash    = 'destroy'
boot='dc'
vnc=1
sdl=0
vncdisplay=5
import os, re
arch = os.uname()[4]
if re.search('64', arch):
arch_libdir = 'lib64'
else:
arch_libdir = 'lib'
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'

Launch the guest domain to begin the installation.

cd /ubuntu-hvm/
xm create install.py

Bring up VNC Session to proceed with the installation. In the install.py script we specified to bind VNC on port 127.0.0.1:5

vncviewer 127.0.0.1:5

If HVM guest is unsupported on your system, you may see following error message.
Error: HVM guest support is unavailable: is VT/AMD-V supported by your CPU and enabled in your BIOS?

Step-4: Install guest domain.

Snap-01

Snap-02

Snap-03

Snap-04

Snap-05

Do not click on the restart. Follow step-5 instead.

Step-5: Destroy the guest domain and create new configuration file to boot the domU from the disk image.

xm destroy ubuntu-904-hvm-install
cd /ubuntu-hvm/

Create new configuration file “domu.py” with following script at /ubuntu-hvm

disk = ['file:/ubuntu-hvm/ubuntu-9.04-desktop-i386.img,hda,w']
memory = 1024
vcpus=1
name = "ubuntu-904-hvm-guest"
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
vif = [ '' ]
on_poweroff = 'destroy'
on_reboot   = 'destroy'
on_crash    = 'destroy'
boot='c'
vnc=1
sdl=0
vncdisplay=5
import os, re
arch = os.uname()[4]
if re.search('64', arch):
arch_libdir = 'lib64'
else:
arch_libdir = 'lib'
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'

Step-6: Finally launch the HVM guest domain

cd /ubuntu-hvm/
xm create domu.py

Bring up VNC Session

vncviewer 127.0.0.1:5

Snap-05

Snap-06

Automated Time Table Management System generates intelligent time table that manages faculty work loads on the basis of lecture requirements.

System allows entering course details, years and semester details, visiting faculty preferences and subject load per week by respective faculties. Based on input data, engine generates time table with shared work load. System is customized to set working days and number of lectures per day.

Client part installed on each faculty desk gives reminder for the next lecture before customized period of time

Technologies:
Visual Basic, SQL Server, Microsoft Transaction Server(MTS), Crystal Repot 8.5