Simulation environment:
1 Storage system (physical storage or software-based storage with protocol of SAN, NAS ...)
1 (or more) testing server.
On server, install vdbench:
Download vdbench : http://www.oracle.com/technetwork/server-storage/vdbench-downloads-1901681.html
This utility require java and csh shell, install it before you can use vdbench.
Stress test template file:
Refer this guide for further testing case (or vdbench home directory for some example): http://www.oracle.com/technetwork/server-storage/vdbench-1901683.pdf
My basic example:
Edit a text file in /home/user/template.vdbench
With:
sd : storage definition (use any: sd1, sd2 ...sdtest...)
lun=/dev/vdb : i use RAW device (that mounted from storage, create LUN or Volume on Storage system and mount it to testing server. There are many kind of storage if you want to stress, disk, raw device, file system etc.)
threads: maximum number of concurrent outstanding I/O that we want to flush.
wd: workload definition (use any)
xfersize: data transfer size
(1M,70, 10M, 30): Generate xfersize as a random value between 1 Megabyte and 10 Megabyte with weight for random value is 70%.
rdpct: read percentage (70% is read and 30% is write).
rd: run definition (use name any)
iorate=max: Run an uncontrolled workload. (iorate=100 : Run a workload of 100 I/Os per second)
elapsed: time to run this test (second)
interval: report interval to your screen in second.
Run command for test:
Change to vdbench directory:
# cd /opt/vdbench
#./vdbench -f /home/user/template.vdbench -o <ouput_directory_for_log>
Watch stressing status on screen.
Tuesday 19 August 2014
Sunday 3 August 2014
LINUX NETWORK INTERFACE BONDING
Summary task: configure and un-configure NIC bonding on Linux
Installation
Step 1: Ensure kernel support
Before Ubuntu can configure your network cards into a NIC bond, you need to ensure that the correct kernel module bonding is present, and loaded at boot time.
Edit your /etc/modules configuration:
Ensure that your network is brought down:
Step 3: Checking and Start up bonding interface
Remove
Bring down the device bond0
Remove slave interface from bond0 device, first eth0
and eth1:
Next, remove the config and files related to bond0 device
Installation
sudo apt-get install ifenslaveInstall ifenslave to attach or detach slave network interface to bonding device
Step 1: Ensure kernel support
Before Ubuntu can configure your network cards into a NIC bond, you need to ensure that the correct kernel module bonding is present, and loaded at boot time.
Edit your /etc/modules configuration:
sudo vi /etc/modulesEnsure that the bonding module is loaded:
# /etc/modules: kernel modules to load at boot time. # # This file contains the names of kernel modules that should be loaded # at boot time, one per line. Lines beginning with "#" are ignored. loop lp rtc bondingStep 2: Configure network interfaces
Ensure that your network is brought down:
sudo stop networking
Then load the bonding kernel module and edit network configuration:
sudo modprobe bonding
sudo vi /etc/network/interfaces
For example, to combine eth0 and eth1 as slaves to the bonding interface bond0 using a simple active-backup setup, with eth0 being the primary interface:
#eth0 is manually configured, and slave to the "bond0" bonded NIC auto eth0 iface eth0 inet manual bond-master bond0 bond-primary eth0 #eth1 ditto, thus creating a 2-link bond. auto eth1 iface eth1 inet manual bond-master bond0 # bond0 is the bonding NIC and can be used like any other normal NIC. # bond0 is configured using static network information. auto bond0 iface bond0 inet static address 192.168.1.10 gateway 192.168.1.1 netmask 255.255.255.0 bond-mode active-backup bond-miimon 100 bond-slaves none
Step 3: Checking and Start up bonding interface
# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008) Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2 (0) MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 802.3ad info LACP rate: fast Aggregator selection policy (ad_select): stable bond bond0 has no active aggregator Slave Interface: eth1 MII Status: up Link Failure Count: 0 Permanent HW addr: 00:0c:29:f5:b7:11 Aggregator ID: N/A Slave Interface: eth2 MII Status: up Link Failure Count: 0 Permanent HW addr: 00:0c:29:f5:b7:1b Aggregator ID: N/ATo bring up the bonding interface, run
ifup bond0To bring down the bonding interface, run
ifdown bond0
Remove
Bring down the device bond0
ifconfig bond0 down
Remove slave interface from bond0 device, first eth0
echo "-eth0" > /sys/class/net/bond0/bonding/slaves
and eth1:
echo "-eth1" > /sys/class/net/bond0/bonding/slaves
Next, remove the config and files related to bond0 device
echo "-bond0" > /sys/class/net/bonding_mastersand
rmmod bonding
Source:
https://help.ubuntu.com/community/UbuntuBonding
Using resize2fs for resizing Linux partition (Centos/Ubuntu)
In vitualization world, sometime, when you create a VM, you assign certain amount of disk space and then you realize that you need more. Resizing a partition is not a easy task, especially is boot partition, you have to reboot the machine to take effect. This memories will take me through some basically step to help me resize linux partition online/offline (this step apply for both Centos and Ubuntu).
Before go through the guidance, let talk a bit to make clear my concept about Linux disk, there include 3 type of disk(or partition) will be mention in this document is physical disk, underlying partition and parttion. Physical disk is the this that we attach to the server (hard disk), underlying partition is just a partition that occupies a whole physical disk (ext: /dev/sda), and partition is a parted disk (/dev/sda1; /dev/sda2).
First, on hypervisor layer, try to extend the physical disk and check by fdisk -l, we should see the disk has ready to extend the partition.
Please NOTE,
1) We can not extend root disk on-line, this will not take effect after we reboot the server. So just extend the physical disk and reboot the server (on Centos, resize2fs is automatically run after boot, otherwise we have to run resize2fs manually with Ubuntu).
2) resize2fs or grow_xfs utilities, as their document, can extend partition without un-mount it, but after test, i realize it can be only done when we using underlying partition. Not effect to other partition.
For offline re-size: (resize partition such as /dev/sda1, /dev/sda2)
In this example, partition will be resized using ext4 format, first, umount the partition and check for error and disable journal as well.
The next step will be stress you up with delete partition steps, we have to delete the partition which we want to extend the capacity, don't worry, we are not going to lose the data, this step just make sure the partition table is updated with new size
Now, recheck the partition and resize it
We almost done here, re-enable something that we throw away from our first step
Now, the partition is ready to mount and use.
For online re-size: (resize underlying partition /dev/sda)
Just run command:
Before go through the guidance, let talk a bit to make clear my concept about Linux disk, there include 3 type of disk(or partition) will be mention in this document is physical disk, underlying partition and parttion. Physical disk is the this that we attach to the server (hard disk), underlying partition is just a partition that occupies a whole physical disk (ext: /dev/sda), and partition is a parted disk (/dev/sda1; /dev/sda2).
First, on hypervisor layer, try to extend the physical disk and check by fdisk -l, we should see the disk has ready to extend the partition.
Please NOTE,
1) We can not extend root disk on-line, this will not take effect after we reboot the server. So just extend the physical disk and reboot the server (on Centos, resize2fs is automatically run after boot, otherwise we have to run resize2fs manually with Ubuntu).
2) resize2fs or grow_xfs utilities, as their document, can extend partition without un-mount it, but after test, i realize it can be only done when we using underlying partition. Not effect to other partition.
For offline re-size: (resize partition such as /dev/sda1, /dev/sda2)
In this example, partition will be resized using ext4 format, first, umount the partition and check for error and disable journal as well.
The next step will be stress you up with delete partition steps, we have to delete the partition which we want to extend the capacity, don't worry, we are not going to lose the data, this step just make sure the partition table is updated with new size
Now, recheck the partition and resize it
We almost done here, re-enable something that we throw away from our first step
Now, the partition is ready to mount and use.
For online re-size: (resize underlying partition /dev/sda)
Just run command:
Monday 30 June 2014
Install and Configure Openvpn
Configure OPENVPN Server:
Assume that this VPN server we have 2 NIC as below:
eth0: use for VPN network with 192.168.1.0/24; gateway .254
eth1: use for management network with 10.0.0.100/24 (or public IP address as you want)
1) Installation
# sudo apt-get install -y openvpn bridge-utils easy-rsa
2) Configuration
# cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf /etc/openvpn/
# vi /etc/openvpn/server.conf
Do this:
3) Create CA certificate and CA key.
# cp -R /usr/share/easy-rsa/* /etc/openvpn/easy-rsa
# cd /etc/openvpn/easy-rsa
# mkdir keys
# vi vars
Do this:
# source ./vars
# ./clean-all
# ./build-ca
4) Create certificate and key for server
# ./build-key-server server
5) Generate Diffie Hellman ( DH ) parameter.
# ./build-dh
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
..........................................................
6) Create certificate and key for client.
# ./build-key-pass client
7) Start OpenVPN
# cp /usr/share/doc/openvpn/examples/sample-scripts/bridge-start /etc/openvpn/openvpn-startup
# cp /usr/share/doc/openvpn/examples/sample-scripts/bridge-stop /etc/openvpn/openvpn-shutdown
# chmod 755 /etc/openvpn/openvpn-startup
# chmod 755 /etc/openvpn/openvpn-shutdown
# vi /etc/openvpn/openvpn-startup
Now start VPN server:
# /etc/rc.d/init.d/openvpn start
Remember, we dont need to run /etc/openvpn/openvpn-startup script to enable bridge, this will be run by /etc/rc.d/init.d/openvpn.
Configure VPN Client (in window):
1) Install Openvpn
Go to http://openvpn.net/index.php/open-source/downloads.html for download and install vpn client.
2) Configure
sFPT or use other way to transfer some below file from vpn server:
/etc/openvpn/easy-rsa/keys/ca.crt
/etc/openvpn/easy-rsa/keys/client.crt
/etc/openvpn/easy-rsa/keys/client.key
To <vpn_client_installed_directory>/config/ in Window desktop
Copy example client configure to /config dir
Copy <vpn_client_installed_directory>\sample-config\client.ovpn to <vpn_client_installed_directory>\config\client.ovpn
and Edit
3) Run OVPN
Right-click on OVPN icon
at taskbar and click "Connect", type password and connect to VPN Server.
Friday 6 June 2014
NGINX - Part 1 - Reverse Proxy Configuration
1_ Reverse Proxy with NginX
The topology for this test:
<Client> ====> [Nginx - Reverse Proxy] ====> [Backend Server]
Install Proxy server on a separate server. Before we get into Nginx configuration, first thing, we should check our server limit configuration for advance test. Run command
If the number is too small, we should change it as below procedure:
1) Open the file /etc/security/limits.conf with your right permission, then paste the following line towards end:
2) Open /etc/pam.d/common-session, Add following line:
//Just forget it if it already had
3) Open /etc/sysctl.conf. Add the following;
And then run:
After this step, you can verify the new limit.
Now, move to configure Nginx working as Proxy server:
Edit /etc/nginx/nginx.conf and paste below configuration:
In nginx.conf we just create a memory caching allocation named "nhutcache" with cache directory on local disk at /proxy_test. So, we should create that directory for store the cache files (and directory for Proxy_temp_path as well).
Next, create virtual site for Nginx connection:
Change directory to virtual site /etc/nginx/sites-available, create a file and paste following :
Note, this file we will include 2 file from outside is proxy.conf and block-country.conf, so that we should create both file and put in same dir with nginx.conf, the content of both file is below:
Proxy.conf
Block-country.conf (this file is optional, i put this because in this test, i use public IP for some case)
Create a link to enable your site:
Now, start Nginx
If you want to mornitor Nginx cached, just:
If you want to show HIT/MISS/EXPIRE status, just run:
1) Installation (on both server Proxy and Backend)
#sudo apt-get update
#sudo apt-get install nginx
2) Configuration
Backend server (Using Nginx also)
After install nginx, edit nginx.conf and put some line
#sudo vi /etc/nginx/nginx.conf
Content of file:
user www-data; worker_processes 4; pid /run/nginx.pid; worker_rlimit_nofile 30000; events { worker_connections 10240; # multi_accept on; } http { access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## Start: Timeouts ## client_body_timeout 10; client_header_timeout 10; keepalive_timeout 5 5; send_timeout 10; ## End: Timeouts ## #server_names_hash_bucket_size 128; # this seems to be required for some vhosts index index.html index.htm index.php; include /etc/nginx/sites-enabled/*; }
Save the file and move to edit virtual host:
Change directory to /etc/nginx/sitess-available, create a file with the name as what you want and fill some thing (as sample below):
This configuration will create a website working under named "backend.nhut.com" with root dir site at /web_data and port listen on port 8888. Some next line, just inside "location" will make your site working as "directory listing".
This purpose of configuration is that we will do a Proxy Stress test using cosbench (or Jmeter, ab... as which as you want). Under /web_test, we will create a directory structure for tressing test purpose.
After create web virtual host, create a link to enable site
and start nginx
You should not got error ^^ for this step (checking error log : #sudo tail -f /var/log/nginx/error.log).
Proxy server (Nginx)
Change directory to /etc/nginx/sitess-available, create a file with the name as what you want and fill some thing (as sample below):
server { listen 8888; server_name backend.nhut.com; root /web_data; location / { include block-country.conf; autoindex on; autoindex_exact_size off; autoindex_localtime on; } }
This configuration will create a website working under named "backend.nhut.com" with root dir site at /web_data and port listen on port 8888. Some next line, just inside "location" will make your site working as "directory listing".
This purpose of configuration is that we will do a Proxy Stress test using cosbench (or Jmeter, ab... as which as you want). Under /web_test, we will create a directory structure for tressing test purpose.
After create web virtual host, create a link to enable site
#sudo ln -s /etc/nginx/sites-available/nhut_backend /etc/nginx/sites-enabled/nhut_backend
and start nginx
#sudo /etc/init.d/nginx start
You should not got error ^^ for this step (checking error log : #sudo tail -f /var/log/nginx/error.log).
Proxy server (Nginx)
Install Proxy server on a separate server. Before we get into Nginx configuration, first thing, we should check our server limit configuration for advance test. Run command
#ulimit -Sn
#ulimit -Hn
If the number is too small, we should change it as below procedure:
1) Open the file /etc/security/limits.conf with your right permission, then paste the following line towards end:
* hard nofile 500000 * soft nofile 500000 root hard nofile 500000 root soft nofile 500000
2) Open /etc/pam.d/common-session, Add following line:
session required pam_limits.so
//Just forget it if it already had
3) Open /etc/sysctl.conf. Add the following;
fs.file-max = 2097152
And then run:
#sysctl -p
After this step, you can verify the new limit.
Now, move to configure Nginx working as Proxy server:
Edit /etc/nginx/nginx.conf and paste below configuration:
user www-data; worker_processes 4; pid /run/nginx.pid; worker_rlimit_nofile 30000; events { worker_connections 10240; # multi_accept on; } http { access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## Start: Timeouts ## client_body_timeout 10; client_header_timeout 10; keepalive_timeout 5 5; send_timeout 10; ## End: Timeouts ## #server_names_hash_bucket_size 128; # this seems to be required for some vhosts #index index.html index.htm index.php; # Cach configure #limit_conn_zone $binary_remote_addr zone=nhutcache:10m; proxy_cache_path /proxy_test/ levels=1:2 keys_zone=nhutcache:100m max_size=300000m inactive=30d; #if inactive value is reached, Cache-status in ngxin cache log will be MISS proxy_temp_path /proxy_test/tmp; log_format cache 'BEGIN-- ***$time_local ' '***Cache-status: $upstream_cache_status ' '***Cache-Control: $upstream_http_cache_control ' '***Remote-addr: $remote_addr ' '***Expires: $upstream_http_expires ' '***Action: "$request" ($status) ($body_bytes_sent Bytes) ' '***HTTP-referer: "$http_referer" ' '***Client-Browser: "$http_user_agent" --END'; access_log /var/log/nginx/cache.log cache; #Test if command #if ($status = ^(403|404)$ ) { #access_log off; #} include /etc/nginx/sites-enabled/*; }
In nginx.conf we just create a memory caching allocation named "nhutcache" with cache directory on local disk at /proxy_test. So, we should create that directory for store the cache files (and directory for Proxy_temp_path as well).
Next, create virtual site for Nginx connection:
Change directory to virtual site /etc/nginx/sites-available, create a file and paste following :
upstream proxytest { server 10.76.0.46:8888; # this is IP of backend server that we just create on step above } server { listen 8888; server_name proxy.nhut.com; #for static file #location ~* ^.+.(jpg|jpeg|gif|png|ico|css|txt|js|htm|html|avi|mpeg)$ { location ~* .(jpg|jpeg|gif|png|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|tar|wav|bmp|rtf|swf|ico|flv|txt|xml|docx|xlsx|js|css)$ { include proxy.conf; #include block-country.conf; proxy_pass http://proxytest; proxy_cache nhutcache; proxy_cache_valid 200 302 301 304 7d; #this config for expiration of statis files with response 200 302 ... #expires 5m; #return 500; }#for any queries to site that begin with "file"
location ~ ^/file { include proxy.conf; #include block-country.conf; proxy_pass http://proxytest; proxy_cache nhutcache; proxy_cache_valid 200 302 301 304 7d; #if this value (7d) is reached, Cache-status in nginx cache log will be EXPIRED #return 500; } #For all remain queries location / { include proxy.conf; #include block-country.conf; proxy_pass http://proxytest; proxy_cache nhutcache; proxy_cache_valid 200 302 301 60m; proxy_cache_valid 404 403 1m; #return 500; } }
Note, this file we will include 2 file from outside is proxy.conf and block-country.conf, so that we should create both file and put in same dir with nginx.conf, the content of both file is below:
Proxy.conf
proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_ignore_headers Expires Cache-Control; client_max_body_size 10m; client_body_buffer_size 128k; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffer_size 4k; proxy_buffers 32 4k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k;
Block-country.conf (this file is optional, i put this because in this test, i use public IP for some case)
allow <IP of range/mask>; allow ......./24; allow 127.0.0.1; deny all;
Create a link to enable your site:
#sudo ln -s /etc/nginx/sites-available/nhut_proxy /etc/nginx/sites-enabled/nhut_proxy
Now, start Nginx
#sudo /etc/init.d/nginx start
If you want to mornitor Nginx cached, just:
#sudo tail -f /var/log/nginx/cache.log
If you want to show HIT/MISS/EXPIRE status, just run:
#sudo awk '{print $5}' /var/log/nginx/cache.log | sort | uniq -c | sort -r
Subscribe to:
Posts (Atom)