Using JMeter for DB (Apache Derby) benchmarking

Recently I had to do check or I was for support in a case where the Apache Derby was used in embedded mode. The questions were related to performance, demiensioning and also some non-functional aspects like backup and restore. Going through documentation is one thing but being able to verify some performance figures is another. This article will describe how I used the JMeter to setup environment measuring reads, writes and doing the backup in the background (to check the impact on processed transactions) for the Apache Derby working in the embedded mode (the external one is out of scope of this article).

Obtaining JMeter

Although the JMeter is available in the repos for most popular distribtions (e.g. Ubuntu), I recommend to download it from the webpage (newer version, more features etc.): http://jmeter.apache.org/ - the installation is about unzipping an archive (you can use e.g. unzip).

Instaling JMeter

After unpacking you will need to add the jdbc driver to the classpath - in my case it basically meant that I needed to copy the derby.jar into the lib subdirectory of jmeter. If you already have a derby database on which you want to do benchmark (my case) it would be better to find the Derby version matching the one which was used to create the database. As I mentioned before in this article the embedded mode for derby will be used.

Install additional chart/monitoring JMeter plugins

There is a set of nice, additional plugins available on google code: http://code.google.com/p/jmeter-plugins/ that I would recommend to install. Not only they provide nicer charting listeners but also monitoring capabilities (each system to be monitor will need to have a special agent started). The installation procedure is described in the README and on the webpage.
What I did is after downloading I unpacked the archive to a subdirectory under JMETER_HOME. Next I copied the JMeterPlugins.jar to the lib/ext subdirectory. That's it, the new plugins should be available now.

Building the testcase

Ok, now you are ready to prepare the testcase.

1. Start the JMeter:
# $JMETER_HOME/bin/jmeter.sh

2. Add the database (ApacheDerby) connection definition (JDBC Connection Definition) as follows:

The database URL in case of embedded derby shall contain the path to the database location on the filesystem.
The JDBC Driver class is the driver class - here a specific one for Derby.
In addition one needs to provide username and password for connecting to the database.

3. Add thread pools and samplers

I added three different thread pools: one for testing the reads, one for testing the writes and one (single threaded) for backup. As you see all are using the same JDBC connection definition - defined on the global level

4. Add JDBC request samplers

Inside each thread pool you should define the JDBC request sampler, responsible for generating the appropriate traffic type. Below you can find example used by me for read, write and starting the backup:

The JDBC request sampler for read access

The JDBC request sampler for write access

The JDBC request sampler for initiating the backup

5. Add listeners for summary, CPU, memory, IO statistics.

For getting the summary you should add to each thread group the so called 'Aggregate report' listener. It will provide statistics like: number of samples, average response time, q90, median, min, max, error rate and throughput.

Example aggregate report listener for read access

In addition on the top level you should add the collectors for CPU, memory and disk, network - these are available after installing the additional google code plugin in the Listener section. The important thing is you need to start the agent on the monitored node as follows:



$ $JMETER_PLUGIN/serverAgent/startAgent.sh 
JMeterPlugins Agent version 1.4
No port specified, the default value is used: 4444
------ File Systems init: ------
File System detected: /dev/mapper/krystianek-root
File System detected: /dev/sda1
--------------------------------
--- Network Interfaces init: ---
Network interface detected: lo
Network interface detected: wlan0
Network interface detected: virbr1
Network interface detected: virbr2
Network interface detected: virbr0
Network interface detected: tun0
Network interface detected: eth0
--------------------------------
Waiting for incoming connections...
Client id=0 connected!
Client id=1 connected!
Client id=2 connected!
Client id=3 connected!
Client id=3 disconnected!
Client id=0 disconnected!
Client id=1 disconnected!
Client id=2 disconnected!

And the statistics collector shall be configured to connect to that agent:

Example listener for collecting disk, network statistics

Example listener for collecting CPU statistics

I also recommend to put the Transaction per second listener from google code to each thread pool section, which will provide nice charts for per second statistics.

Example lstener providing graphical represenation of throughput per second

Now you just have to start the execution of the test and do the measurements;)

Resizing root filesystem: KVM, LVM and EXT4

Ever faced a problem that you run out of the root filesystem space? I recently faced that problem on my KVM CentOS 6.0 box (a lot of experiments done on it) and wanted to solve it without reinstallation of the system. It seems to be possible without losing any data and not that difficult on all levels:

• KVM virtual disk size
• LVM volume group size
• EXT4 filesystem size

Below you can find the instruction on how to do that:

1. Resize the KVM virtio disk image. First one need to resize the virtual disk for the KVM. In order to do that you shoudl create a new add on raw disk and append it to the existing disk image. I recommend to first move the original disk for the backup purposes. The steps to be done are shown below:


root@krystianek:/var/lib/libvirt/images# qemu-img create -f raw addon.raw 4G
Formatting 'addon.raw', fmt=raw size=4294967296


root@krystianek:/var/lib/libvirt/images# mv centos6.img centos6.img.backup
root@krystianek:/var/lib/libvirt/images# cat centos6.img.backup addon.raw > centos6
centos6.img.backup    centos6-oracle-1.img  centos6-oracle-2.img  centos6-oracle.img    
root@krystianek:/var/lib/libvirt/images# cat centos6.img.backup addon.raw > centos6.img

2. Boot from  LiveCD. It is required since you will be modyfing the root filesystem's volume group if you plan to modify another volume group/filesystem it may not be required. I used the CentOS LiveCD in order to do that.

3. Create a new partition. Create a new partition from the free disk space (4G). In my case I created a /dev/vda3 partition using the CentOS graphical disk utility.

4. Extend the LVM volume group. Add the newly /dev/vda3 device to the volume group

[root@livecd ~]# vgdisplay 
  --- Volume group ---
  VG Name               vg_centos6hosta
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               7.51 GiB
  PE Size               4.00 MiB
  Total PE              1922
  Alloc PE / Size       1922 / 7.51 GiB
  Free  PE / Size       0 / 0   
  VG UUID               DIeDAo-5Y1O-GEC0-JCb6-keUy-8Yc1-MhRBdN
   
[root@livecd ~]# vgextend vg_centos6hosta /dev/vda3 
  No physical volume label read from /dev/vda3
  Physical volume "/dev/vda3" successfully created
  Volume group "vg_centos6hosta" successfully extended
[root@livecd ~]# vgdisplay 
  --- Volume group ---
  VG Name               vg_centos6hosta
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               1
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               11.50 GiB
  PE Size               4.00 MiB
  Total PE              2945
  Alloc PE / Size       1922 / 7.51 GiB
  Free  PE / Size       1023 / 4.00 GiB
  VG UUID               DIeDAo-5Y1O-GEC0-JCb6-keUy-8Yc1-MhRBdN
   
[root@livecd ~]# lvresize -L+3995M /dev/mapper/vg_centos6hosta-lv_root 
  Rounding up size to full physical extent 3.90 GiB
  Extending logical volume lv_root to 8.50 GiB
  Logical volume lv_root successfully resized
[root@livecd ~]# 

5. Resize the filesystem residing on the LVM volume. Last but not least you have to resize the filesystem residing on the LVM volume group. For ext4 you can do it as follows (where vg_centos6hosta-lv_root is the root fs volume group):

[root@livecd ~]# e2fsck -f /dev/mapper/vg_centos6hosta-lv_root 
e2fsck 1.41.12 (17-May-2010)
_CentOS-6.0-x86_: recovering journal
Clearing orphaned inode 131145 (uid=27, gid=27, mode=0100600, size=0)
Clearing orphaned inode 131144 (uid=27, gid=27, mode=0100600, size=0)
Clearing orphaned inode 131140 (uid=27, gid=27, mode=0100600, size=0)
Clearing orphaned inode 131136 (uid=27, gid=27, mode=0100600, size=0)
Clearing orphaned inode 131130 (uid=27, gid=27, mode=0100600, size=0)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

_CentOS-6.0-x86_: ***** FILE SYSTEM WAS MODIFIED *****
_CentOS-6.0-x86_: 139003/303104 files (0.1% non-contiguous), 1195597/1206272 blocks
[root@livecd ~]# resize2fs /dev/mapper/vg_centos6hosta-lv_root 
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/mapper/vg_centos6hosta-lv_root to 2229248 (4k) blocks.
The filesystem on /dev/mapper/vg_centos6hosta-lv_root is now 2229248 blocks lo

CentOS: Creating rpm packages (ceph-0.38)

This article describes step by step how to create own packages on the example of the ceph package (0.38)

1. Create the directory structure - the easiest way is to install the rpmdevtools and use it to create the structure:

[root@localhost /]# yum install rpmdevtools

2. Download the sources as a tar balls and put it to the SOURCES directory:

[root@localhost /]# cd ~/rpmbuild/SOURCES
[root@localhost rpmbuild]# wget http://ceph.newdream.net/download/ceph-0.38.tar.gz

3. Prepare and customize the spec. If you want to pass some options to the configure utility just customize the configure options and specify the directories to be included in the rpm package (all marked with red color below). Check also if the source tar archive name matches the one placed in the SOURCES folder.


[root@localhost rpmbuild]# vim ./SPECS/ceph.spec 
Name: ceph
Version: 0.38
Release: 1%{?dist}
Summary: Krystianek's package


Group: HA-experiments
License: GPL
URL:        http://http://ceph.newdream.net/
Source0: ceph-0.38.tar.gz
BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)


%description
CEPH utilities 


%prep
%setup -q


%build
%configure --without-tcmalloc
make %{?_smp_mflags}


%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT


%clean
rm -rf $RPM_BUILD_ROOT


%files
%defattr(-,root,root,-)
%doc

/etc/
/var/
/usr/



%changelog


4. Create the rpm. Go to the directory structure root and invoke the rpmbuild to create the rpm

[root@localhost rpmbuild]# cd ~/rpmbuild
[root@localhost rpmbuild]# rpmbuild -ba SPECS/ceph.spec

You might need to set the mask for RPATH to ignore the errors regarding the standard paths:

[root@localhost rpmbuild]# QA_RPATHS=$[ 0x0001|0x0010 ] rpmbuild -ba SPECS/ceph.spec

5. Install the previously create rpm.


[root@localhost rpmbuild]# rpm -i ./RPMS/x86_64/ceph-0.38-1.el6.x86_64.rpm


Remarks: For cuilding the ceph you need additional development packages for tcmalloc (part of google performance, can be downloaded from http://code.google.com/p/google-perftools/) and libatomic (available in the standard CentOS repo as libatomic_ops-devel)

CentOS 6.0: Updating the kernel to 3.0.8

I wanted to continue the work on the clustered filesystems and found out that linux kernel since 2.6.37 provides additional cluster filesystem ceph (http://ceph.newdream.net/) to the previosuly available gfs (gfs2), ocfs2. The description sounds promising so why not to give it a try.
However in my virtualized environment I had only CentOS'es (one running the latest 6.0) with much older kernel version (2.6.32). The standard repositories does not contain any updates and the only way to run a newer version is to download it from kernel.org and compile it by yourself. The instruction below describes the steps that I did in order to have the kernel 3.0.8 on the CentOS 6.0.

1. Prepare disk space for kernel sources, compilation output and produced rpm packages (about 5GB needed for creating the binary package, header and sources).
2. Download the sources from the kernel.org and unpack them. You can do it on the fly using the following command:

[root@localhost tmp]# wget -O - http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.0.8.tar.bz2 | bzip2 -d - | tar -xvf -

3. Now copy your existing configuration file and adapt it to your needs as described in step 4 (I used the menuconfig and loaded the file):

[root@localhost linux-3.0.8]# cp /boot/config-2.6.32-71.29.1.el6.x86_64 .
[root@localhost linux-3.0.8]# make menuconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/lxdialog/checklist.o
  HOSTCC  scripts/kconfig/lxdialog/inputbox.o
  HOSTCC  scripts/kconfig/lxdialog/menubox.o
  HOSTCC  scripts/kconfig/lxdialog/textbox.o
  HOSTCC  scripts/kconfig/lxdialog/util.o
  HOSTCC  scripts/kconfig/lxdialog/yesno.o
  HOSTCC  scripts/kconfig/mconf.o
  SHIPPED scripts/kconfig/zconf.tab.c
  SHIPPED scripts/kconfig/lex.zconf.c
  SHIPPED scripts/kconfig/zconf.hash.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/mconf
scripts/kconfig/mconf Kconfig
#
# configuration written to config-3.0.4
#




*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.


[root@localhost linux-3.0.8]# 

4. Apply the changes. I have added support for ceph, ocfs2 and added the module to dump the current kernel config to the /proc filesystem.

5. Now you are ready to compile and prepare the rpms containing the new kernel. I used the target to build only binary packages (you can also create the sources package). The rpms will be created in user's home directory: $HOME/rpmbuild/:


[root@localhost linux-3.0.8]# make help | grep rpm
  rpm-pkg             - Build both source and binary RPM kernel packages
  binrpm-pkg          - Build only the binary kernel package

[root@localhost linux-3.0.8]# time make binrpm-pkg

/bin/sh /mnt/tmp/linux-3.0.8/scripts/package/mkspec prebuilt > /mnt/tmp/linux-3.0.8/binkernel.spec

make KBUILD_SRC=

  CHK     include/linux/version.h

  UPD     include/linux/version.h

  CHK     include/generated/utsrelease.h

  UPD     include/generated/utsrelease.h

  CC      kernel/bounds.s

  GEN     include/generated/bounds.h

  CC      arch/x86/kernel/asm-offsets.s

  GEN     include/generated/asm-offsets.h

  CALL    scripts/checksyscalls.sh

  HOSTCC  scripts/genksyms/genksyms.o

  SHIPPED scripts/genksyms/lex.c

  SHIPPED scripts/genksyms/parse.h

  SHIPPED scripts/genksyms/keywords.c

  HOSTCC  scripts/genksyms/lex.o

  ...

Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1

Processing files: kernel-headers-3.0.8-1.x86_64

Provides: kernel-headers = 3.0.8

Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(VersionedDependencies) <= 3.0.3-1

Obsoletes: kernel-headers

Checking for unpackaged file(s): /usr/lib/rpm/check-files /root/rpmbuild/BUILDROOT/kernel-3.0.8-1.x86_64

Wrote: /root/rpmbuild/RPMS/x86_64/kernel-3.0.8-1.x86_64.rpm

Wrote: /root/rpmbuild/RPMS/x86_64/kernel-headers-3.0.8-1.x86_64.rpm

Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.VjVenh

+ umask 022

+ cd /mnt/tmp/linux-3.0.8

+ rm -rf /root/rpmbuild/BUILDROOT/kernel-3.0.8-1.x86_64

+ exit 0

real 85m20.491s

user 59m40.551s

sys 12m32.836s

[root@localhost linux-3.0.8]# ls -lh /root/rpmbuild/RPMS/x86_64/

-rw-r--r-- 1 root root 262M Nov  5 19:13 kernel-3.0.8-1.x86_64.rpm

-rw-r--r-- 1 root root 786K Nov  5 19:13 kernel-headers-3.0.8-1.x86_64.rpm

6. Install the new kernel rpm package:

[root@localhost linux-3.0.8]# rpm -i /root/rpmbuild/RPMS/x86_64/kernel-3.0.8-1.x86_64.rpm
[root@localhost linux-3.0.8]# 

7. Do the postconfiguration to boot the new kernel - the easier way is to do it using the 7.1 step and only check if the files described in manual procedure are created/configured properly.


7.1 Automatically using the new-kernel-pkg utility:


[root@localhost linux-3.0.8]# new-kernel-pkg --mkinitrd --install --make-default --depmod 3.0.8


7.2 Manually (when previous used it can be skipped - everything already done by the new-kernel-pkg utility)

7.2.1 Create the initrd image:

[root@localhost linux-3.0.8]# mkinitrd -f /boot/initramfs-3.0.8.x86_64.img 3.0.8

7.2.2 Adapt the grub menu so that the new kernel becomes an option, below an example from my system:

[root@localhost boot]# vim /boot/grub/menu.lst
...

title CentOS Linux (3.0.8.x86_64)
        root (hd0,0)
        kernel /vmlinuz-3.0.8 ro root=/dev/mapper/vg_centos6hosta-lv_root rd_LVM_LV=vg_centos6hosta/lv_root rd_LVM_LV=vg_centos6hosta/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet
        initrd /initramfs-3.0.8.x86_64.img

8. Reboot the system and start the new kernel. Afterwards you should see:


[krychu@localhost ~]$ uname -a
Linux localhost.localdomain 3.0.8 #1 SMP Sat Nov 5 18:56:32 CET 2011 x86_64 x86_64 x86_64 GNU/Linux
[krychu@localhost ~]$ 

Please notice: when running on a virtual system with VirtIO disks (recognized by the guest system as vda devices) please check if the virtio_blk module is enabled (otherwise you will not be able to mount the root filsystem).

Ubuntu: SSL/TLS handshake - determine version

Recently I had to figure out what is the SSL and TLS version supported by some framework for which the only information I had was that it supports encryption. After looking into the TLS RFC (http://www.ietf.org/rfc/rfc2246.txt) and information about SSL I figured out that one might get those information by sniffing the handshake procedure.
Having the plan I decided to first check it locally on my host. As a first step I configured the apache2 on my Ubuntu 11.04 box as described: https://help.ubuntu.com/11.04/serverguide/C/httpd.html to support encryption. Next step was to use openssl (plus wireshark to be confident;)) to get the protocol version used in the handshake procedure.

SSL 3.0

Figure 1 SSL 3.0 handshake

Now let's have a test connecting using the openssl to the local apache2 server supporting SSL 3.0.

# openssl s_client -ssl3 -connect 127.0.0.1:443
CONNECTED(00000003)
depth=0 /CN=krystianek
verify error:num=18:self signed certificate
verify return:1
depth=0 /CN=krystianek
verify return:1
---
Certificate chain
 0 s:/CN=krystianek
   i:/CN=krystianek
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICpjCCAY4CCQCXhncRWkNj1DANBgkqhkiG9w0BAQUFADAVMRMwEQYDVQQDEwpr
cnlzdGlhbmVrMB4XDTExMTEwMjE2MzgyOVoXDTIxMTAzMDE2MzgyOVowFTETMBEG
A1UEAxMKa3J5c3RpYW5lazCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
ALU90KhdAKy4dBTGO8GLIpHLvO5L9DNZjLURgVoY2H8r1piYfvZzQ6sgNIrQzZjg
vGDTg2bUT7g4UQFJouSVGGRy8qk5afipoUFS+CxQEoWHkIQoRLdDSEtLxpYlQvk3
ID2qTzLGBeNm+YBSsvzg2e4sWBykKh1ePJ/TFxSdnW9+UZWbwytFlA6qNPVWA2Ks
lX4+N5xHIL1afd1+qGKUR85xJBcXXUbbZM7Fe+neGHmMqxuUXOmrU7uHE7igcHq7
UUXByIertkS33cgpV7QbWr9w+ip8ggKAUVkY/03NpaYfk40L06sef7jyWhMXFO8i
3TbnwDH0m/I3qxZqQZifYbECAwEAATANBgkqhkiG9w0BAQUFAAOCAQEAJ4BIrQ/U
KeJWsWQJsOAfJlpQ2giUiEYLDZyP8gnmhNCYnEdCBl9ltXxPIe1RIFK/O5dnAgV/
GLPYREW+jZ4tyip4cwWMIgixtcpyWyESJlH4vF4lKj6dP5C0xP5cyfbfvR9rlNxk
eejBsUv24FK9xCPN2V8IdkS+CNneGW06yAf+EQP9mF5weNXkbE7tX+yBwKuR8BgE
AUZ1qohBPz5OZqSV61HikVUf3RmqapnIo4lejilj/1uVhU8QJ1FxwJu2UyAJaiPj
qRSVvW4H7LSkz326DaKzoFFUylhAZPImEPN0LHv1ee0yzbWAzN8iEM6bzdNYJfJ2
fGQu5yAue2vxHQ==
-----END CERTIFICATE-----
subject=/CN=krystianek
issuer=/CN=krystianek
---
No client certificate CA names sent
---
SSL handshake has read 1413 bytes and written 319 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: zlib compression
Expansion: zlib compression
SSL-Session:
    Protocol  : SSLv3
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 6C8A75631A4964A27410DF69CFD267C8C1EE6363A6FDED270BD0671B0DFAE99F
    Session-ID-ctx:
    Master-Key: FDA851ACE2E2320690D55C7766A51718FCD6B2A89ED6887A4368583D8EB2FF2D4559D2408135D60A3401711B9C5FC7A9
    Key-Arg   : None
    Compression: 1 (zlib compression)
    Start Time: 1320299309
    Timeout   : 7200 (sec)
    Verify return code: 18 (self signed certificate)
---
GET /

It works!

This is the default web page for this server.


The web server software is running but no content has been added, yet.




closed
You have new mail in /var/mail/root
#


Ok, it works. In my case there was no client certificate sent towards the SSL server.


TLS 1.0

Figure 2 TLS 1.0 handshake

And now let's try to connect again using the openssl to the apache2 server but this time using the tls1 protocol.


# openssl s_client -tls1 -connect 127.0.0.1:443
CONNECTED(00000003)
depth=0 /CN=krystianek
verify error:num=18:self signed certificate
verify return:1
depth=0 /CN=krystianek
verify return:1
---
Certificate chain
 0 s:/CN=krystianek
   i:/CN=krystianek
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICpjCCAY4CCQCXhncRWkNj1DANBgkqhkiG9w0BAQUFADAVMRMwEQYDVQQDEwpr
cnlzdGlhbmVrMB4XDTExMTEwMjE2MzgyOVoXDTIxMTAzMDE2MzgyOVowFTETMBEG
A1UEAxMKa3J5c3RpYW5lazCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
ALU90KhdAKy4dBTGO8GLIpHLvO5L9DNZjLURgVoY2H8r1piYfvZzQ6sgNIrQzZjg
vGDTg2bUT7g4UQFJouSVGGRy8qk5afipoUFS+CxQEoWHkIQoRLdDSEtLxpYlQvk3
ID2qTzLGBeNm+YBSsvzg2e4sWBykKh1ePJ/TFxSdnW9+UZWbwytFlA6qNPVWA2Ks
lX4+N5xHIL1afd1+qGKUR85xJBcXXUbbZM7Fe+neGHmMqxuUXOmrU7uHE7igcHq7
UUXByIertkS33cgpV7QbWr9w+ip8ggKAUVkY/03NpaYfk40L06sef7jyWhMXFO8i
3TbnwDH0m/I3qxZqQZifYbECAwEAATANBgkqhkiG9w0BAQUFAAOCAQEAJ4BIrQ/U
KeJWsWQJsOAfJlpQ2giUiEYLDZyP8gnmhNCYnEdCBl9ltXxPIe1RIFK/O5dnAgV/
GLPYREW+jZ4tyip4cwWMIgixtcpyWyESJlH4vF4lKj6dP5C0xP5cyfbfvR9rlNxk
eejBsUv24FK9xCPN2V8IdkS+CNneGW06yAf+EQP9mF5weNXkbE7tX+yBwKuR8BgE
AUZ1qohBPz5OZqSV61HikVUf3RmqapnIo4lejilj/1uVhU8QJ1FxwJu2UyAJaiPj
qRSVvW4H7LSkz326DaKzoFFUylhAZPImEPN0LHv1ee0yzbWAzN8iEM6bzdNYJfJ2
fGQu5yAue2vxHQ==
-----END CERTIFICATE-----
subject=/CN=krystianek
issuer=/CN=krystianek
---
No client certificate CA names sent
---
SSL handshake has read 1560 bytes and written 293 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: zlib compression
Expansion: zlib compression
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 94FA2FB475C3FA47EDE2373C610F3F3C02CB2714F7344EC433B6B8ACEDC0AC43
    Session-ID-ctx:
    Master-Key: 958861A6AC4D901FA6263C0DA92E81F9430AEB201F82032D562813309957593E9E4259F8548AFBE2CB5A7145026135F5
    Key-Arg   : None
    TLS session ticket:
    0000 - 9a c9 2b 9c e9 54 7e e4-05 de 32 40 38 0a 6d b9   ..+..T~...2@8.m.
    0010 - 6d 21 2c 2e c8 ba 6e b7-de 37 72 0f 5b 5c 69 a8   m!,...n..7r.[\i.
    0020 - da 55 4a f6 73 31 59 4c-c4 3e 37 7b 9c 87 47 97   .UJ.s1YL.>7{..G.
    0030 - 03 f4 c9 62 45 95 a9 ab-11 31 ab de bf c9 5d b4   ...bE....1....].
    0040 - 50 75 ec 6b 54 c4 05 c8-bf 44 d3 14 41 d9 ea e9   Pu.kT....D..A...
    0050 - 0a 57 c8 d1 89 4f 3b 20-c6 0b 1e f6 f4 19 af 8e   .W...O; ........
    0060 - ca f7 18 28 1c 7b c7 9f-d4 03 c1 3f bc 47 be a0   ...(.{.....?.G..
    0070 - e0 74 0c c5 57 d6 16 4b-b3 a4 f5 c7 b7 10 7e 11   .t..W..K......~.
    0080 - 03 6a 5b e6 06 aa d7 75-40 d1 fe b0 62 ae e9 aa   .j[....u@...b...
    0090 - bc 0e 2d 59 06 97 99 78-ac 69 3e 8e c4 7f 34 e8   ..-Y...x.i>...4.
    00a0 - 84 89 c3 01 13 1b 01 b2-49 21 62 b8 4b e5 93 ea   ........I!b.K...
    00b0 - 88 09 bb d2 27 d2 ab cd-b6 94 67 0e a5 9b 7c fc   ....'.....g...|.


    Compression: 1 (zlib compression)
    Start Time: 1320299341
    Timeout   : 7200 (sec)
    Verify return code: 18 (self signed certificate)
---
GET /

It works!

This is the default web page for this server.


The web server software is running but no content has been added, yet.




closed
#

GTP prime library for Java (3GPP 32.295)

Recently I needed to have a client application for a GTP' external system. I did some research on the web but unfortunately did not find anything usable for me (found one java library on kenai.com but the implementation is in very early phase, the other was open-cgf written in Erlang for which I found two obstacles the language + crash dumps when it was starting up on my Ubuntu Natty 64-bit box). Having some development background from the past and having a look on the 3GPP 32.295 specification I decided that it is not a big amount of effort to implement it. I think this is a good opportunity to learn the protocol and additionally check the JBoss Netty framework - which seems to be a perfect match for such a solution. Indeed writing a client/server application using JBoss Netty is pretty straightforward (I will try to provide more details in another article) - it mainly about writing proper encoder/decoder and handlers.

The outcome can be found in the google code under the link: http://code.google.com/p/gtpprime/.

Linux: Creating encrypted partition using cryptsetup

In order to protect sensitive data in case your storage is lost or stolen, one usually encrypts the drive using trusted/safe encryption algorithms. Under Linux one can use e.g. cryptsetup - available in standard Ubuntu repository (I guess for other distros it should be also available in standard repos). Since it works on the block level, one can mix unencrypted partitions with encrypted ones. It also used in most of the distros (e.g. Ubuntu alternate installer) as a tooling providing disk encryption.
 
Creating/mounting encrypted partition


First of all I did not want to experiment with my HDD and decided to use the USB stick for this experiment. One needs to partition the disk - I created two partitions sdb1 and sdb2 using the fdisk. The partition table has been presented below:

root@krystianek:~# fdisk -l /dev/sdb

Disk /dev/sdb: 4156 MB, 4156555264 bytes
255 heads, 63 sectors/track, 505 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e56eb

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         132     1060258+  83  Linux
/dev/sdb2             133         505     2996122+  83  Linux
Now you need to format the partition as shown below (I used the 256-bit key length for encryption). Please type the uppercase 'YES' as prompted below. Also you will have to provide the encryption password.

root@krystianek:~# cryptsetup --verbose --key-size 256 luksFormat /dev/sdb2

WARNING!
========
This will overwrite data on /dev/sdb2 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase:
Verify passphrase:
Command successful.


That's it now you can open the encrypted partition. As a parameter you will provide the alias to which the device will be mapped as presented below:

root@krystianek:~# cryptsetup --verbose luksOpen /dev/sdb2 usbencrypted
Enter passphrase for /dev/sdb2:
Key slot 0 unlocked.
Command successful.

root@krystianek:~# ls -l /dev/mapper/usbencrypted
lrwxrwxrwx 1 root root 7 2011-10-16 20:06 /dev/mapper/usbencrypted -> ../dm-3

root@krystianek:~# cryptsetup status usbencrypted
/dev/mapper/usbencrypted is active:
  cipher:  aes-cbc-essiv:sha256
  keysize: 256 bits
  device:  /dev/sdb2
  offset:  2056 sectors
  size:    5990189 sectors
  mode:    read/write

Next you can create the filesystem - I created the ext4 and mounted it under /mnt/hd1:

root@krystianek:~# mkfs.ext4 /dev/mapper/usbencrypted
mke2fs 1.41.14 (22-Dec-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
187312 inodes, 748773 blocks
37438 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=767557632
23 block groups
32768 blocks per group, 32768 fragments per group
8144 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912

Writing inode tables: done                           
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 34 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@krystianek:~# mount /dev/mapper/usbencrypted /mnt/hd1/
root@krystianek:~# ls -l /mnt/hd1/
total 16
drwx------ 2 root root 16384 2011-10-16 20:12 lost+found

Now you can start copying files to that filesystem.

Unmounting/closing the encrypted partition

In order unmount and close the encrypted partition one needs to:

root@krystianek:~# umount /mnt/hd1/

root@krystianek:~# cryptsetup --verbose luksClose usbencrypted
Command successful.

Next time when you attach the usb stick you should be automatically prompted for password.