msgbartop
Blog di Dino Ciuffetti (Bernardino in realtà)
msgbarbottom

14 Nov 13 LVM Hot backups with snapshot

As you may know, LVM make it possible to create live snapshots of running logical volumes.
Imagine a guest virtual machine that has its virtual disk backed on a LVM logical volume on the host system.
You may create a live hot backup of your virtual machine on the fly, while it is working.

To do this, I created a small script that makes a compressed backup of all the logical volumes on the /dev/vg0 volume group.
The script make use of the standard LVM utilities to have the snapshot, the pv utility to get a cool progress bar and pigz utility to compress (gzip) using all of your processors.
If everything went ok, when the script finishes you’ll find your LVM hot backups on the /backups directory, and the temporary lvm snapshots removed.

This is how I make hot backups of some of my virtual machines (lvm_hot_backup.sh):

#!/bin/bash

for lv in `lvdisplay /dev/vg0 | grep ‘LV Name’ | awk ‘{print $3}’`
do
LV_SIZE=”`lvs –units m –noheadings –nosuffix $lv | cut -d’ ‘ -f7 | cut -d. -f 1`” # LV size in MB
LV_UUID=”`lvdisplay $lv | grep ‘LV UUID’ | awk -F’LV UUID’ ‘{print $2}’ | sed ‘s/^ *//g’`”
LV_SNAPNAME=”SNAP_`basename $lv`”

echo “LVM Logical Volume: $lv”
echo “Size: $LV_SIZE MB”
echo “UUID: $LV_UUID”
echo “Snapshot name: $LV_SNAPNAME”
echo “Removing old snapshot (if any)…”
lvremove -f “/dev/vg0/$LV_SNAPNAME”
echo “Creating snapshot…”
lvcreate -L+2G –snapshot -n”$LV_SNAPNAME” “$lv”
sleep 4
echo “Backing up snapshot…”
dd if=”/dev/vg0/$LV_SNAPNAME” bs=512k of=/dev/stdout | pv -pterbW -i 2 –buffer-size 512k –size “$LV_SIZE”m | /usr/bin/pigz -9 -b 256 > “/backups/$LV_SNAPNAME.lv.gz”
echo “Removing snapshot…”
lvremove -f “/dev/vg0/$LV_SNAPNAME”
echo “–”
done

 

13 Nov 13 Apache HTTPD as 2WAY (mutual) authentication SSL reverse proxy balancer

In this small article I’ll instruct myself (and you too?) how to create a 2 way authentication (mutual authentication) SSL reverse proxy balancer gateway. This configuration is useful in any enterprise environment where it’s requested to separate clients, the frontend and the backend, and when the traffic between clients and the gateway, and between the gateway and the backends must be encrypted.
This also ensure the clients and the backends to be authentic, and avoids Man In The Middle attacks.

Since the reverse proxy is in the middle between the clients and the backends, it’s requested for the clients to send a known client certificate to the gateway (apache), so that the gateway can recognize them. This is done with X509 certificates.
For the same reason, each backend contacted by the gateway is requested to respond with a valid and known server certificate. This is also done with X509 certificates.
Generally, the clients and the backends will also check their peer’s (apache) certificate to be known and valid, so that if someone is going to impersonate the gateway, it will be found and will not be considered authentic.

To do so, we’ll use:

  • apache httpd
  • mod_ssl
  • mod_proxy_balancer + mod_proxy + mod_proxy_http

Everything is done with a simple and single virtualhost in apache to be included in httpd.conf.
A working example is given below (assumes apache to be installed in /opt/apache, working with IP 11.22.33.44 on port 443):

<VirtualHost 11.22.33.44:443>
# General setup for the virtual host
DocumentRoot “/opt/apache/htdocs”
ServerName 11.22.33.44:443
ServerAdmin hostmaster@yoursite.com
CustomLog “|/opt/apache/bin/rotatelogs /opt/apache/logs/ssl_request_%Y%m%d.log 43200” “%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \”%r\” %b”
ErrorLog “|/opt/apache/bin/rotatelogs /opt/apache/logs/error_%Y%m%d.log 43200”
CustomLog “|/opt/apache/bin/rotatelogs /opt/apache/logs/access_%Y%m%d.log 43200” combined

# SSL CONFIGURATION – SERVER SIDE
# Enable SSL Server on this virtualhost
SSLEngine on
# Disable SSLv2 in favor of the more robust and secure SSLv3
SSLProtocol all -SSLv2
# List of supported cryptografic server cipher suites
SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5

# Apache server certificate
SSLCertificateFile “/opt/apache/conf/ssl/server.pem”
# Apache server private key
SSLCertificateKeyFile “/opt/apache/conf/ssl/key.pem”
# Apache server CA certificate (certificate of who released your server certificate)
SSLCertificateChainFile “/opt/apache/conf/ssl/ca.pem”
# Client’s CA certificates (list of certificates of who released your client’s certificates)
SSLCACertificateFile “/opt/apache/conf/ssl/ca.pem”
# It’s mandatory for apache to authenticate the client’s certificate
SSLVerifyClient require
# END OF SSL CONFIGURATION – SERVER SIDE

# SSL CONFIGURATION – CLIENT SIDE
# Enable SSL Client on this virtualhost (the traffic to the backends can be encrypted)
SSLProxyEngine on
# Apache client CA certificate (certificate of who released your client certificate)
SSLProxyMachineCertificateChainFile “/opt/apache/conf/ssl/ca.pem”
# Apache client private key + client certificate (concatenated in a single file)
SSLProxyMachineCertificateFile “/opt/apache/conf/ssl/client.pem”
# Backends’ CA certificates (list of certificates of who released your backends’ certificates)
SSLProxyCACertificateFile “/opt/apache/conf/ssl/ca.pem”
# It’s mandatory for apache to authenticate the backends’ certificate
SSLProxyVerify require
# END OF SSL CONFIGURATION – CLIENT SIDE

<FilesMatch “\.(cgi|shtml|phtml|php)$”>
SSLOptions +StdEnvVars
</FilesMatch>
<Directory “/opt/apache/cgi-bin”>
SSLOptions +StdEnvVars
</Directory>

BrowserMatch “MSIE [2-5]” \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0

# Define a load balancer worker to be used to balance the HTTPS traffic to three backends.
# The traffic between apache and the backends is encrypted
<Proxy balancer://httpslb>
# Define the first backend (https) with 2 way auth
BalancerMember https://192.168.1.11:443/ route=worker1 retry=10
# Define the second backend (https) with 2 way auth
BalancerMember https://192.168.1.12:443/ route=worker2 retry=10
# Define the third backend (https) with 2 way auth
BalancerMember https://192.168.1.13:443/ route=worker3 retry=10
</Proxy>

# Don’t send the “/balancer-manager” uri to the backends
ProxyPass /balancer-manager !
# Distribute the traffic (any url, since it is “/”) to the backends with round robin + cookie based session persistence
ProxyPass / balancer://httpslb/ lbmethod=byrequests stickysession=JSESSIONID

</VirtualHost>

If the clients and the backends are configured to check the gateway (apache) certificates, this is considered to be a very secure configuration.

Enjoy!

03 Nov 13 SSH connection is slow? Did you try to disable DNS lookups?

If your SSH connection is slow, it may depends on your SSH server that is executing reverse DNS lookups to try to identify your details.

Try setting the parameter below to your /etc/ssh/sshd_config and restart your ssh server daemon:

UseDNS no

It worked perfectly for me, it may work perfectly with you too.

10 Ott 13 How to fix svn error: OPTIONS ‘https://1.2.3.4/svn/prj’: SSL handshake failed: SSL error: Key usage violation in certificate has been detected

If you encounter an error like this one on your SVN client:

svn: OPTIONS di ‘https://192.168.1.36/svn/myprj‘: SSL handshake failed: SSL error: Key usage violation in certificate has been detected. (https://192.168.1.36)

you can try to fix your problem linking your libneon-gnutls.so.27 library used by your svn client to /usr/lib/libneon.so.27.

Try with this one:

mv /usr/lib/libneon-gnutls.so.27 /usr/lib/libneon-gnutls.so.27.old
ln -s /usr/lib/libneon.so.27 /usr/lib/libneon-gnutls.so.27

Tested on Debian 6.0 and Ubuntu 11.10

03 Mar 13 How to install php 5.2 on debian squeeze (6.0.x), the debian way

You may know that the PHP version coming with debian squeeze is 5.3. Since the 5.3 version of PHP breaks some compatibility with 5.2 you may find that an old PHP application is no longer working with the new version of PHP on squeeze.
The steps required to install PHP 5.2 on debian squeeze are very simple, you just need to setup APT to install the PHP packages coming with debian lenny.

The first thing to do is to add the lenny repository to the end of /etc/apt/sources.list:

deb http://archive.debian.org/debian lenny main contrib non-free

Then you need to make sure that your favourite PHP packages will be downloaded from lenny instead of squeeze. You can do this creating the file /etc/apt/preferences.d/lenny, with this stuff inside:

Explanation: choose Lenny as installation source if the package is not already installed and not available from Squeeze
Package: *
Pin: release n=lenny*
Pin-Priority: 100

Explanation: choose Lenny as installation source for those packages
Package: libapache2-mod-php5 php5-common php5-curl php5-gd php5-mcrypt php5-mysql php5-cli php5-mhash php5-xsl php5-imap php5-xmlrpc php5-sqlite
Pin: release n=lenny*
Pin-Priority: 999

After that, remove any previously installed PHP 5.3 package, for example with the command

apt-get remove –purge php5\*

and then install the PHP 5.2 packages from lenny:

apt-get update
apt-get clean
apt-get install libapache2-mod-php5 php5-common php5-curl php5-gd php5-mcrypt php5-mysql php5-cli php5-mhash php5-xsl php5-imap php5-xmlrpc php5-sqlite

That procedure saved my life twice, I hope it will save yours too!

02 Dic 12 New NuvolaBase Dashboard

Tonight at 03.00 GTM the NuvolaBase team publicly released the new NuvolaBase Dashboard.
As you may know, with NuvolaBase you can handle your private database on the cloud.

The new dashboard aims to be simple, stable and powerful. You can login using your google, twitter, facebook, linkedin account.
In the next days the NuvolaBase guys will release many new cool features like a powerful REST API to handle your databases in the cloud from your application.

This is the official article on the NuvolaBase blog: http://nuvolabase.blogspot.it/2012/12/nuvolabase-dashboard-upgrade.html

17 Set 12 SSH local and remote tunnels

Sometimes you may need to forward remote traffic to a local host through a SSH connection. In other words you can bind a given TCP port to a server running SSH and make remote clients connecting to it, letting the traffic to be redirected to a local server.

    You may need to add the parameter GatewayPorts clientspecified to /etc/ssh/sshd_config on the SSH server and restart sshd. This is to enable ssh clients to bind remote connections on a given ip, otherwise you can only connect to the remote port just from 127.0.0.1.

    At this point, let me do an example:

    • A TCP client wants to connect to 192.168.1.2 on TCP port 18443
    • You want to forward TCP traffic from 192.168.1.2:18443 to 11.22.33.44:18443
    • You have a client host with IP address 11.22.33.41 that can reach 11.22.33.44
    • You can establish a SSH connection from your client (11.22.33.41) to the remote server (192.168.1.2)

    If you have the given situation, you can execute the following command to bind the TCP port 18443 on the remote server:

    ssh -l root 192.168.1.2 -R:18443:11.22.33.44:18443

    Now, you can apply your changes:

    • Substitute “root” with your SSH user on the remote SSH server
    • Substitute “192.168.1.2” with your remote SSH server IP/host
    • Substitute the first “18443” with the port your remote TCP clients need to connect
    • Substitute “11.22.33.44” with your internal TCP server to forward traffic coming from the outside
    • Substitute the second “18443” with the TCP port listening on the internal host

    You can even do the reverse, letting local traffic flowing to an external host, passing through a SSH connection.

    Let me do another example:

    • You have a client host with IP address 11.22.33.41
    • A TCP client wants to connect to 11.22.33.41 (your IP) on TCP port 18443
    • You want to forward local TCP traffic from 11.22.33.41:18443 to 192.168.1.3:18443
    • You can establish a SSH connection from your client (11.22.33.41) to a remote server (192.168.1.2) that can reach 192.168.1.3

    If you have the given situation, you can execute the following command to bind the TCP port 18443 of your computer to the remote server:

    ssh -g -l root 192.168.1.2 -L18443:192.168.1.3:18443

    Now, apply your changes:

    • Substitute “root” with your SSH user on the remote SSH server
    • Substitute “192.168.1.2 with your remote SSH server IP/host
    • Substitute the first “18443” with the port your local TCP clients need to connect
    • Substitute “192.168.1.3” with the remote TCP server you want to reach from your local TCP clients
    • Substitute the second “18443” with the TCP port of the remote server

     

    03 Set 12 apache2 + mod_fastcgi + php 5.3 + PHP-FPM

    This is a very quick guide to get your feet wet with PHP 5.3 + PHP-FPM fastcgi support and apache webserver.
    The PHP-FPM is basically a fastcgi compliant pool of PHP processes spawned on the system, ready to quickly accept connections, for example via TCP. It’s generally used to greatly improove PHP scalability, security and performance.

    Start by installing apache, no matter if it’s a binary installation or if it’s compiled from source code (I assume this step is already done).

    Once you have a valid apache installation, you need to compile the mod_fastcgi module.
    NOTE: don’t use mod_fcgid or any other fastcgi provider but mod_fastcgi: it’s proved to be stable and to work well with PHP-FPM.

    To install mod_fastcgi you have to:

    1. download mod_fastcgi: http://www.fastcgi.com/dist/mod_fastcgi-2.4.6.tar.gz
    2. untar the package, then compile the apache module with: /your_apache_path/bin/apxs -o mod_fastcgi.so -c *.c
    3. install the module with: /your_apache_path/bin/apxs -i -a -n fastcgi .libs/mod_fastcgi.so

    Now, compile PHP with the fpm support, or install a already compiled PHP binary package.

    Here I’ll cover how to compile it from source.

    Start by downloading the latest php 5.3 version from http://www.php.net/downloads.php

    When you have done, untar the PHP source package and enter into the extracted php-5.3.x directory.
    Now create a file called conf.sh and put this stuff inside it:

    ./configure  \
    –prefix=/usr/local/php53 \
    –with-libdir=lib64 \
    –enable-pcntl \
    –enable-mbstring=shared \
    –enable-mbregex \
    –with-gd=shared \
    –enable-bcmath=shared \
    –with-xmlrpc=shared \
    –with-mysql=shared,/usr \
    –with-mysqli=shared,/usr/bin/mysql_config \
    –enable-dom=shared \
    –enable-soap=shared \
    –with-xsl=shared,/usr \
    –enable-xmlreader=shared –enable-xmlwriter=shared \
    –with-pdo-mysql=shared,/usr \
    –enable-json=shared \
    –enable-zip=shared \
    –with-readline \
    –with-jpeg-dir=/usr \
    –with-png-dir=/usr \
    –with-pear \
    –with-ldap=shared \
    –enable-fpm \
    –with-fpm-user=apache \
    –with-fpm-group=apache

    Your mileage may vary here, so please double check row by row if you need to modify something. The FPM part are the last 3 lines.

    NOTE: you cannot compile PHP as FPM and SAPI at the same time.

    Now, make the file executable with: chmod 755 conf.sh
    and run the executable with: ./conf.sh

    Wait that the configure script is done. If no errors are encountered you can proceed with make and make install as usual.
    Remember to create the php.ini configuration file if you need it.
    You should now end up with a fresh PHP installation into /usr/local/php53 (or any other path you given to the prefix configure attribute).

    Ok, now it’s time to configure the php-fpm (change /usr/local/php53 with your path if it’s different):

    cd /usr/local/php53/etc
    cp php-fpm.conf.default php-fpm.conf
    vi php-fpm.conf

    You generally don’t need to modify anything here, but if you want you can touch something.
    Now start the php-fpm process pool by running this command by the root user: /usr/local/php53/sbin/php-fpm

    If anything gone ok you should have some process up and running, something like this:

    25976 ?        Ss     0:00 php-fpm: master process (/usr/local/php53/etc/php-fpm.conf)
    4945 ?        S      0:00  \_ php-fpm: pool www
    4946 ?        S      0:00  \_ php-fpm: pool www
    4947 ?        S      0:00  \_ php-fpm: pool www

    If you didn’t modify the php-fpm.conf, the process pool listen for fastcgi requests to TCP 127.0.0.1:9000.

    It’s time to configure a apache virtualhost with PHP support using this brand new fpm.

    Edit the httpd.conf apache configuration file (or another included file where you store the virtualhost) and append this stuff (I assume that apache is installed into /opt/apache2):

    <VirtualHost *:80>
    ServerAdmin webmaster@dummy-host.example.com
    DocumentRoot “/opt/apache2/htdocs”
    ServerName “your_servername.com”
    ErrorLog “logs/your_servername-error_log”
    CustomLog “logs/your_servername-access_log” common

    FastCgiExternalServer /opt/apache2/htdocs/php5.sock -host 127.0.0.1:9000
    AddHandler php5-fcgi .php
    Action php5-fcgi /tmp/php5.sock
    Alias /tmp /opt/apache2/htdocs

    <Directory “/opt/apache2/htdocs”>
    Options FollowSymLinks
    AllowOverride All
    Order deny,allow
    Allow from all
    </Directory>
    </VirtualHost>

    Any file whose name ends for “.php” into your document root should now be associated to the PHP fastcgi handler and the requests should be routed to the php-fpm process pool. Each php-fpm process is reused according to the php-fpm.conf configuration file.

    Restart apache and enjoy (any comment are welcome).

    18 Mag 12 The OrientDB 1.0 stable is finally there!

    Congratulation to Luca Garulli and his dev team to the public release of OrientDB 1.0 Stable!
    After a year of release candidates and bug fixing it’s finally time to the stable version.
    Many may bugs fixed, new indexing algorithms, improved clustering with multi master replication, new Object Database interface with lazy object loading, new studio (web interface) and much more.

    The community is growing fast and people get rapidly moving to new technologies.
    Words like nosql, object and graph databases, cloud and mobile are big buzzwords of nowadays.

    If you didn’t already, subscribe to the orientdb mailinglist to get in touch: http://groups.google.com/group/orient-database?pli=1.

    08 Mar 12 How to execute a HTTP/Rest Query to NuvolaBase distributed database with PHP

    As previously said, nuvolabase.com is a great service that permits you to have a distributed nosql document database in the cloud. This is very cool: think each time you would had the need of a database always available in the cloud that you would access via simple HTTP/Rest queries. The possibilities are endless.

    Here is a very simple but powerful PHP curl agent to submit commands (queries) to nuvolabase via HTTP.

    <?php
    /*
    * Author: Dino Ciuffetti <dino@tuxweb.it>
    * Object: Execute a remote query to a distributed database on nuvolabase.com (free account) using HTTP (OrientDB REST API)
    */

    /* user configurable parameters */
    $nuvolabasedb = ‘db$free$youruser$yourdb’;
    $command = ‘select from yourclass’;
    $user = ‘admin’;
    $password = ‘qwerty’;
    $useragent = “NuvolaBase PHP REST agent/v0.8 (compatible; Mozilla 4.0; MSIE 5.5; http://www.nuvolabase.com/)”;
    /* END of user configurable parameters */

    $nuvolabasehost = ‘studio.nuvolabase.com’;
    $url = ‘http://’.$user.’:’.$password.’@’.’studio.nuvolabase.com/command/’.$nuvolabasedb.’/sql/’;

    $ch = curl_init();

    // set user agent
    curl_setopt($ch, CURLOPT_USERAGENT, $useragent);

    // return the result or false in case of errors
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

    // set the target url
    curl_setopt($ch, CURLOPT_URL, $url);

    // do basic login authentication
    curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);

    // howmany parameter to post
    curl_setopt($ch, CURLOPT_POST, 1);

    // the post data to send
    curl_setopt($ch, CURLOPT_POSTFIELDS, $command);

    // execute curl,fetch the result and close curl connection
    $res = curl_exec ($ch);
    curl_close ($ch);

    // display result
    if ($res !== FALSE);
    print_r (json_decode($res));

    ?>

    Please use the attached file.

    test.php