09 Ago 11 DNSBL GeoIP service at countries.nerd.dk
I recently discovered a wonderful DNSBL service reporting you where public Internet IPs are from.
The service is countries.nerd.dk: http://countries.nerd.dk/more.html
You can for example block any mail at your mailserver coming from china or russia, simply integrating this DNSBL with your MTA.
You can even get the country of your IP with dig!
Warning: You need to swap IP octets. If for example the IP to check is 192.162.132.171, you have to call 171.132.162.192.zz.countries.nerd.dk.
root@nbvirtdns1:/# dig TXT 201.65.24.151.zz.countries.nerd.dk
…
201.65.24.151.zz.countries.nerd.dk. 1047 IN TXT “it”
…
04 Lug 11 How to use Huge Pages with Java and Linux
Hi.
Today we get how to use Huge Pages with Java from a Linux powered system.
While a Linux system generally splits memory segments into pages of 4 kb, Huge Pages are memory pages large 2Mb or more.
This is proved to increment speed when the application make use of large quantity of ram, like Java with a large heap (2 GB or more).
It’s correct to say that this is not always the correct configuration choice because the memory setted to be dedicated to Huge Pages cannot be accessed by the kernel (buffer cache) or by the applications, so that memory is subtracted from the virtual memory pool of the system. Since it’s very fast to make it a try and decide if use it or not, let me play with it.
Here in this example we will set 2,5 GB of RAM to be used as Huge Pages. Your mileage may vary.
HPM (huge page memory) is expressed in GB.
First: set the quantity of memory (bytes) to be defined as a shared memory segment
This is quickly found calculating this simple formula: ((HPM * 1024 * 1024 * 1024) – 1).
In our example: ((2,5 * 1024 * 1024 *1024) – 1) = 2684354559
Set it up online with this command:
echo 2684354559 > /proc/sys/kernel/shmmax
If you want to set it permanent at the next system reboot, append those two lines to your /etc/sysctl.conf file:
# Shared memory – max segment size: 2,5 Gb (-1 b)
kernel.shmmax = 2684354559
Second: set the number of reserved large memory pages
This is the number of reserved pages. Each page is large 2 Mb, so finding the number of pages to reserve is simple:
((HPM * 1024) / 2). In our example: ((2,5 * 1024) / 2) = 1280
Set it up online with this command:
echo 1280 > /proc/sys/vm/nr_hugepages
If you want to set it permanent at the next system reboot, append those two lines to your /etc/sysctl.conf file:
# Enable kernel to reserve 2,5GB / 2Mb large pages
vm.nr_hugepages = 1280
Third: set the system group id enabled to use huge pages
Java programs usually should not be fired by the root user. In my case, the group id of my program is “1001”.
Set it up online with this command:
echo 1001 > /proc/sys/vm/hugetlb_shm_group
If you want to set it permanent at the next system reboot, append those two lines to your /etc/sysctl.conf file:
# System group id that can use huge pages (hugepages gid: 1001)
vm.hugetlb_shm_group = 1001
Fourth: run the java program with the Huge Page support
In our example we are using the JVM distributed by Oracle. Other Java vendors may use different parameters to enable Huge Pages. They can even call Huge Pages differently.
The program can now be fired with “-XX:+UseLargePages -XX:LargePageSizeInBytes=2m”
My complete java parameters for my java program are:
java -d64 -server -Xms1900m -Xmx1900m -Xss192k -XX:+UseLargePages -XX:LargePageSizeInBytes=2m -XX:+UseParNewGC
Ciao ciao.
Dino Ciuffetti.
13 Giu 11 How to unpack a patch.e2 enigma2 file from linux
The enigma2 patch file, generally named patch.e2, is a packet binary file containing a cramfs root filesystem and a linux kernel in zboot format.
It is commonly used by some TV decoder linux distribution systems to update the system firmware.
I didn’t find a quick way to extract the “/” cramfs filesystem from patch.e2 files from a linux system, so I decided to write a small utility from myself.
It is in attach, it’s called unpack_e2 and it’s very easy to use.
Just compile it with:
gcc -o unpack_e2 -O2 -Wall unpack_e2.c
And call it with the patch.e2 file as the first argument.
The two files produced are cram.img and kernel.img.
dino@dam2k:~/AZBOX_RTi_E2$ ./unpack_e2 /tmp/patch.e2
Team name: RTi Team
Description: Core 1.0
Version: 1.0.0
About: v.1.0
Kernel description: #78_May27
Size of cram image: 47542272 bytes (45.34 Mb)
Size of kernel image: 6584320 bytes (6.28 Mb)
Unpacking cramfs image to cram.img
Unpacking kernel image to kernel.img
Warning: it will work only with the new E2 image format (I think >= Core RC12).
If for some weird reason you need to extract the linux binary kernel file, you must use those commands:
# mount -t auto -o loop kernel.img /mnt
# dd if=/mnt/xrpc_xload_vmlinux_ES4_prod.bin skip=1 bs=836 |zcat >/tmp/vmlinux.bin
P.S. To do the reverse (pack a patch.e2) follow this link: http://sourceforge.net/projects/rticoree2/files/image_tools/
Ciao, Dino.
01 Giu 11 Openssl e ciphers… questi sconosciuti
La suite openssl supporta differenti meccanismi di crittografia asimmetrica.
Il client e il server negoziano in fase di handshake la modalita’ di cifratura che utilizzeranno per il trasferimento sicuro dei dati.
In openssl le ciphers implementano 4 algoritmi:
1) Key Exchange Algorithm (scambio delle chiavi)
Sono RSA o Diffie-Hellman
2) Authentication Algorithm (autenticazione dei sistemi)
RSA, Diffie-Hellman, DSS o nessuno
3) Cipher/Encryption Algorithm (cifratura dello stream di dati)
DES, Triple-DES, RC4, RC2, IDEA o nessuno
4) MAC Digest Algorithm (verifica della validita’ del pacchetto)
MD5, SHA o SHA1
Il comando “openssl s_client -ciphers <parametro cipher>” permette di forzare il client (in questo caso il comando openssl stesso) ad utilizzare i meccanismi di cifratura piu’ deboli (parametro LOW), medi (MEDIUM) o piu’ sicuri (HIGH). Tuttavia per poter colloquiare in modo corretto, anche il server SSL deve supportare tali modalita’.
La suite openssl presente al momento sul mio pc (0.9.8k) implementa le seguenti ciphers:
LOW (tutti hanno chiave di cifratura inferiore a 128 bit, e firma hash SHA1 o MD5):
ADH-DES-CBC-SHA SSLv3 Kx=DH Au=None Enc=DES(56) Mac=SHA1
EDH-RSA-DES-CBC-SHA SSLv3 Kx=DH Au=RSA Enc=DES(56) Mac=SHA1
EDH-DSS-DES-CBC-SHA SSLv3 Kx=DH Au=DSS Enc=DES(56) Mac=SHA1
DES-CBC-SHA SSLv3 Kx=RSA Au=RSA Enc=DES(56) Mac=SHA1
DES-CBC-MD5 SSLv2 Kx=RSA Au=RSA Enc=DES(56) Mac=MD5
MEDIUM (tutti hanno chiave di cifratura uguale a 128 bit, e firma hash SHA1 o MD5):
ADH-RC4-MD5 SSLv3 Kx=DH Au=None Enc=RC4(128) Mac=MD5
RC4-SHA SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=SHA1
RC4-MD5 SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=MD5
RC2-CBC-MD5 SSLv2 Kx=RSA Au=RSA Enc=RC2(128) Mac=MD5
RC4-MD5 SSLv2 Kx=RSA Au=RSA Enc=RC4(128) Mac=MD5
HIGH (tutti hanno chiave di cifratura superiore o uguale a 128 bit, e firma hash SHA1 o MD5):
ADH-AES256-SHA SSLv3 Kx=DH Au=None Enc=AES(256) Mac=SHA1
DHE-RSA-AES256-SHA SSLv3 Kx=DH Au=RSA Enc=AES(256) Mac=SHA1
DHE-DSS-AES256-SHA SSLv3 Kx=DH Au=DSS Enc=AES(256) Mac=SHA1
AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1
ADH-AES128-SHA SSLv3 Kx=DH Au=None Enc=AES(128) Mac=SHA1
DHE-RSA-AES128-SHA SSLv3 Kx=DH Au=RSA Enc=AES(128) Mac=SHA1
DHE-DSS-AES128-SHA SSLv3 Kx=DH Au=DSS Enc=AES(128) Mac=SHA1
AES128-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA1
ADH-DES-CBC3-SHA SSLv3 Kx=DH Au=None Enc=3DES(168) Mac=SHA1
EDH-RSA-DES-CBC3-SHA SSLv3 Kx=DH Au=RSA Enc=3DES(168) Mac=SHA1
EDH-DSS-DES-CBC3-SHA SSLv3 Kx=DH Au=DSS Enc=3DES(168) Mac=SHA1
DES-CBC3-SHA SSLv3 Kx=RSA Au=RSA Enc=3DES(168) Mac=SHA1
DES-CBC3-MD5 SSLv2 Kx=RSA Au=RSA Enc=3DES(168) Mac=MD5
e altre modalita’ che non fanno capo ai tre alias descritti (LOW, MEDIUM e HIGH).
Per quanto riguarda la parte SERVER (per i bravi che usano APACHE) e’ possibile impostare le ciphers supportate tramite l’utilizzo del parametro SSLCipherSuite, ad esempio mettendo qualcosa del genere:
SSLCipherSuite +HIGH:+MEDIUM:!LOW:+SSLv2
Sempre parlando di apache, se non specificato, il default e’: SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP
ovvero si cerca di mantenere la compatibilita’ con i client piu’ vecchi e che quindi supporano solo ciphers poco robuste.
Scusate la “lezione di crittografia” ma secondo me serve per fare un po’ di chiarezza generale sull’argomento, spesso un po’ oscuro a molti.
Ciao, Dino.
31 Mag 11 liborient: A simple LGPLv3 linux C library proof of concept
Today i posted to the orientdb mailinglist and I’ve written about liborient, my very first orientdb C library implementation.
We are searching for new developers to join. This is what I putted to the list.
Hi all.
I’m making an attempt to write a proof of concept, simple, LGPLv3
OrientDB C library for linux.
The library is written in best effort, so don’t kill me if you see bad
code for now…
As a starting point, there is already a very first implementation of
some simple binary protocol methods.
For those there are interested, this is the API that it’s just (it
seems…) working with the latest OrientDB SVN version:
http://www.tuxweb.it/temp/apishot/liborient/liborient_8h.html#func-members
You can view development code here:
http://svn.tuxweb.it/cgi-bin/viewvc.cgi/liborient/trunk/main/liborient/src/
INSTALL:
1) Install the latest GNU autoconf, automake and libtool
2) svn co http://svn.tuxweb.it/SVN/projects/liborient/trunk/main/liborient
3) cd liborient
4) ./autogen.sh
5) ./configure –prefix=/tmp/liborient
6) make
7) make check
8) make install
Warning: this is a very first proof of concept implementation that I
started to study OrientDB. Do not use it in production environments.
Even if I think “the scalable way”, I’m a Linux SysAdmin and not a
full time developer, so may be the API is not well designed and the
code is ugly.
We need people that write code. If you are interested, please join in
and contribute.
This is a sample C program that links liborient… and works 🙂
http://svn.tuxweb.it/cgi-bin/viewvc.cgi/liborient/trunk/main/liborient/test/single_orient.c?view=markup
<snip>
orientdb *oh;
o_conh *och;
unsigned long cid;
// create a new liborient handler
oh = orient_new();
// setup library debug level to “ORIENT_DEBUG”
orient_debug_setlevel(oh, ORIENT_DEBUG);
// setup debug callback
orient_debug_sethook(oh, &your_debug_function);
// preparing to open a new binary connection handler for orientDB
och = orient_prepare_connection(oh, ORIENT_PROTO_BINARY, “localhost”, “2424”);
// setting admin credentials
orient_set_credentials(oh, och, ORIENT_ADMIN, “root”, “pippo”);
// setting user credentials
orient_set_credentials(oh, och, ORIENT_USER, “reader”, “reader”);
// create the real connection with orientdb server
cid = orient_connect(oh, och, timeout);
// open the database “demo”
orient_dbopen(oh, och, cid, “demo”, timeout);
// get the DB size
dbsize = orient_db_size(oh, och, cid, timeout);
// get the total number of records
records = orient_db_countrecords(oh, och, cid, timeout);
// close the database
orient_dbclose(oh, och, cid, timeout);
// free library stuff
orient_free(oh);
</snip>
Any thoughts?
Ciao, Dino Ciuffetti.
18 Mag 11 orientdb svn build
OrientDB is a fast, scalable, open source object / graph database server written in Java.
After more than 20 years of RDBMS predominance it’s now time to switch to non relational database systems, specially where scalability and query response time are two fundamental things to achieve a better user (web or not) experience.
So, how to get up and running quickly with orientdb?
Here we will build on a linux system the latest development version from source in no time: the simple way ™.
First thing to do is to download JDK Java Standard Edition 6 from http://www.oracle.com/technetwork/java/javase/downloads/index.html.
Please note that you will need the JDK and not JRE.
After that you will need Apache Ant. Download it from here: http://ant.apache.org/bindownload.cgi.
# cd /opt
# tar jxf apache-ant-1.8.2-bin.tar.bz2
Installed? Good. Now install subversion (svn). You can install it for example using your favorite distribution specific package manager, for example if you are using debian or ubuntu you could use apt-get utility, like that:
# apt-get update; apt-get install subversion
You have now to create a directory where you like on the system and begin to download the OrientDB development snapshot:
# mkdir /home/dino/orientdb-source
# cd /home/dino/orientdb-source
# svn checkout http://orient.googlecode.com/svn/trunk/ orient-read-only
When finished cd to orient-read-only.
You have now to set your JDK and ANT bin directories into your PATH system variable. You can do it by this way:
# export PATH=/opt/apache-ant-1.8.2/bin:/opt/jdk1.6.0_25/bin:$PATH
You can now begin to compile orient source code.
# ant clean
# ant
# ant test
# ant install
Ok. If compiled successfully, you now have to startup orient for the first time.
# cd /home/dino/releases/1.*-SNAPSHOT/bin
# chmod 754 *.sh
# ./server.sh
Ok. Now stop it with CTRL+C and modify the configuration file as you like:
# cd ../config
# vi orientdb-server-config.xml
The first thing to configure, if you need to publish the service on your network/internet, is the bind address. For example, to bind on any ip on the system: <listener ip-address=”0.0.0.0″ port-range=”2424-2430″ protocol=”distributed”/>
The second parameter to change is the root password: <user name=”root” password=”pippo” resources=”*”/>
Now start orientdb again:
# cd /home/dino/releases/1.*-SNAPSHOT/bin
# nohup ./server.sh 1>/dev/null 2>/dev/null &
You should now have done.
Connect your browser to http://127.0.0.1:2480/ and begin to play with your brand new orientdb studio web console:
host: localhost
user: writer
password: writer
database: demo
You can find open and solved issues here: http://code.google.com/p/orient/issues/list
Subscribe yourself to the orientdb users mailing list service: http://groups.google.com/group/orient-database, and enjoy!!
15 Mag 11 How to cleanly install apache httpd webserver from source
Apache is a great piece of software and it’s very powerful. You can here find how to install apache web server, “The right and quick way” ™ 🙂 Apache will be compiled with modules support.
We begin downloading source code from httpd.apache.org official website.
Important: for security reasons download apache only from the official website or by your distribution vendor (redhat, debian, etc)!
Important: choose the right version for you, but if you can pick the last stable version of 2.2.X.
You can then untar your fresh apache source distribution and beginning to play with configure:
tar jxvf httpd-2.2.??.tar.bz2
cd httpd-2.2.??
./configure –with-included-apr –with-expat=builtin –prefix=/your_apache_installation_path –enable-mods-shared=most –enable-ssl –enable-proxy –enable-proxy-connect –enable-proxy-http –enable-proxy-balancer
Here we are going to compile apache with most of its modules compiled as shared objects and obviously with the shared object support enabled.
May be that the configure process will not find something. If so, remember to download and install the development version of the needed library. I always use the development package of the distribution (ex. libssl-dev, openssl-devel, etc).
If the configure process fails because of a dependency (development!!!) library not found but you already installed it, it means that the apache building tool does not find it.
Sometimes, in not standard environments (hpux, etc) apache will not find the openssl dev library but it’s installed. For example if apache fails a dependency check on openssl, you can fix the problem passing the argument –with-ssl=/ssl_path to configure.
If the configure pass, you have to call:
make
If the make process pass without problems, you have to execute:
make install
If you have problems compiling apache, most probably the problem is that you failed to install building prerequisites: development libraries, or the building tool cannot find where they are installed. If the configure fails you can take a read on config.log or reading the configure file (it’s a generated shell script) searching for the reported error and finding the cause by yourself (wrong path of the library?). If the failing stuff fires on compiling (make) you have to find the problem going to the source directory reported by the make utility and searching for the source file that it’s failing. You may require a patch, may be you caught a apache bug or may be your compiling chain is not clean.
I have to go now, let me know if you have problems.
Ciao, Dino.
24 Ago 10 Why do not host sites from your own home
I would not recommend you to host sites that way, you have to be sure that your ISP give you public IP(s) and setup your router to port forward ports 80, 443, 53, and so on.
There are other problems too:
1) if you want to host more than one site with SSL you must have one public IP for each SSL site or use different SSL ports for each site, because name virtualhosting with SSL is not possible;
2) dsl lines are not designed to be stable. The connection can go down and make your site not visible. This is a major problem if you make the mistake to have your own DNS server on it!! The ISP assigned public IP address can change more than one time a day and you have to sync the DNS zone each time.
3) dsl ips are putted into DNS based blacklists zones. You may not be reached from various HTTP proxy servers around the world. For the same reason you cannot send mails, for example originated from your sites.
4) adsl lines are asymmetric (unbalanced for download). You have few kbytes per second in upload, that is just what you need to publish web sites, so this can be a problem when you have just more than 3 users.
5) you probably have problems with High-Availability and Load-Balancing on domestic hardware and you may have blackouts.
6) DNS subsystem may need primary and secondary DNS servers.
The best way (imho) is to use services like slicehost where you have a HA virtual server slice running linux, public IP addresses, free primary and secondary DNS hosting service, large public bandwidth, disk space… and not last your own root password that you can use to have maintenance on your own server for your own.
https://manage.slicehost.com/customers/new?referrer=af57db3020e04bb27352e271753a7a18
26 Mar 10 Il tuo server linux personale
Se quello che hai sempre cercato e’ avere il tuo personalissimo server linux up and running 24 ore su 24, SliceHost e’ l’opzione giusta per te.
Questa meravigliosa azienda americana (in Italia purtroppo certe cose ce le sogniamo alla grande!) ha sviluppato un sistema automatico con interfaccia web in grado di fornirti in tempo reale per pochi dollari al mese una tua personalissima macchina virtuale con cui potrai realizzare e gestire il tuo server linux in tutta tranquillita’.
Banda e connettivita’ internazionale a internet non sono un problema e potrai scegliere tra vari tagli di offerte pronte per te.
Se sei interessato, dai un’occhiata al sito https://manage.slicehost.com/customers/new?referrer=af57db3020e04bb27352e271753a7a18 e affiliati anche tu.
Avrai la possibilita’ di scegliere la distribuzione linux che piu’ ti aggrada e il tuo server linux personale sara’ in piedi in pochi secondi.
Noi di TuxWeb lo stiamo utilizzando con successo per gestire i siti internet di alcuni nostri clienti.
Ciao, Dino – http://www.tuxweb.it/
13 Mar 10 DynaBlast, and your access_log give you something
If you want an apache httpd server access_log scanner written in C that check for you every IP address that exceed a given number of hits in the last given time period, then DynaBlast is a tool that make sense for you.
This tool, released as usual in GPLv2, can be spawned every minute in crontab without the risk of having two instances running: we make use of a lock mechanism.
It has a fast mode that can skip already parsed line from a consecutive scan, so each time it runs, it play very fast.
In stdout you’ll get the scanning result. Ex:
blacklisted:10.248.220.43
blacklisted:23.212.121.165
You can download DynaBlast from here: http://www.tuxweb.it/?section=progetti/dynablast
Ciao, Dino.
Follow me!
Stack Overflow
IPv6 Certified (he.net)