11 Feb 23 How to create persistent Queues, Exchanges, and DLXs on RabbitMQ to avoid loosing messages

What happens when you publish a message to an exchange in RabbitMQ with the wrong topic, or better, routing key? What happens if you send a message to the broker in a queue with a TTL policy or the TTL property is in the message itself and that TTL expires? What happens when a consumer discard your message got from the queue with no republish? What if a queue overflows due to a policy?

It’s simple, the broker will simply discard your message forever.

If this thing will make you mad like it does to me this blog article is for you. Here I will tell you how to create a simple tree of queues, DLX and policies to overcome this problem.

I think that starting with commands and examples is better than 10000 written words, and since I don’t have any ads on my blog I don’t have to write a long article to get money from the ads, so here we are.

I consider your RabbitMQ installation and your admin account ready, so we start with the commands.

# Create your VirtualHost rabbitmqctl add_vhost vhtest --description "Your VH" --default-queue-type classic

# Give your admin user permissions to to everything on your virtualhost rabbitmqctl set_permissions --vhost vhtest admin '.*' '.*' '.*'

# Create the user that will publish messages to the exchange rabbitmqctl add_user testuserpub yourpassword1

# Create the user that will subscribe to your queue to read messages rabbitmqctl add_user testusersub yourpassword2

Now, we have 3 users (admin, testuserpub and testusersub) and a virtualhost (vhtest). We are ready to create 2 DLX, one to handle overflow, expired TTL and discarded messages, the other to handle messages sent with the wrong routing key. A DLX (or Dead Letter Exchange) is a particular exchange that is designed to handle dead lettered (discarded) messages.

# Create the DLX to handle overflowed, expired or discarded by consumers rabbitmqadmin declare exchange --vhost=vhtest name=DLXexQoverfloworttl type=headers internal=true

# Create the DLX to handle messages with wrong routing key rabbitmqadmin declare exchange --vhost=vhtest name=DLXexQwrongtopic type=fanout internal=true

We’ll now declare and bind queues to the first DLX using three different policies

rabbitmqadmin declare queue --vhost=vhtest name=DLXquQoverflow rabbitmqadmin declare queue --vhost=vhtest name=DLXquQttl rabbitmqadmin declare queue --vhost=vhtest name=DLXquQrejected rabbitmqadmin declare binding --vhost=vhtest source=DLXexQoverfloworttl destination=DLXquQoverflow arguments='{"x-first-death-reason": "maxlen", "x-match": "all-with-x"}' rabbitmqadmin declare binding --vhost=vhtest source=DLXexQoverfloworttl destination=DLXquQttl arguments='{"x-first-death-reason": "expired", "x-match": "all-with-x"}' rabbitmqadmin declare binding --vhost=vhtest source=DLXexQoverfloworttl destination=DLXquQrejected arguments='{"x-first-death-reason": "rejected", "x-match": "all-with-x"}'

And now we’ll declare and bind queues to the second DLX to handle messages with wrong topic (routing key)

rabbitmqadmin declare queue --vhost=vhtest name=DLXquQwrongtopic rabbitmqadmin declare binding --vhost=vhtest source=DLXexQwrongtopic destination=DLXquQwrongtopic

Now we have 1 DLX with 3 queues and another DLX with 1 queue bound. The first will route expired, discarded and overflowed messages to the respective queues (DLXquQttl, DLXquQoverflow, DLXquQrejected), the second will route messages with invalid routing key to the respective queue (DLXquQwrongtopic).

Now we are going to create our main queue and the normal Exchange that will send message to it

rabbitmqadmin declare queue --vhost=vhtest name=quQ rabbitmqadmin declare exchange --vhost=vhtest name=exQ type=direct

In this example, we want to route all messages with routing key NBE

rabbitmqadmin declare binding --vhost=vhtest source=exQ destination=quQ routing_key=NBE

We now want to create the policy that is needed to associate the wrong topic DLX to our main exchange

rabbitmqctl set_policy --vhost vhtest wrongtopicQ1 "^exQ$" '{"alternate-exchange":"DLXquQwrongtopic"}' --apply-to exchanges

This is an example policy to set limits to 100 messages, 1073741824 bytes, 30 seconds TTL to the quQ queue.

rabbitmqctl set_policy --vhost vhtest shorttimedqunbe '^quQ$' '{"max-length":100,"max-length-bytes":1073741824,"message-ttl":30000,"overflow":"reject-publish-dlx","dead-letter-exchange":"DLXexQoverfloworttl"}' --priority 0 --apply-to queues

Going to give proper permissions to our publish and subscriber users. The user testuserpub can only write to its exchange, while testusersub can read from its queue. No other permissions here.

rabbitmqctl set_permissions --vhost vhtest testuserpub '' '^exQ$' '' rabbitmqctl set_permissions --vhost vhtest testusersub '' '' '^quQ$'

Mission complete. Please try this at home and write to the comments below! Happy RabbitMQ hacking!

Tags: Linux, opensource, RabbitMQ, server, sysadmin
Filed in General, Linux OS, Open source, Sysadmin pills, What I'm doing with 0 Comments

05 Apr 22 Parse mailcleaner infolog

Here is my mailcleaner infolog log parser.

It will give you clear informations about identified spam messages: spam score, mail id, source ip address, source mail and destination domain on your mailcleaner /var/mailcleaner/log/mailscanner/infolog log.

#!/bin/bash

echo "SPAMSCORE|MAILID|SRCIP|SRCMAIL|DSTDOMAIN" 1>&2
grep -P '(?=.*?spam decisive\))(?=.*?Spamc \()' /var/mailcleaner/log/mailscanner/infolog | sed -E -e 's/^[A-Za-z]*.* Message ([-a-zA-Z0-9]*) from ([.:a-zA-Z0-9]*) \(([-_+=.a-zA-Z0-9]*@[-_+=.a-zA-Z0-9]*\.[-_=.a-zA-Z]*)\) to ([-_a-zA-Z]*\.[-_a-zA-Z.]*) .*, Spamc \(score=([0-9.]*), .*$/\5|\1|\2|\3|\4/'

The output is something like this:

SPAMSCORE|MAILID|SRCIP|SRCMAIL|DSTDOMAIN
47.7|1naYCh-00HLaZ-NR|46.253.16.31|evdamoaffaeomaadfeigfmaueh.aehiaohkee@u271525.rmh2.net|mydomain1.com
5.2|1naYps-00HOPl-OJ|91.222.96.128|g-7247847043-6761-652195341-1648887032817@bounce.m.loffertadioggi.net|mydomain1.com
6.6|1naZKI-00HQHn-I0|185.251.132.173|ge3tgmjnhezdgljxgmzdgmbv@e.monshopactu.com|mydomain1.com
51.0|1naZWg-00HQza-1J|35.227.130.66|mailer@infusionmail.com|mydomain2.com
5.2|1naZsO-00HT1g-3V|2a02:180:6:1::51b2|ag@lpe.mxgaleri.rest|mydomain2.com
51.0|1naa9L-00HTxx-8v|35.227.130.212|mailer@infusionmail.com|mydomain2.com
5.2|1naaZE-00HVna-00|163.47.180.142|delivery_20220402060126.27221310.120563@mx.sailthru.com|mydomain1.com

Tags: bash, domain, Linux, mailcleaner, mailserver, script, server, spam
Filed in General with 0 Comments

06 Nov 21 Eddaje cor Cloud e la “Trasformazione digitale”

Daje che mo col cloud e grazie alla trasformazione digitale diventate tutti sistemisti e architetti!! Co du’ click create tutte le infrastrutture provando a caso, senza conosce cosa significhi CIDR o netmask, eppero’ funziona…

Potete mette su al volo le macchine virtuali linux senza esse root, potete mette su un loadbalancer ma non sapete che roba è il roundrobin. Daje coi backup. Chili e chili de backup, a caso, uno dopo l’altro, co du click.

Metti su mysql e postgres in PaaS, tanto li gestisce il provider, sticazzi dei WAL. Poi succede un casino e il supporto ènnnammmerda ™.

Poi a fine mese, finiti i crediti aggratise ariva la bolletta, dove t’hanno billato ogni click, ogni byte consumato da storage, rete interna e esterna, dai log, dai backup. Ogni workload che prima non pagavi na lira… mo cor cloud non sai quanto sarà grossa la billata, de sicuro più ‘gnorante de prima, non ce piove.

Eddaje co la trasformazione diggitaleee!! Se riempiono la bocca cor cloud ma er cloud esiste da vent’anni. Parlano de agile ma esiste da vent’anni pure quello e poi non fanno sviluppo ma solo operation. Se vantano der devops pero’ daje cor reboot a mano, cor deploy a mano, daje co le architetture a tre livelli cor mega db transazionale, magari l’Oracolo (ah no non se po’ mette sur cloud perche’ l’Oracolo non vole). No, mica uso i microservizi faccio le architetture a tre livelli cor reverse proxy apache. Continuamo co la foundation su na macchina virtuale custom cor firewall corporate grafico su n’artra region, pero’ security è contenta se usi i vecchi strumenti che conosce. Tutti centralizzati cosi’ vedono tutto. Sur cloud c’e’ er backup fico, su snapshot, automatico. Invece no, usamo no strumento esterno a pagamento unico a tutti i proggetti, cosi’ controllo tutto da na parte sola. E poi er backup fallisce, lo paghi un botto, ce mette na vita. Se lancia a mano, come na vorta.

In pratica, giocamo a fa er cloud, la trasformazione digitale, a vende i proggetti alle grandi aziende, a fa er devops e l’agile, pero’ famo tutto come facevamo na volta, coi stessi modi. Perche’ noi non famo system integrator. Semo partner. Sapemo fa bene l’operation. Ma i servizi da vende? Booohh!!!!! Dovemo fa lift ‘n shift, dovemo porta’ i workload sur cloud. Ma che so poi sti workload? Boooh, non lo sanno. Forse so le macchine virtuali dicono, visto che è lift&shift… boh. La chiamano trasformation ma mica fanno “Move And Improove”, fanno lift&shift. E pero’ almeno sto sur cloud!! Voi mette? Tanto pe le cose fiche ce so i progetti greenfield!

Me chiedo se n’era mejo che tornavamo tutti indietro e chiamavamo er sistemista pe gesti’ tutto on premise, come se faceva prima della pandemia de covviddeee. Almeno sapevi quanto pagavi, come taja i costi, sapevi co chi parla’ se c’avevi problemi, e i dati rimanevano i tua.

Tags: cloud, Linux, server, sysadmin
Filed in General with 0 Comments

29 Lug 21 Don’t use SORBS DNSBL

Sorbs is a infamous DNS blacklist born to try to fight spam. They have a too much restrictive policy that won’t block spam at all but they just create some pain to your MTA.

They can delist your IP a couple of times but if one of your clients get hacked your good quality MTA will be listed for months, even years!

What they don’t understand is that we work with our IP reputation, so just change the IP is not a valid solution. When a valid MTA got abused to send spam the problem is generally solved in hours, or even minutes. A one month list will generate a great problem to the hacked MTA’s clients.

The only solution is: DON’T USE SORBS AT ALL. The list is not business oriented and so is only valid for lamers, bofh sysadmins and nerds…. ok I’m a bofh, nerd sysadmin but I don’t use Sorbs anymore. Sorry.

NOTA: Tell your sysadmins to NOT USE SORBS.

Tags: blacklist, dns, dnsbl, Linux, mailserver, server, sorbs
Filed in General with 0 Comments

13 Mag 21 How to create a Certification Authority with CRL, OCSP and SAN on OpenSSL

Anyone knows that OpenSSL is a very cool full featured, free and open source SSL/TLS framework and toolkit but few people use it to create a custom Private Certification Authority.

The reasons to create a Private CA are many, but they are out of scope here, so I’ll just say how to achieve the goal.

First you must create a personalized version of my openssl.conf configuration file. You can safely do it modifing this labels below and running this one, on a single line:

# C="IT"; ST="Italy"; L="Rome"; O="My org"; OU="My Unit"; CN="My CA"; eml="my@email.com"; CABASEDIR="/tmp/B"; DD=730; mkdir -p "$CABASEDIR"; cd "$CABASEDIR"; echo 'H4sIAGM8nWACA7VVbYvjNhD+rl8hCIYuhOx2r3fQFMOlyRbChc2SXD4cIQRFmsTq
2ZJPkjeb+/UdyXZsX5JCW/rJmmdG8/povOZsQwTsWZG6LWfxeLStJLJuzhui4Ljl
YJzdCmniaDz6fbR8mkwXERHMsR2z0Abvg+lA7M6uMxHbhD2+/0BynUp+irMThtuW
ArFgJEuveLDwjXCTqiLbgenqTVre8vofNYMcMvL2/uHXLbw5UFZqZePCmlAC4To/
tRVeJoplsNW5izGrugPe+kesLkiwk42jySRqumfSEn3387WyDWC+rxArXbeg/Gwz
5nhC1m1pgzkWypnTM2YVlwbWMQdz82L0q1QcWppUc5ZK1zbW5sCU/M4cFngDZulK
SReUWGIAMGqWVRdskeepBEEgYzIdCYH528ZybeAbEkdaJ9WhkDYBsfUtjBHfXsKE
OWfkrnBgg0UjXozp9R0Sg3ijLogI9sCgX+yR/UpjqnS2QyTkciVmt4nj8ky9QH96
pCk4B4ZyLeCubVgPGikVdfBMqvixi7A3RK7NZekxqg2t0SrsvkhT6nO7u3atCb38
HHWnOquEyg8c+pSjeNexau7PIvIwuODAvAW0HeksZwp9XV5pPM6jW+SZd2Dq8ZZz
C9wr7m7cbgVYRW36jcOx5ejLfLWoWteYhRF8+KXL0Scv0ErqqJpokKVRV1V5WnfZ
uSFr/6yRiRvCCpdog73+BKepAOXkXuJS+gonKYYsPeLTJ2vNbb4huA8lHyNxnWFS
OYtrdfjHaLZ8IrbY/Ykt6bpImE3+1n1fWluAISisLDtAzNFQ4uT7Qh6kY+lSHhRz
hQESHo0A8enCdD5evng7fCVkXS/D67nuGS7XJprSagF5IWSY3kXMPho+KS7zBEyG
iff9L6ENXCYVVqEZYcl9jltGOX/8j80pP6O0pGUpDcNuxxFOpC1nigW86FDnRz9Z
qfaa9M7upyiOOPc0+uhHWerXYSdt/mV+FTeqNGumXGu7M0Wr63wx8y323R3jpPy5
zqAuMjC4rPF/rL623JAe/aILQ8eLGbWJLlJBd0DzYpeGrYuvXVOeaG2Bnrwdzjz3
0chqMR08xIlz+fD+PjtxNsA3fP5Rr8+xmggj6tkZmIub9EowjJXBEQkGhLNpKNf+
diMOC2Eq1/4ZUHzyuUZGmn9Wh7/bDeJTH5wjBaaEP2OPftaUCUGXo2daKL+ykBXU
JUB7lClBc8zXb0L5HW5PtdefPC+Hx+NxkGjrfIgA1AL5CyDty6++CQAA' | openssl base64 -d | gunzip |sed "s/%C%/$C/g"|sed "s/%ST%/$ST/g"|sed "s/%L%/$L/g"|sed "s/%O%/$O/g"|sed "s/%OU%/$OU/g"|sed "s/%eml%/$eml/g"|sed "s|%CABASEDIR%|$CABASEDIR|g"|sed "s/%DD%/$DD/g" > openssl.conf

So, those labels must be modified to your needs:

C="IT"; ST="Italy"; L="Rome"; O="My org"; OU="My Unit"; eml="my@email.com" CABASEDIR="/tmp/B"; DD=730

where C is your Country, ST is your State or Province name, L is locality, O is organization, OU is organization unit, eml is your CA email (if any), CABASEDIR is the directory that will hold all your CA stuff (private keys, certificates, config files, certificate serials and ca db) and DD is your default certificate validity in days.

At this time you should have a file called openssl.conf into your CABASEDIR directory.

As an alternative, you could directly copy and modify the openssl.conf file here.

Next, you obviously need to create the private key and self sign the certificate of your brand new CA, in this example we’ll create a clear RSA private key with 4096 bit encryption length, and a CA certificate that is valid for about 10 years. You mileage may vary, feel free to customize things:

# openssl genrsa -out ca.key 4096
# openssl req -new -x509 -days 3650 -key ca.key -out ca.crt -extensions 'v3_ca' -config openssl.conf

This way you’ll get a clear (not encrypted) private key, so a password is not needed when you’ll going to use it to sign things, generate new certificates, etc. btw, for security reasons, you may need to encrypt your PK with a passphrase. In that case, add -des attribute to your openssl genrsa command. Also you’ll have your precious CA certificate.
Choose a cool CN for your CA Name.

Now it’s time to create a large random number that will be used by OpenSSL as a starting point for your certificates’ Serial Numbers. You can create a large random number with this one (you could also create one by hand…):

# hexdump -n 20 -e '20/1 "%02X" 1 "\n"' /dev/random > certs.seq
C17FA21B11EE604633317891658DCC421F2EDFD5

Perfect. Now proceed creating an empty file called certs.db:

# touch certs.db

Also create a starting serial number for revoked certificates and an empty CRL:

# echo 00 > crlserial
# openssl ca -config openssl.conf -keyfile ca.key -cert ca.crt -gencrl -out crl.pem 
Using configuration from openssl.conf

At this point, you should have something like this:

# ls -lrth
totale 16K
-rw-r--r-- 1 dino dino 1,5K mag 5 20:06 openssl.conf
-rw------- 1 dino dino 3,2K mag 5 20:06 ca.key
-rw-r--r-- 1 dino dino 2,0K mag 5 20:07 ca.crt
-rw-r--r-- 1 dino dino 41 mag 5 20:07 certs.seq
-rw-r--r-- 1 dino dino 3 mag 5 20:08 crlserial
-rw-r--r-- 1 dino dino 999 mag 5 20:08 crl.pem
-rw-r--r-- 1 dino dino 0 mag 5 20:08 certs.db

Well. Now we can start creating our server (or client) certificates.
We start from its private key (here at 2048 bit but you can choose your own key length):

# openssl genrsa -out server.key 2048

And now the certificate:

# openssl req -new -key server.key -out server.csr -extensions 'v3_req' -config openssl.conf
# openssl ca -cert ca.crt -keyfile ca.key -in server.csr -out server.crt -config openssl.conf

If you don’t want to use the default certificates expiry days setted into openssl.conf (param default_days), you can pass the -days attribute to the last command, for example -days 365.
When asked, pay attention to correctly set all the requested attributes, principally the Common Name. Press Y when asked to sign and commit.

If everything gone OK you’ll have your brand new key and certificate:

# ls -lrth server.*
-rw------- 1 dino dino 1,7K mag 5 22:26 server.key
-rw-r--r-- 1 dino dino 1,2K mag 5 22:26 server.csr
-rw-r--r-- 1 dino dino 5,7K mag 5 22:28 server.crt

You can check the certificate with this command:

# openssl x509 -in server.crt -noout -text

Please note that the new certificate is signed by our CA, and also has the following useful properties:

    X509v3 extensions:
        X509v3 Key Usage: 
            Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment
        X509v3 Basic Constraints: 
            CA:FALSE
        X509v3 Extended Key Usage: 
            TLS Web Server Authentication, TLS Web Client Authentication
Signature Algorithm: sha256WithRSAEncryption

Also note that the generated certificate is valid for both Server and Client purposes and has a X509v3 CRL Distribution Points extension that points to URI:http://myca.com/crl.pem: please change this value inside your openssl.conf before generating the CA certificate. This is where you’ll publish your CRL.

As an added bonus, you could also add SANs (Subject Alternative Names) to your certificate, if you like. This permits you to have cool certificates like wildcard domains, multiple domains and IP address on your certificates.

To add a SAN you can modify the last line of openssl.conf configuration file, so that you can include your SANs. For example:

...
[v3_req]
# To add SAN uncomment the # and personalize
subjectAltName=email:copy,DNS:www.host.com,DNS:host.com

and create your CSR and certificate like done above (but first remember to revoke or manually remove the certificate from certs.db or you’ll get the “ERROR:There is already a certificate for…” error):

# openssl req -new -key server.key -out server.csr -extensions 'v3_req' -config openssl.conf
# openssl ca -cert ca.crt -keyfile ca.key -in server.csr -out server.crt -config openssl.conf

To revoke a certificate (in this example is called 792FCB9AE9BBBFFAE33796CF3D1D0D7B6AF399DF.pem) you can simply do this (this will set the given certificate as revoked into the certs.db file):

# openssl ca -config openssl.conf -keyfile ca.key -cert ca.crt -revoke 792FCB9AE9BBBFFAE33796CF3D1D0D7B6AF399DF.pem -crl_reason unspecified
Using configuration from openssl.conf
Revoking Certificate 792FCB9AE9BBBFFAE33796CF3D1D0D7B6AF399DF.
Data Base Updated

After that you need to update the CRL with all the revoked certificates inside. Also, remember to refresh the CRL with the same command almost every default_crl_days (check openssl.conf) even if no certificates are revoked or your CRL will expire:

# openssl ca -config openssl.conf -keyfile ca.key -cert ca.crt -gencrl -updatedb -out crl.pem

At this poin you might want to arrange your OCSP responder with its key and certificate.

Please note that the configuration of OCSP Stapling or responder is out of scope in this article, we just realized how to create its certificates with OpenSSL. If you don’t need OCSP on your certificates, left commented out the authorityInfoAccess attribute in openssl.conf and skip this last step, btw I can tell you, as a testing purposes, how to create a OCSP test responder:

# openssl genrsa -out ocsp.key 2048
# openssl req -new -key ocsp.key -out ocsp.csr -extensions 'v3_req' -config openssl.conf
# openssl ca -cert ca.crt -keyfile ca.key -in ocsp.csr -out ocsp.crt -extensions ocsp -config openssl.conf

When you create the OCSP certificate, keep in mind that the common name must match the OCSP;URI.0 attribute defined into the [ocsp_info] section of your openssl.conf.

# openssl ocsp -index certs.db -port 9999 -rsigner ocsp.crt -rkey ocsp.key -CA ca.crt
ocsp: waiting for OCSP client connections...

And then, to test:

# openssl ocsp -issuer ca.crt -CAfile ca.crt -cert server.crt -url http://ocsp:9999
Response verify OK
server.crt: good
	This Update: May 13 15:10:17 2021 GMT

Now, we try to revoke the server certificate, just for test:

openssl ca -config openssl.conf -keyfile ca.key -cert ca.crt -revoke server.crt -crl_reason unspecified
Using configuration from openssl.conf
Adding Entry with serial number 2DE87D684C64D0BB4B23D0BC9959B8EB23AF932F to DB for /C=IT/ST=Italy/L=Rome/O=My org/OU=My Unit/CN=myserver/emailAddress=my@email.com
Revoking Certificate 2DE87D684C64D0BB4B23D0BC9959B8EB23AF932F.
Data Base Updated

We must also update our CRL and check if the revoked certificate is inserted into our CRL:

# openssl ca -config openssl.conf -keyfile ca.key -cert ca.crt -gencrl -updatedb -out crl.pem                            
Using configuration from openssl.conf
# openssl verify -crl_check -CAfile ca.crt -CRLfile crl.pem server.crt 
C = IT, ST = Italy, L = Rome, O = My org, OU = My Unit, CN = myserver, emailAddress = my@email.com
error 23 at 0 depth lookup: certificate revoked
error server.crt: verification failed

If we recheck now our OCSP responder:

openssl ocsp -issuer ca.crt -CAfile ca.crt -cert server.crt -url http://ocsp:9999 
Response verify OK
server.crt: revoked
	This Update: May 13 15:23:34 2021 GMT
	Reason: unspecified
	Revocation Time: May 13 15:22:14 2021 GMT

Finaly, the mission is complete!!!!

I spent hours getting this things done!! It’s now time to collect and share back to everybody. Bye!!! Ciao, Dino 🙂

openssl.conf.zip Download

Tags: ca, certificate, certification authority, Linux, opensource, openssl, server, x509
Filed in General with 0 Comments

28 Dic 19 How to encrypt and decrypt a text file with vim

I always use VIM any single day of my life starting from 1994, and I did not realize that it’s now possible to encrypt and decrypt files with it without using gnupg! This is one of the reasons why I love vim so much: there are always things that you can learn from it!
So, I’ll now show you how to use vim to encrypt and decrypt your text files.

We can start creating a new encrypted text file with the command below:

# vim -x encryptedfile.txt
Enter encryption key:
Enter same key again:

Once you’ve entered an encryption key (WARNING: don’t forget it or you will lose your clear text data forever!!!) you can use vim as always, write your clear text data, save and quit.

# file encryptedfile.txt
encryptedfile.txt: Vim encrypted file data
# cat encryptedfile.txt
VimCrypt~03!�-
�
��[�v��>P��(�%

So, your file is now encrypted. To recover access to your clear text data, just open the file back with vim using the correct passphrase you used to create your file:

# vim encryptedfile.txt
Need encryption key for "encryptedfile.txt"
Enter encryption key:

You could also decide to change your passphrase or encrypt a clear text file, using the :set key=pwd vim command, like this:

:set key=mystrongpasswordyou'llneverfind
:wq
# file encryptedfile.txt 
encryptedfile.txt: Vim encrypted file data

If you want to decrypt the file and save it clear text, just use an empty key with the :set key= command, after you opened the encrypted file with the correct passphrase:

:set key=
:wq
# file encryptedfile.txt
encryptedfile.txt: ASCII text

So, this way you can create an encrypted file protected with a passphrase, get read/write access to the encrypted file, decrypt the file to clear text and change the passphrase.

Hope you all will enjoy this like me!!

Tags: decrypt, encrypt, file, Linux, text, vim
Filed in General with 0 Comments

23 Ott 18 How to disable Diffie-Hellman ciphers on apache

If you are getting errors like “DH key too small” you can avoid using DH ciphersuites on apache.
You can obtain that using Perfect forward secrecy, or disabling all DH ciphersuites like this:

SSLCipherSuite ALL:!EXP:!NULL:!DH:!LOW

Tags: apache, cipher, debian, http, https, Linux, openssl, server, sysadmin
Filed in Linux OS, Open source, Sysadmin pills with 0 Comments

06 Lug 18 How to setup xtables GeoIP for iptables on OpenWRT

I’m not racist, but sometimes you may need to keep some people out of your network based on its geographical region… in reality I’m referring to its public source IP address.

Requests are coming from the big internet, so you can check if the source IP of the peers that are trying to connect to your powerful OpenWRT Linux router are coming from a particular region and you may want to stop all those requests accordingly.

If you are using iptables and ip6tables (if you are on linux this is the standard) you can setup GeoIP based packet filtering with iptables: it’s an iptables extension that make use of the glorious and free MaxMind GeoLite Legacy Downloadable Database (that is now dying…).

On OpenWRT this is supported automatically, but generally your router does not have all the disk space to host the entire GeoLite GeoIP database. So, you need to prepare all the records of the geographical regions that you want to block on your firewall.

Take a linux workstation or server (your notebook, a raspberry pi you own always on, one of your client’s server… better not IMHO…) and run those commands:

root@firegate2:~# mkdir /tmp/temp
root@firegate2:~# cd /tmp/temp
root@firegate2:/tmp/temp# /usr/share/xt_geoip/xt_geoip_dl
–2018-07-05 23:54:40– http://geolite.maxmind.com/download/geoip/database/GeoIPv6.csv.gz
Risoluzione di geolite.maxmind.com (geolite.maxmind.com)… 104.16.37.47, 104.16.38.47, 2400:cb00:2048:1::6810:262f, …
Connessione a geolite.maxmind.com (geolite.maxmind.com)|104.16.37.47|:80… connesso.
Richiesta HTTP inviata, in attesa di risposta… 200 OK
Lunghezza: 1715246 (1,6M) [application/octet-stream]
Salvataggio in: “GeoIPv6.csv.gz”

GeoIPv6.csv.gz 100%[================================================================>] 1,64M –.-KB/s in 0,1s

2018-07-05 23:54:40 (10,9 MB/s) – “GeoIPv6.csv.gz” salvato [1715246/1715246]

–2018-07-05 23:54:40– http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
Riutilizzo della connessione esistente a geolite.maxmind.com:80.
Richiesta HTTP inviata, in attesa di risposta… 200 OK
Lunghezza: 2540312 (2,4M) [application/zip]
Salvataggio in: “GeoIPCountryCSV.zip”

GeoIPCountryCSV.zip 100%[================================================================>] 2,42M 11,3MB/s in 0,2s

2018-07-05 23:54:40 (11,3 MB/s) – “GeoIPCountryCSV.zip” salvato [2540312/2540312]

TERMINATO –2018-07-05 23:54:40–
Tempo totale: 0,4s
Scaricati: 2 file, 4,1M in 0,4s (11,1 MB/s)
Archive: GeoIPCountryCSV.zip
inflating: GeoIPCountryWhois.csv
root@firegate2:/tmp/temp#
root@firegate2:/tmp/temp# /usr/share/xt_geoip/xt_geoip_build -D . *.csv
209370 entries total
0 IPv6 ranges for A1 Anonymous Proxy
32 IPv4 ranges for A1 Anonymous Proxy
0 IPv6 ranges for A2 Satellite Provider
36 IPv4 ranges for A2 Satellite Provider
3 IPv6 ranges for AD Andorra
26 IPv4 ranges for AD Andorra
51 IPv6 ranges for AE United Arab Emirates
302 IPv4 ranges for AE United Arab Emirates
… and continues …
11 IPv6 ranges for ZM Zambia
64 IPv4 ranges for ZM Zambia
14 IPv6 ranges for ZW Zimbabwe
59 IPv4 ranges for ZW Zimbabwe
root@firegate2:/tmp/temp#

The script /usr/share/xt_geoip/xt_geoip_build it’s part of xtables-addons package that you may or may not find on your distribution. Search on google for that.

This will create LE and BE directories. Now you want to extract only the geo regions that you need (ex: CN,UA,TW,VN,VG,KP,VI,KR), like that:

root@firegate2:/tmp/temp# du -csh BE/CN.iv? BE/UA.iv? BE/TW.iv? BE/VN.iv? BE/VG.iv? BE/KP.iv? BE/VI.iv? BE/KR.iv? LE/CN.iv? LE/UA.iv? LE/TW.iv? LE/VN.iv? LE/VG.iv? LE/KP.iv? LE/VI.iv? LE/KR.iv?
36K BE/CN.iv4
48K BE/CN.iv6
24K BE/UA.iv4
16K BE/UA.iv6
8,0K BE/TW.iv4
4,0K BE/TW.iv6
4,0K BE/VN.iv4
4,0K BE/VN.iv6
4,0K BE/VG.iv4
4,0K BE/VG.iv6
4,0K BE/KP.iv4
0 BE/KP.iv6
4,0K BE/VI.iv4
4,0K BE/VI.iv6
8,0K BE/KR.iv4
4,0K BE/KR.iv6
36K LE/CN.iv4
48K LE/CN.iv6
24K LE/UA.iv4
16K LE/UA.iv6
8,0K LE/TW.iv4
4,0K LE/TW.iv6
4,0K LE/VN.iv4
4,0K LE/VN.iv6
4,0K LE/VG.iv4
4,0K LE/VG.iv6
4,0K LE/KP.iv4
0 LE/KP.iv6
4,0K LE/VI.iv4
4,0K LE/VI.iv6
8,0K LE/KR.iv4
4,0K LE/KR.iv6
352K totale

So, in this case we will need 352 Kb of router disk space. After we created the /usr/share/xt_geoip/BE/ and /usr/share/xt_geoip/LE/ directories on our router’s filesystem:

root@dam2ktplinkrouter:~# mkdir -p /usr/share/xt_geoip/BE /usr/share/xt_geoip/LE

We are going to copy those files on our OpenWRT router:

root@firegate2:/tmp/temp# scp LE/CN.iv? LE/UA.iv? LE/TW.iv? LE/VN.iv? LE/VG.iv? LE/KP.iv? LE/VI.iv? LE/KR.iv? root@192.168.10.1:/usr/share/xt_geoip/LE
root@192.168.10.1’s password:
CN.iv4 100% 35KB 109.8KB/s 00:00
CN.iv6 100% 47KB 76.0KB/s 00:00
UA.iv4 100% 23KB 108.4KB/s 00:00
UA.iv6 100% 15KB 87.9KB/s 00:00
TW.iv4 100% 4704 47.5KB/s 00:00
TW.iv6 100% 3104 77.8KB/s 00:00
VN.iv4 100% 3888 98.9KB/s 00:00
VN.iv6 100% 3584 87.9KB/s 00:00
VG.iv4 100% 536 40.1KB/s 00:00
VG.iv6 100% 224 20.0KB/s 00:00
KP.iv4 100% 40 3.9KB/s 00:00
KP.iv6 100% 0 0.0KB/s 00:00
VI.iv4 100% 392 30.6KB/s 00:00
VI.iv6 100% 160 14.4KB/s 00:00
KR.iv4 100% 8128 105.8KB/s 00:00
KR.iv6 100% 3616 87.0KB/s 00:00
root@firegate2:/tmp/temp# scp BE/CN.iv? BE/UA.iv? BE/TW.iv? BE/VN.iv? BE/VG.iv? BE/KP.iv? BE/VI.iv? BE/KR.iv? root@192.168.10.1:/usr/share/xt_geoip/BE/
root@192.168.10.1’s password:
CN.iv4 100% 35KB 114.4KB/s 00:00
CN.iv6 100% 47KB 66.0KB/s 00:00
UA.iv4 100% 23KB 115.2KB/s 00:00
UA.iv6 100% 15KB 74.9KB/s 00:00
TW.iv4 100% 4704 93.8KB/s 00:00
TW.iv6 100% 3104 75.7KB/s 00:00
VN.iv4 100% 3888 108.8KB/s 00:00
VN.iv6 100% 3584 76.2KB/s 00:00
VG.iv4 100% 536 40.3KB/s 00:00
VG.iv6 100% 224 20.0KB/s 00:00
KP.iv4 100% 40 4.1KB/s 00:00
KP.iv6 100% 0 0.0KB/s 00:00
VI.iv4 100% 392 33.1KB/s 00:00
VI.iv6 100% 160 15.4KB/s 00:00
KR.iv4 100% 8128 111.1KB/s 00:00
KR.iv6 100% 3616 71.8KB/s 00:00
root@firegate2:/tmp/temp#

OK, now we have our pieces of geo ip informations. We now need to install the iptables-mod-geoip from the LuCI web interface (or by hand if you like).
Now you can create your Firewall Traffic Rules. Go to Network -> Firewall -> Traffic Rules on the router’s LuCI web interface and add a custom traffic rule. In my case I created 2 that I called “CinamerdaMuoriUDPeTCP” and “CinamerdaMuoriICMP”, the first to block TCP and UDP and the second to block ICMP traffic.

After that you can setup your custom rule setting your protocol and address families, set “source zone” to WAN, “destination zone” to “Any zone (forward)”, action to DROP, and the “Extra arguments” field like this:

-m geoip –source-country CN,UA,TW,VN,VG,KP,VI,KR

You can check the image below.

OpenWRT Firewall geoip example

Now you can say hello to most chinaspammers and something like that, but don’t abuse, this is not ethic if you set up this on a server that offers some sort of public service.

Tags: blacklist, firewall, geoip, iptables, ipv6, Linux, openwrt, raspberry pi, server, sysadmin, tcp
Filed in General, Linux OS, Open source, Sysadmin pills with 0 Comments

21 Apr 17 How to create a sparse file from a block device

This is how to create a sparse file dump from a block device:
cp --sparse=always <(dd if=/dev/vg0/vmservice1 bs=8M iflag=direct | pv -pre --size=20G) /opt/backups/vmservice1.dat

Tags: Linux, lvm, server, sysadmin
Filed in Linux OS, Open source, Sysadmin pills with 0 Comments

14 Gen 17 Come creare un gateway IPv6 su Fastweb

Fastweb ha deciso per il momento di non fornire IPv6 nativo ai propri clienti, e inoltre da qualche giorno ha disabilitato il tunnel TSP (tsp-auth.ipv6.fastweb.it) il quale non risulta piu’ raggiungibile.

Visto che nel mio caso ho un router fastweb Argo 55+ su fibra 100, e il suddetto router non supporta IPv6, quando vado sulla MyFastPage e cerco di attivare il protocollo IPv6 il sistema mi dice che devo sostituire il router con un nuovo modello. Io NON voglio cambiare router perche’ lo considero estremamente stabile e performante.

Come posso quindi attivare IPv6 nella mia rete domestica senza dover cambiare router?

Fastweb porta IPv6 ai suoi utenti tramite 6rd (https://en.wikipedia.org/wiki/IPv6_rapid_deployment). Questo significa che e’ probabilmente possibile ottenere la subnet in tunnel anche su linux.

Ho preso uno dei miei raspberry pi con raspbian e ci ho installato il pacchetto radvd (sudo apt-get install radvd), poi nel mio /etc/network/interfaces ho messo questo:

iface eth0 inet6 static
address 2001:b07:27b:7b7b::1
netmask 64

auto ipv6fastweb
iface ipv6fastweb inet6 v4tunnel
netmask 64
endpoint 81.208.50.214
up ip -6 route add default dev ipv6fastweb
down ip -6 route del default dev ipv6fastweb

Invece di usare come indirizzo ip 2001:b07:27b:7b7b::1 devi calcolarti il tuo a partire dal tuo IP pubblico fisso che ti ha fornito fastweb. Puoi ottenere il tuo ip pubblico ad esempio da qui: http://whatismyipaddress.com/

Facciamo finta che tu abbia l’IP 2.123.123.123, devi convertirlo in esadecimale, ad esempio cosi:

printf “%x%02x:%x%02x::\n” `echo 2.123.123.123 |tr . ” “`

Quello che otterrai, ad esempio 27b:7b7b:: va accodato al prefisso di fastweb (2001:b07:) e come suffisso accodi il numero 1.

In questo caso, quindi, l’IP diventa:

2001:b07:27b:7b7b::1 che e’ stato costruito da [2001:b07]:[b07:27b:7b7b]::[1]. La prima e’ fissa, la seconda dipende dal tuo IP pubblico e infine 1. Questo andra’ messo nella riga “address” nel file /etc/network/interfaces che ti dicevo prima e va anche messo nella direttiva “prefix” del radvd.conf, ma in ques’ultimo caso senza l’1 finale.

Crea il file /etc/radvd.conf e mettici dentro questo:

interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;

prefix 2001:b07:27b:7b7b::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr on;
};

RDNSS 2001:4860:4860::8888
{
AdvRDNSSLifetime 20;
};
};

Riavvia il raspberry pi e se tutto va bene, sempre nel raspberry pi dovrai avere una scheda di rete virtuale chiamata ipv6fastweb senza IP usabili ma che serve per creare il tunnel con il border gateway di fastweb (81.208.50.214). Se non funziona, prova a cercare un altro border gateway, magari chiedendo al numero verde o cercando su internet. A me funziona con questo. Metti quello giusto alla direttiva “endpoint” del file “interfaces”.

Poi avrai l’IP pubblico IPv6 che ti sei calcolato (nel caso di esempio 2001:b07:27b:7b7b::1/64) sulla scheda eth0 e avrai il tuo radvd che invia i router advertisement ipv6 alla tua rete.

Ogni PC nella tua rete che supporta IPv6 otterrà un IP pubblico nella subnet che ti sei calcolato, e sara’ raggiungibile direttamente da internet tramite IPv6.

Bello no? Ovviamente se abilitate questo dovete disabilitare IPv6 sul router di fastweb perche’ sara il vostro raspberry pi a fare da router ipv6.

Commentate sotto, mi raccomando! Fatemi sapere.

A me funziona perfettamente e sono molto felice. Credo che Fastweb dovrebbe creare una guida ufficiale su questo per il bene degli utenti. Ci ho messo 2 ore a farlo funzionare, con una guida ci avrei messo 2 minuti.