NCDU - How to detect larger directory in linux

If you want search for big files, or big directory..
If you want to do maintenance for you free space,

You have to install NCDU.

I try it! It is wonderful..
It works with a intuitive text gui.

I install it by using
yum install ncdu

and launch by command ncdu after installing.

Http Url and Wildfly HeartBeat

For my projects I need a script that check for http response correctly with 200 or 301 and then check for wildfly process status.

So I write the following script.

The script write in a log file the various steps.

The mail dir is a directory where other daemon check for new file and send it to mail..

The lock files avoid to write too many file on mail dir in case of heart down. It rewrite the error at most each hour.

The script include two url heartbeat, you can copy & paste other ones.

If wildfly process is down the script try to restart.

I'm not a bash script expert feel free to comment in order to correct or improve the scripts!

#!/bin/bash

LOG_FILE=/home/you/url_heartbeat.log #FILE FOR LOGGING

MAIL_DIR_FILE=/home/mailDir/heartbeat.log #FILE PATH FOR COPING THE LOG IF TEST IS NOT PASSED (IN THIS DIR A DAEMON WILL SEND A MAIL)

WILDFLY_SERVER_LOG=/opt/wildfly/standalone/log/server.log #WILDFLY SERVER.LOG TO SEE IF PARTICULAR ERROR OCCURS

CURRENT_HOUR_LOCK_FILE="$(date +%H).fail"

PREVIOUS_HOUR_LOCK_FILE="$(expr $(date +%H) - 1).fail"

rm -f $PREVIOUS_HOUR_LOCK_FILE #REMOVE PREVIOUS HOUR FILE

#DO TEST IF CURRENT_HOUR_LOCK_FILE DOESN'T EXIST

if [ -f $CURRENT_HOUR_LOCK_FILE ];

then

echo "$(date) Skip Test file $CURRENT_HOUR_LOCK_FILE exist" >> $LOG_FILE

else

echo -e "$(date) start url_heartbeat" > $LOG_FILE

TEST_PASSED=true

RESPONSE_CODE=$(curl -sL -w "%{http_code}" "http://www.example1.com/index.jsp" -o /dev/null) #GET URL RESPONSE CODE

if [ "$RESPONSE_CODE" = 301 ] || [ "$RESPONSE_CODE" = 200 ] #IF 301 REDIRECT OR 200 OK

then

echo -e "OK Example1: $RESPONSE_CODE" >> $LOG_FILE

else

echo -e "FAIL Example1: $RESPONSE_CODE" >> $LOG_FILE

TEST_PASSED=false

RESPONSE_CODE=$(curl -sL -w "%{http_code}" "http://example2/index.jsp" -o /dev/null)

if [ "$RESPONSE_CODE" = 301 ] || [ "$RESPONSE_CODE" = 200 ]

then

echo -e "OK Example2: $RESPONSE_CODE" >> $LOG_FILE

else

echo -e "FAIL Example2: $RESPONSE_CODE" >> $LOG_FILE

TEST_PASSED=false

WILDFLY_STATUS=$(/etc/init.d/wildfly status) #CHECK WILDFLY PROCESS STATUS

echo -e $WILDFLY_STATUS

if [[ "$WILDFLY_STATUS" == *running* ]]

then

echo -e "\nOK Wildfly Status: $WILDFLY_STATUS\n" >> $LOG_FILE

else

echo -e "\nFAIL Wildfly Status: $WILDFLY_STATUS\n" >> $LOG_FILE

TEST_PASSED=false #IF WILDFLY DOWN TRY TO RESTART

/etc/init.d/wildfly restart #RESTART COMMAND

sleep 10m #WAIT RESTARTING

WILDFLY_STATUS=$(/etc/init.d/wildfly status) #CHECK WILDFLY PROCESS STATUS AGAIN

echo -e "\nAfter Restart Wildfly Status: $WILDFLY_STATUS\n" >> $LOG_FILE

if [[ "$WILDFLY_STATUS" == *running* ]]

then

echo -e "Restart Successfully\n" >> $LOG_FILE

rm -f $CURRENT_HOUR_LOCK_FILE #TEST FAILED BUT RESTART SUCCESSFULLY SO DELETE LOCK FILE IN ORDER TO DOESN'T SKIP THE TESTS AT NEXT CRON HIT

echo -e "test passed: $TEST_PASSED" >> $LOG_FILE

echo -e "test passed: $TEST_PASSED"

if [ "$TEST_PASSED" = false ]

then

echo -e "NOT PASSED\n\n\n $(tail -100 $WILDFLY_SERVER_LOG)" >> $LOG_FILE

touch $CURRENT_HOUR_LOCK_FILE #MAKE A FILE IF TEST IS NOT PASSED WITH CURRENT HOUR AS NAME

cp $LOG_FILE $MAIL_DIR_FILE

#TEST CASE

#CURRENT_HOUR_LOCK_FILE EXIST -> SKIP TEST

#CURRENT_HOUR_LOCK_FILE NOT EXIST -> DO TEST

#ALL TEST OK -> CHECK LOG FILE FOR ALL OK ECHOES

#AT LEAST ONE RESPONSE_CODE URL TEST NOT OK ->

#1) SEE THE CURRENT_HOUR_LOCK

#2) THE LOG_FILE IS COPIED IN MAIL_DIR PATH

#3) THE NEXT CRON HIT SKIP

#WILDFLY IS NOT RUNNING -> AS THE PREVIOUS CASE AND TRY TO RESTART

#IF RESTART IS OK THE CURRENT_HOUR_LOCK IS DELETED AND NEXT CRON HIT DO THE TEST

#IF RESTART GOES WRONG THE NEXT TEST IS SKIPPED

SSH won't Work - expecting SSH2_MSG_KEX_DH_GEX_GROUP

Short Post for a little-big Problem.

If your firewall is open for ssh, telnet on ssh port works but ssh won't work.

Try to debug the ssh connection by using verbose debug

ssh -v user@yourip

if ssh hangs on

"expecting SSH2_MSG_KEX_DH_GEX_GROUP"

and then ssh returns with the following error: "Read from socket failed: Operation timed out"

Probably you have an MTU/fragmentation problem and you will solve the problem by set correctly a new mtu value for network interface.

So launch by terminal the following command:

sudo ifconfig en1 mtu 576

Where en1 is your active network interface.

Wildfly SSL Client Authentication

In order to ask for a client ssl authentication you have to setup standalone.xml, web.xml and jboss-web.xml in different section.

1) First of all say to UNDERTOW -> https listener that ssl authentication could occurs

<buffer-cache name="default"/>

<http-listener name="default" socket-binding="http"/>

<https-listener name="https-listener" socket-binding="https" security-realm="MySecurityRealm"/>

<access-log pattern="common" directory="${jboss.server.log.dir}" prefix="access"/>

<filter-ref name="server-header"/>

<filter-ref name="x-powered-by-header"/>

</host>

</server>

<http-listener name="default_secondary" socket-binding="http_secondary"/>

<https-listener name="https-listener_secondary" socket-binding="https_secondary" security-realm="MySecurityRealm_secondary" verify-client="REQUESTED"/>

<access-log pattern="common" directory="${jboss.server.log.dir}" prefix="access"/>

<filter-ref name="server-header"/>

<filter-ref name="x-powered-by-header"/>

</host>

</server>

<servlet-container name="default" default-encoding="utf-8">

<jsp-config/>

</servlet-container>

</handlers>

<response-header name="server-header" header-name="Server" header-value="WildFly/8"/>

<response-header name="x-powered-by-header" header-name="X-Powered-By" header-value="Undertow/1"/>

</filters>

</subsystem>

2) Set also in the https-listener security realm that ssl authentication can occurs.

So add <authentication> in order to setup the truststore path (the <ssl> section instead indicate the keystore for server certificate).

<security-realm name="MySecurityRealm_secondary">

<server-identities>

</ssl>

</server-identities>

</authentication>

</security-realm>

3) In order to link the war application with wildfly configuration you have to define a security-domain using login module (certificateRoles login module)

<security-domain name="sdi_webservice_client_cert_domain" cache-type="default">

<login-module code="CertificateRoles" flag="required">

<module-option name="verifier" value="org.jboss.security.auth.certs.AnyCertVerifier"/>

<module-option name="securityDomain" value="sdi_webservice_client_cert_domain"/>

<module-option name="rolesProperties" value="file:${jboss.server.config.dir}/app-roles.properties"/>

</login-module>

</authentication>

</security-domain>

Make also the app-roles.properities file with the format

certificate DN=nome_ruolo

# sample app-roles.properties file

CN\=SDI-MRRGPP80L31D960U,\ OU\=Servizi\ Telematici,\ O\=Agenzia\ delle\ Entrate,\ C\=IT=SDI_USER

You have to escape the = symbol and space with backslash like the example.

4) In the web.xml war setup the client-cert authentication type, the role, and set as realm name the security domain name

<?xml version="1.0" encoding="UTF-8"?>

<web-app version="3.1" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd">

<session-config>

<session-timeout>

</session-timeout>

</session-config>

<security-constraint>

<display-name>Constraint1</display-name>

<web-resource-collection>

<web-resource-name>sdi</web-resource-name>

<url-pattern>/*</url-pattern>

</web-resource-collection>

<auth-constraint>

<role-name>SDI_USER</role-name>

</auth-constraint>

</security-constraint>

<login-config>

<auth-method>CLIENT-CERT</auth-method>

<realm-name>sdi_webservice_client_cert_domain</realm-name>

</login-config>

<security-role>

<role-name>SDI_USER</role-name>

</security-role>

</web-app>

5) Finally also the jboss-web.xml have to set the security domain used

<jboss-web>

<server-instance>default-server_secondary</server-instance>

<virtual-host>testfoo.com</virtual-host>

<security-domain>sdi_webservice_client_cert_domain</security-domain>

</jboss-web>

Wildfly Deploy on different server and host

If you setup different <server> in undertow subsystem and each <server> has its own virtual host then when deploy a war you have to setup the jboss-web.xml correctly

<jboss-web>

<server-instance>default-server_secondary</server-instance>

<virtual-host>server.mydomain.com</virtual-host>

</jboss-web>

Wildfly Multiple IP and Multiple SSL Certificate

Wildfly can manage different ip in the same standalone server.

The final target is two manage different ssl certificate for each ip.

Due to ssl handshake limitation each hostname needs to be associate with different ip

For example

https://localhost:8181 send certificate1

https://192.168.1.3:8181 send certificate 2

But first start to setup the new ip interface

<inet-address value="${jboss.bind.address.management:127.0.0.1}"/>

</interface>

<inet-address value="${jboss.bind.address:127.0.0.1}"/>

</interface>

<inet-address value="${jboss.bind.address:192.168.1.3}"/>

</interface>

<inet-address value="${jboss.bind.address.unsecure:127.0.0.1}"/>

</interface>

</interfaces>

Adding the public_secondary

Next add to socket bindings section a new socketbinding using the new interface

<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">

<socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>

<socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>

<socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>

<socket-binding name="http" port="${jboss.http.port:8080}"/>

<socket-binding name="https" port="${jboss.https.port:8181}"/>

<socket-binding name="http_secondary" interface="public_secondary" port="${jboss.http.port:8080}"/>

<socket-binding name="https_secondary" interface="public_secondary" port="${jboss.https.port:8181}"/>

<socket-binding name="jacorb" interface="unsecure" port="3528"/>

<socket-binding name="jacorb-ssl" interface="unsecure" port="3529"/>

<socket-binding name="messaging-group" port="0" multicast-address="${jboss.messaging.group.address:231.7.7.7}" multicast-port="${jboss.messaging.group.port:9876}"/>

<socket-binding name="txn-recovery-environment" port="4712"/>

<socket-binding name="txn-status-manager" port="4713"/>

<outbound-socket-binding name="mail-smtp">

<remote-destination host="localhost" port="25"/>

</outbound-socket-binding>

</socket-binding-group>

the new socket bindings will be used in undertow configuration in <server> section

<buffer-cache name="default"/>

<http-listener name="default" socket-binding="http"/>

<https-listener name="https-listener" socket-binding="https" security-realm="MySecurityRealm"/>

<access-log pattern="common" directory="${jboss.server.log.dir}" prefix="access"/>

<filter-ref name="server-header"/>

<filter-ref name="x-powered-by-header"/>

</host>

</server>

<http-listener name="default_secondary" socket-binding="http_secondary"/>

<https-listener name="https-listener_secondary" socket-binding="https_secondary" security-realm="MySecurityRealm_secondary"/>

<access-log pattern="common" directory="${jboss.server.log.dir}" prefix="access"/>

<filter-ref name="server-header"/>

<filter-ref name="x-powered-by-header"/>

</host>

</server>

<servlet-container name="default" default-encoding="utf-8">

<jsp-config/>

</servlet-container>

</handlers>

<response-header name="server-header" header-name="Server" header-value="WildFly/8"/>

<response-header name="x-powered-by-header" header-name="X-Powered-By" header-value="Undertow/1"/>

</filters>

</subsystem>

We maked the secondary-server that references the new socket binding.

In order to send a different ssl certificate for https-listener_secondary we assign it a new security realm: MySecurityRealm_secondary

We have to setup 2 security realm one for https-listener and other one for https-listener_secondary

Each realm references same keystore but different certificate alias

<security-realm name="MySecurityRealm">

<server-identities>

</ssl>

</server-identities>

</security-realm>

<security-realm name="MySecurityRealm_secondary">

<server-identities>

</ssl>

</server-identities>

</security-realm>

</security-realms>