Miscelaneus

Timing a command

 time ls 

Edit a file with cat

cat </tmp/test

Dasource deployment with Tomcat

Deploying a datasource shows the following error:

 org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of class '' for connect URL 'null', cause:
java.sql.SQLException: No suitable driver

Check if /conf/catalina/localhost/ is up to date with your current configuration.

Check if your application has the datasource definition on META-INF/context.xml

Check if both /conf/catalina/localhost/ and META-INF/context.xml match in the datasource definition.

Squid configuration samples

Http Authentication

debug_options ALL,9
http_port 80 accel defaultsite=wamp
cache_peer wamp parent 80 0 no-query originserver name=myAccel
external_acl_type ext_acl children=5 %PATH /etc/squid3/external_acl_test.sh admin123 paso123
acl script external ext_acl
acl aport port 80
http_access allow script aport
http_access deny all

For offline cache:

 visible_hostname www.demo.es


## DEF DE CACHES
cache_dir ufs /usr/local/squid/cache 200 16 256

## DEF DE LOGS
access_log /usr/local/squid/logs/access.log
cache_store_log /usr/local/squid/logs/store.log
cache_log /usr/local/squid/logs/cache.log
pid_filename /usr/local/squid/logs/squid.pid

## Directiva de configuracion
## se define el puerto de servicio 3380
## como acelerador y le decimos q se asocia a www.demo.es
http_port 3380 accel defaultsite=www.demo.es

## Directiva de configuracion
## el servidor que tiene los contenidos es localhost
## el cache es de tipo superior
## el puerto es el 80
## el puerto del icp no esta definido
## no se solicita ICP queries
## es un originserver
cache_peer localhost parent 80 0 no-query originserver name=myAccel

## Directiva de configuracion
##  definimos la acl "reverse_proxy" cuyo dstdomain es "www.demo.es"
acl reverse_proxy dstdomain www.demo.es


## Directiva de configuracion
## Directiva de configuracion
http_access allow reverse_proxy
offline_mode on


maximum_object_size 64 MB
refresh_pattern -i (/cgi-bin/|\?) 5 20% 20 ignore-no-store ignore-no-cache

Jailing environmente with chroot.

First: directory structure

 ./
.script.rc  bin  config  lib64  spool

./bin:
.lsd  bash  cat  init.rc  ls  sqlite3 echo 

./config:
current

./lib64:
ld-linux-x86-64.so.2  libc.so.6        libpthread.so.0   libselinux.so.1
libacl.so.1           libdl.so.2       libreadline.so.5  libsqlite3.so.0
libattr.so.1          libncurses.so.5  librt.so.1

./spool:
commands

Second. Creating a custom console.

 exec /usr/bin/rooted /var/cage /bin/init.rc

Third. Creating suid exec for chroot.

 cp /usr/bin/chroot /usr/bin/rooted
chmod u+s /usr/bin/rooted

Adendum. /bin/init.rc

 /bin/bash --init-file /.script.rc

Top shows incorrect amount of memory in use

Just in case that there's someone as clueless as i was... "top" command lies about your memory usage with "new" kernels (2.6.x series) .Let me explain how this game works. 

jupiter:/disponible/cargaMasivaDesa # top
Tasks: 108 total,   1 running, 105 sleeping,   2 stopped,   0 zombie
Cpu0  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  1.0%sy,  0.0%ni, 99.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   6127780k total,  6072248k used,    55532k free,   297820k buffers
Swap:  2096472k total,     3320k used,  2093152k free,  3969068k cached

After a reboot everything seems in place, I mean you can "top" and watch your memory report correctly but after some hours (in my case more or less a day) you'll see that almost your memory is gone. But if you calm your nerves you'll see that counting process and the memory used by them will report that your memory is okey... So???

It seems that the important info is not on free/used fields reported by this tool. The stuff that really matters is in /proc/meminfo.

 jupiter:/disponible/cargaMasivaDesa # cat /proc/meminfo
MemTotal:        6127780 kB
MemFree:           56568 kB
Buffers:          297864 kB
Cached:          3865668 kB
SwapCached:         1268 kB
Active:          2135488 kB
Inactive:        3655580 kB

Active: means memory in used

Inactive: means memory free but not available

Now i can sleep better!! . For more information you can have  a look to the following articles:

http://www.netadmintools.com/art416.html

http://www.tldp.org/LDP/tlk/mm/memory.html

Cheers

P.S: memory reporting command by process

 ps -eo pmem,pcpu,rss,vsize,args | sort -k 1 -r | head -10

Practical rc sripting in FreeBSD.

 Check out this link

Basic monitorization

Filesystem usage

#!/bin/bash
export data=`df -h |awk {'print $5 "@" $6'}|grep -v Use|sed 's/%//g'`
export result=""
for value in $data
do
export number=`echo $value|cut -d @ -f1`
export filesys=`echo $value|cut -d @ -f2`



if [ $number -gt 80 ]
then
result="${result} ${filesys}"
fi
done




export count=`echo $result|wc|awk {'print $2'}`
if [ $count -eq 0 ]
then
exit 0
else
echo $result
exit 1
fi

Cpu and memory usage per process

ps -eo pcpu,pid,size,vsize,user,cmd|sort -k 1 -r|head -10

RPC service with Bash

Xinetd script to execute shell scripts

 #!/bin/bash
function launch {
export params=$*
for pair in $params
do
export name="PARM_`echo $pair|cut -d = -f1`"
export val=`echo $pair|cut -d = -f2`
export $name=$val
done
. $PARM_cmd
}

read uri
export uri=`echo $uri|awk {'print $2'}|cut -d ? -f2`
export script=`echo $uri|cut -d ? -f1`
export param=`echo $uri|cut -d ? -f2`
export paramters=`echo $param|sed 's/&/ /g'`
launch $paramters

Using a datasource with Drupal

This php fragment code shows you how to use a Drupal datasource.

 


motd;
    //echo $row->id;
    ?>
    

Code fragment for creating a unix password

This fragment can help you out in case you need to manage the native passwords from C for more information type: man crypt.h

 #include 
#include 
#include 
#include 

int main (int argc,char *argv[]) {

        if(argc<3){
                printf("ERROR: params incorrect you must provide a password\n");
                return 1;
        }else{
        char *key=argv[1];
        printf ("%s|",key);
        char *salt=argv[2];
        //char *salt="$5$10$";
        printf ("%s\n",salt);
        char *password = crypt(key, salt);
        puts(password);
        return 0;}

}

creating users samba

This script helps you in the way of creating a new samba user.

#!/bin/bash
useradd -m -d /home/$USER -s /bin/bash $USER
echo "Account created!"
mknod /tmp/$USER p
echo "${PASSWORD}">/tmp/$USER &
echo "${PASSWORD}">/tmp/$USER &
cat /tmp/$USER|smbpasswd -a -s $USER

echo "Password assigned!"

rm /tmp/$USER
echo "Operation performed!!"

ftp scripting

#!/bin/bash function lista_ficheros(){ #export fecha=`date +%y%m%d` export lista="fichero1.* fichero2.*" for elemento in $lista do echo $elemento done } function descarga(){ export host=$1 export usuario=$2 export password=$3 export fichero=$4 echo "trying $fichero" ftp=/tmp/leap.tmp echo "cd .." >$ftp echo "ascii" >>$ftp echo "hash" >>$ftp echo "mget $fichero" >>$ftp echo "bye" >>$ftp ftp ftp://$usuario:$password@$host <$ftp rm $ftp } export host=$1 export entorno=$2 export usuario=$3 export password=$4 export args=$# if [ $args -eq 0 ] then echo " usage ./ftpscript.sh host env user pass" exit 2 fi export ficheros=`lista_ficheros $entorno` for fichero in $ficheros do export lzi=`descarga $host $usuario $password $fichero` echo $lzi done

HA linux

High Availability Linux/Unix

Creating a cluster. Pacemaker

csync2.

Csync2 is the configuration manager of the cluster.
The service mantains the same configuration for all the files involved in a cluster deployment.

Software installation.

  apt-get install openssh-server
  apt-get install lynx
  apt-get update
  apt-get install openssh-server
  apt-get install gcc
  apt-get install make
  apt-get install csync2
  apt-get install pacemaker
  apt-get install xinetd

Configuration.

file: etc/corosync/corosync.conf

# Please read the openais.conf.5 manual page

totem {
version: 2

# How long before declaring a token lost (ms)
token: 3000

# How many token retransmits before forming a new configuration
token_retransmits_before_loss_const: 10

# How long to wait for join messages in the membership protocol (ms)
join: 60

# How long to wait for consensus to be achieved before starting a new round of membership configuration (ms)
consensus: 5000

# Turn off the virtual synchrony filter
vsftype: none

# Number of messages that may be sent by one processor on receipt of the token
max_messages: 20

# Limit generated nodeids to 31-bits (positive signed integers)
clear_node_high_bit: yes

# Disable encryption
secauth: off

# How many threads to use for encryption/decryption
threads: 0

# Optionally assign a fixed node id (integer)
# nodeid: 1234

# This specifies the mode of redundant ring, which may be none, active, or passive.
rrp_mode: none

interface {
# The following values need to be set based on your environment
# change these parameters to fit your machine network configuration.
ringnumber: 0
bindnetaddr: 192.168.0.101
mcastaddr: 224.0.0.1
mcastport: 5605
}
}

amf {
mode: disabled
}

service {
# Load the Pacemaker Cluster Resource Manager
ver:       0
name:      pacemaker
}

aisexec {
user:   root
group:  root
}

logging {
fileline: off
to_stderr: yes
to_logfile: no
to_syslog: yes
syslog_facility: daemon
debug: on
timestamp: on
logger_subsys {
subsys: AMF
debug: off
tags: enter|leave|trace1|trace2|trace3|trace4|trace6
}
}

/etc/csync2.cfg

group ha
{
       host ubuntu ubuntu2;
       key /etc/csync2.key_mygroup;

   # Configuration files to keep synchronized.
       include /etc/csync2.cfg;

   # Executables (scripts) to keep synchronized.
   # include /usr/sbin/LinkCheck;

   # Directories to keep synchronized.
   #    include /shared_stuff/;
   #    include %homedir%/weibullguy;

   # Excluded items.
   #    exclude *~ .*;
   #    exclude /shared_stuff/;

   # What action to take when the csync2.cfg file is synced.
   # Execute the mailer script to send me an e-mail.
   # Log this action in the csync2 log file.
       action
       {
               pattern /etc/csync2.cfg;
               #exec "/etc/csync2/mailer";
               logfile "/var/log/csync2_action.log";
               do-local;
       }

   # Create backups.
#       backup-directory /var/backups/csync2;
#       backup-generations 3;

       auto none;
}

# The homedir variable on frodo and legolas (hosts 2 and 3)
# is different than it is on all other hosts (only aragorn in this case).
#prefix homedir
#{
#       on host[23]: /home/users;
#       on *:        /home;
#}

HealthCheck

/etc/services
servermon       9999/tcp                        # fidonet EMSI over TCP



/etc/xinet.d/servermon
service servermon
{
       socket_type    = stream
       wait           = no
       user           = root
       server         = /usr/sbin/checkstatus
       disable        = no
}

/usr/sbin/checkstatus
#!/bin/bash
export tss=`date +%Y%m%dT%H%M%S`
export host_conn="192.168.0.101"
export name=`cat /etc/hostname`
echo "server_name=$name"
echo "timestamp=$tss"
ping -c 1 -w 1 $host_conn>/dev/null
export network_stat=$?
if [ $network_stat == 0 ]
then echo "network_stat=up"
else
echo "network_stat=down"

fi



/etc/ha/nodes.conf
nodes=ubuntu2 ubuntu3



/usr/sbin/nodes_stat
#!/bin/bash
function getParam(){
export param=$1
export file=$2
cat $file|grep -i $param|cut -d = -f2
}
export host_list=`getParam nodes /etc/ha/nodes.conf`
for host_n in $host_list
do
nc ${host_n} 9999 > /dev/null
export exit_stat=$?
if [ $exit_stat == 0 ]
then
echo "${host_n}=up"
else
echo "${host_n}=down"
exit 2
fi
done
exit 0

Monitoring Watchdog

 

/etc/ha/service.conf
[monitor definition]
wdgd_log_dir=/var/log/
process_monitor=ssh
crash_notification=echo
monitor_services_ssh=/etc/init.d/ssh
monitor_services_option_ssh= restart
monitor_script_ssh=/usr/sbin/port_mon localhost 22
monitor_postaction_ssh=sleep 5

/usr/sbin/show_wdgd
#!/bin/bash
export proc=$1
while true; do clear; cat /var/log/wdgd_${proc}; sleep 1; done


/usr/sbin/port_mon
#!/bin/bash
nc $1 $2 -w 1
if [ "$?" == "0" ]
then
exit 0
else
exit 1
fi

/usr/sbin/wdgd_srv
#!/bin/bash
export lock_file="/tmp/lock_wdgd.lock"
while true
do
if [ -f $lock_file ]
then
watchdog
else
exit 1
fi
done

/etc/init.d/wdgd

#!/bin/bash

function getParam(){
export param=$1
export file=$2
cat $file|grep -i $param|cut -d = -f2
}

export logDir=`getParam wdgd_log_dir /etc/ha/service.conf`
export tdate=`date +%Y%m%dT%H%M%S`
# debug
# export logFile="${logDir}/wdgd_${tdate}.log"


export wdg_home=/usr/sbin/
case $1
in
start)
export logFile=/dev/null
touch /tmp/lock_wdgd.lock
${wdg_home}wdgd_srv 2>${logFile} 1>>${logFile} &
exit 0;;

debug)
export logFile="${logDir}/wdgd_${tdate}.log"
touch /tmp/lock_wdgd.lock
${wdg_home}wdgd 2>${logFile} 1>>${logFile} &
exit 0;;

stop)
rm /tmp/lock_wdgd.lock
exit 0;;

status)
if [ -f "/tmp/lock_wdgd.lock" ]
then
echo "running"

else
echo "stopped"
fi
exit 0;;

*)
echo "please type one of the following options: start, debug, stop, status "
exit 2;;

esac

/usr/sbin/watchdog
#!/bin/bash

function getParam(){
export param=$1
export file=$2
cat $file|grep -i $param|cut -d = -f2
}
###
function testMonScript()
{
export nombre=$1
export script_mon=`getParam monitor_script_${nombre} /etc/ha/service.conf`
export script_cut=${script_mon}|awk {'print $1'}
if [ -x $script_cut ]
then
echo "$script_mon"
else
echo "NOT_FOUND"

fi
}
###
function logService()
{
export service_name=$1
export base_log=`getParam wdgd_log_dir /etc/ha/service.conf`
export base_dir_log=$base_log
export message=$2
export file_output="${base_dir_log}/wdgd_${service_name}"
export tdate=`date`
echo "${tdate}:> ${message}">$file_output
}
export srvlist=`getParam process_monitor /etc/ha/service.conf`
echo "================================================================"
## for every service in process_monitor we are going to launch the monitor script

for service in $srvlist
do

## first we test if the service to monitor exists, else there is nothing to mon

export process=`getParam monitor_services_${service} /etc/ha/service.conf`
export pars_p=`getParam monitor_services_option_${service} /etc/ha/service.conf`

echo "(*) launching monitor for:  ${service}"
if [ -x $process ]
then
echo "file exist $process"

export test_monitor_scr=`testMonScript $service`
echo "monitor script  $test_monitor_scr"

## if the monitor is not found exit the process and show the error

if [ "$test_monitor_scr" == "NOT_FOUND" ]
then
echo "monitor_services_$service value in service.conf is not executable or does not exist"
exit 5
else

## the monitor exists and is an executable file

echo "ready to lauch monitor: $test_monitor_scr"

$test_monitor_scr
export status=$?
if [ $status == "0" ]
then

## the monitor returned 0 so it seems the monitor to process is ok

echo "monitor for $service ok"
echo "----------------------->"
logService $service  "${service}=ok"
else

## the monitor returned something else than 0 so it seems the monitor to process is down

logService $service $base_dir_log "${service}=down"
echo "*******************************************************************"

echo "WARNING !!!! service $service is dead "
echo "launching $process with the following params: $pars_p"
export rescue_cmd="$process $pars_p"
$rescue_cmd
logService $service "${service}=starting"
export stat_cm=$?
echo "Command status: $stat_cm"
## we must check the exit status for the rescue command
export command_post_start=`getParam monitor_postaction_${service} /etc/ha/service.conf`
echo "launching post-startup command $command_post_start"

$command_post_start
if [ $stat_cm == "0" ]
then
## we check that really the server startup was sucessfull
$test_monitor_scr
export status_r=$?
if [ $status_r == "0" ]
then
## it was !!! :)
logService $service "${service}=started"
echo "success"
else
## it was not !!! :(
export ldate=`date`
export message="CRITICAL: service ${service} was not successfully started after automatic restart. time ( $ldate )"
echo $message
logService $service "${service}=ko"

export crash_cmd=`getParam crash_notification /etc/ha/service.conf`
$crash_cmd $message
fi
else
## ups!!!
export ldate=`date`
export message="CRITICAL: service ${service} was not successfully started after crash. time ( $ldate )"
echo $message
export crash_cmd=`getParam crash_notification /etc/ha/service.conf`
$crash_cmd $message
export command_post_start=`getParam monitor_postaction_${service} /etc/ha/service.conf`
echo "launching post-startup command on fault $command_post_start"
$command_post_start
fi
echo "*******************************************************************"
fi
fi
else
echo "file does not exist"
fi
done

eDirectory Command Wrapper

 #!/bin/bash


#############Vars
export DEBUG="debug"
export INFO="info"
export DEFAULT_LOG_LEVEL="debug"
export DIR_NDS=`cat /etc/opt/novell/eDirectory/conf/.edir/instances.0`
export cmd_stop="/opt/novell/eDirectory/bin/ndsmanage stop --config-file ${DIR_NDS}"
export cmd_start="/opt/novell/eDirectory/bin/ndsmanage start --config-file ${DIR_NDS}"

#export cmd_stop="/root/test.sh stop ${DIR_NDS}"
#export cmd_start="/root/test.sh start ${DIR_NDS}"

#### exit 0

############Funciones

function countProcesses {
export proc=`ps -ef |grep -i ndsd|grep -v "grep" |wc|awk {'print $1'}`
echo $proc
}


function log {
export tlevel=$1
export mesg=$2
export level=$DEFAULT_LOG_LEVEL
export fdat=`date +%s`
if [ $tlevel == $level ];
then
echo "${fdat} [${tlevel}]: ${mesg} "
elif [ $tlevel == "info" ];
then
echo "${fdat} [info]: ${mesg} "

fi
}

function mailroot {
echo $1|mail root
}
#####################Logica general

case $1 in
start)
$cmd_start
export stat_code=$?
if [ $stat_code -eq 0 ];
then
log $INFO "ejecucion script arranque correcta"
else
log $INFO "ejecucion script arranque incorrecta"
mailroot "El script ndsmanage ha mostrado un error."
exit 1
fi
log $INFO "Esperando 5 segundos para comprobar los procesos de eDirectory"
sleep 5
export num=`countProcesses`
log $DEBUG "contando procesos: $num"
if [ $num -eq 1 ];
then
log $INFO "el servicio ha arrancado correctamente"

exit 0
else
mailroot "El servicio no se ha arrancado correctamente."
exit 1
fi
## FIN CASE START
;;
stop)
trap 'echo "salida anormal";exit 126' 2
log $INFO "se va a parar el servicio del directorio"
$cmd_stop
export stat_code=$?
if [ $stat_code -eq 0 ];
then
log $INFO "ejecucion script parada correcta"
else
log $INFO "ejecucion script parada incorrecta"

mailroot "El script mdsmanage de eDirectory ha mostrado un error mientras se lanzaba la parada del directorio. Se va a abortar la copia"
exit 1
fi
log $INFO "Esperando 60 segundos para dar tiempo a la parada ordenada de eDirectory"
sleep 10
log $INFO "Lanzando comprobacion"
export num=`countProcesses`
log $DEBUG "contando procesos: $num"
if [ $num -eq 1 ];
then
log $INFO "el servicio ndsd no se ha parado todavia"

for try in `seq 5`
do
sleep 5
log $INFO "comprobacion numero: $try "

export inum=`countProcesses`
if [ $inum -eq 0 ];
then
echo "el directorio ha parado correctamente"
exit 0
else
echo "el directorio todavia no ha parado"
fi
done
mailroot "El directorio no se ha parado. Abortamos copia de seguridad"
exit 1
else
## salida ok!!!
log $INFO "el servicio de directorio se ha parado. Devolvemos codigo de status correcto"
exit 0
fi
### FIN CASE STOP
;;
rep_t)
ndsrepair -T
;;
rep_s)
ndsrepair -E
;;
status)

export inum=`countProcesses`
if [ $inum -eq 0 ];
then
echo "el directorio esta parado"
exit 3
else
echo "el servicio de directorio esta arrancado"

exit 0
fi
### FIN STATUS
;;
*)
echo "se debe pasar un argumento de tipo start/stop/rep_t/rep_s"
;;
esac