Linux 服务器配置记录

NULL.IDA 扫描者IP地址的统计脚本

现象描述 (查看我在 chinaunix.net 发表的帖子 )

在 apache 的 log 文件中经常会有如下的记录行(以下是缩短后的例子)

202.98.151.13 - - [28/Jan/2004:18:44:34 +0800] "GET /NULL.IDA?CCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC CCCCC%u0
aeb%ub890%uf533% u77e6%u0000%u0000%u838b%u0094%u0000%u408b%
u0564%u0150%u0000%ue0ff%u9090=x&\x90\x90\x90\x90\x90x90\x90\x90\x90
\x90\x90\x90\x90\x90 \x90\x90\x90\x90\x90\x90\x90\x 90\x90\a\x99\x99\x10\x1
c\xde\x9 c\x99\x99q'\x98\x99\x99\x10\x1c\xd6\x9c\x99\x99\x12\x1c\xde\x9c\x
99\x99q\xe6\x9b\x99\x99\x10\x1c\d2\x9c\x99\x99q\xc7\x99\x99\x99q\b\x99\x99
\x99\x1aa\x99\xedy\x12\x1c\xd2\xf7\xf8\xf4\xfc\x99\xea\fc\xed\xea\xf6\xfa\xf2\
xf6\xe9\xed\x99\x99\x99\xd5\xf6\xf8\xfd\xd5\xf0\xfb\xeb\xf8\xeb\xe0\xd8\x99\x
de\xfc\xed\xc9\xeb\xf6\xfa\xd8\xfd\xfd\xeb\xfc\xea\xea\x99\xea\xeb\x7f\xee\xa
8\xe9\x7f\xee\x99\ xfa\xa 4\x01b\xddcmd.exe$ HTTP/1.1" 404 4917 "-" "-"

这些记录会使得一些 log 统计分析软件在统计的时候出现错误警告,同时我们也不希望经常被这些扫描者所骚扰,于是就试着写了一个脚本,把 apache 的 log 中的这些 IP 地址读取出来,以便以后在做 Iptables 的时候可以使用。 (这里参照了车东的脚本并感谢 Xiyang 的支持。)

脚本 下载

#!/bin/sh

### null.ida_stat.sh, Ver. 0.2 2004/05/18 13:41:32 Kreny

LOG_FILE1='/usr/local/apache2/logs/www.domain1.com-access_log'
LOG_FILE2='/usr/local/apache2/logs/www.domain2.com-access_log'

# You can add more log files here.
#LOG_FILE3='/usr/local/apache2/logs/www.domain3.com-access_log'
#LOG_FILE4='/usr/local/apache2/logs/www.domain4.com-access_log'
#LOG_FILE5='/usr/local/apache2/logs/www.domain5.com-access_log'

IDE_FILE='/home/user/public_html/log/ida.txt'
DATE=`date`

######## Rank the IP addresses of NULL.IDA scanner ########
# Add your LOG_FILEs after the $LOG_FILE2

less $LOG_FILE1 $LOG_FILE2 | grep NULL.IDA | awk '{print $1}'| sort | \
uniq -c | sort -rn > $IDE_FILE

echo "" >> $IDE_FILE
echo "Last update:".$DATE >> $IDE_FILE

仅列出IP地址: (感谢 随风漂 指导)

#!/bin/sh

####### null.ida_stat.sh, Ver. 0.2 2004/05/18 13:41:32 Kreny #######
# Only List the Scanner's IPs #

LOG_FILE1='/usr/local/apache2/logs/www.domain1.com-access_log'
LOG_FILE2='/usr/local/apache2/logs/www.domain2.com-access_log'
# You can add more log files here.
#LOG_FILE3='/usr/local/apache2/logs/www.domain3.com-access_log'
#LOG_FILE4='/usr/local/apache2/logs/www.domain4.com-access_log'
#LOG_FILE5='/usr/local/apache2/logs/www.domain5.com-access_log'

IDE_FILE='/home/user/public_html/log/ida_ip.txt'

awk '/NULL.IDA/{print $1}' $LOG_FILE1 $LOG_FILE2 | sort -u > $IDE_FILE

运行结果

查看NULL.IDA扫描者IP统计结果

相关资料

扫描目的:"IIS Index Server ISAPI扩展远程溢出"漏洞 ( /NULL.IDA )

另外就是题外话了,虽然此类扫描对Linux主机的影响不是很大,但是类似的扫描时不可避免的,关键还是看怎么去发现和预防它。(某某人语) ;-)

 

应用:自动统计病毒邮件发信者IP并添加相应规则至iptables ( 查看我在 chinaunix.net 发表的帖子 )

awk 的强大功能可以运用在更多的地方,例如通过检查在qmailscaner的病毒隔离目录中的邮件,得出发邮件主机的IP地址,以便在Iptables 中进行一些设置。

统计并排序病毒邮件SMTP IP 地址

$ less /var/spool/qmailscan/quarantine/new/*| grep 'HELO' | awk '{print $6}' | cut -d \( -f2 | cut -d \) -f1 | sort | uniq -c | sort -rn > /home/kreny/public_html/log/viruse_smtp.txt
$ echo 'Last Update:' >> /home/kreny/public_html/log/viruse_smtp.txt
$ date >> /home/kreny/public_html/log/viruse_smtp.txt

查看病毒邮件SMTP排序统计结果


[总结] 获取NULL.IDA扫描者和病毒邮件发送者的IP地址,并添加相应规则到iptables。

为了不重复,要将三个得到的文件: NULL.IDA 扫描者 IP List 病毒邮件发送者 IP List 和现有的 Iptables Rules 中的REJECT IP List 地址整理一下,然后再一次加到iptables中去。在这里文件都以 _ip.txt 结尾, 以便使用通配符。
注:你也可以将这些 IPs 添加到 tcpserver 的 rules 中去,详细请看The tcprules program。   [下载 auto_ida_viruse.sh 脚本]

#!/bin/sh
# auto_ida_viruse.sh,v 0.3 2004/05/22 18:19:18 Kreny

LOG_FILE1='/usr/local/apache2/logs/www.domain1.com-access_log'
LOG_FILE2='/usr/local/apache2/logs/www.domain2.com-access_log'
CHAIN_NAME='RH-Lokkit-0-50-INPUT'
DATE=`date`
SPAM_MAIL_DIR='/home/user/sa-learn/spam'
FILE_PATH='/home/user/public_html/log'

IDE_FILE='ida_ip.txt'
VIRUSE_FILE='viruse_ip.txt'
REJECT_FILE='reject_ip.txt'
SPAM_FILE='smap_ip.txt'

### List the NULL.ida Scanner's IPs
printf "Listing the NULL.ida Scanner's IPs. Please wait...\n"
awk '/NULL.IDA/ {print $1}' $LOG_FILE1 $LOG_FILE2 | sort -u > $FILE_PATH/$IDE_FILE

### List the viruse SMTP IPs
printf "Listing the viruse SMTP IPs. Please wait...\n"
awk '/HELO/ {print gensub(/\((.*)\)/,"\\1", 1, $6)}' /var/spool/qmailscan/quarantine/new/* | sort -u > $FILE_PATH/$VIRUSE_FILE

### List the Spam-mail SMTP IPs
printf "Listing the Spam-mail SMTP IPs. Please wait...\n"
awk '/HELO/&&$6~/\[/ {print gensub(/\(\[(.*)\]\)/,"\\1", 1, $6)}' $SPAM_MAIL_DIR/* | sort -u > $FILE_PATH/$SPAM_FILE
awk '/HELO/&&$6!~/\[/ {print gensub(/\((.*)\)/,"\\1", 1, $6)}' $SPAM_MAIL_DIR/* | sort -u >> $FILE_PATH/$SPAM_FILE

### If you copy the file from windows, you have to do a dos2unix
#/usr/bin/dos2unix -k -n $FILE_PATH/$SPAM_FILE $FILE_PATH/spam.tmp
#mv -f $FILE_PATH/spam.tmp $FILE_PATH/$SPAM_FILE
#/bin/chmod 0644 $FILE_PATH/$SPAM_FILE

### List the REJECT IPs from iptables
printf "Listing the REJECT IPs. Please wait...\n"
awk '/REJECT/&&/-s/{print $5}' /etc/sysconfig/iptables |sort -u > $FILE_PATH/$REJECT_FILE

### Add the rules to iptables # NOTE: you HAVE to add the /sbin/ before iptables.
printf "Updating the iptables rules. Please wait...\n"
/sbin/iptables --flush
### Add the default or custom rules ###
/sbin/iptables -A INPUT -j RH-Lokkit-0-50-INPUT
/sbin/iptables -A FORWARD -j RH-Lokkit-0-50-INPUT
/sbin/iptables -A RH-Lokkit-0-50-INPUT -i lo -j ACCEPT
/sbin/iptables -A RH-Lokkit-0-50-INPUT -i eth0 -j ACCEPT
/sbin/iptables -A RH-Lokkit-0-50-INPUT -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j REJECT
/sbin/iptables -A RH-Lokkit-0-50-INPUT -p udp -m udp -j REJECT
/sbin/iptables -A RH-Lokkit-0-50-INPUT -s "220.98.121.7" -j ACCEPT

### Add the new ip list ###
sort -u $FILE_PATH\/*_ip.txt | awk '{system("/sbin/iptables -A '$CHAIN_NAME' -s "$0" -j REJECT ")}'
/sbin/service iptables save
awk '/REJECT/&&/-s/{print $5}' /etc/sysconfig/iptables |sort -u > $FILE_PATH/$REJECT_FILE
printf "Done! \t Completed on `date`\n"

对于从邮件中列出IP地址的方法,也可以用一下语句:(感谢 Mercury_cn 网中人 daniey bjgirl 指导,查看原帖

#awk '/HELO/ {print $6}' $SPAM_MAIL_DIR/* | sed 's/[^0-9.]//g' | sort -u >> $FILE_PATH/$SPAM_FILE

#awk '/HELO/ {print $6}' $SPAM_MAIL_DIR/* | tr -s '([])' ':' | awk -F: '{print $2}' | sort -u >> $FILE_PATH/$SPAM_FILE

#awk '/HELO/ {gsub(/\(|\)|\[|\]/,""); print $6}' $SPAM_MAIL_DIR/* | sort -u >> $FILE_PATH/$SPAM_FILE

当然,如果你想一了百了的话,可以将 log 中的含有 NULL.IDA 记录删除。(不是很建议此脚本,因为有风险——没有加上和用户交互信息功能——而且意义不大。)

#!/bin/sh
# null.ida_del.sh v 0.1 Kreny 05/19/2004 6:53

LOG_FILE='/usr/local/apache2/logs/www.domain.com-access_log'
printf "Deleting the NULL.IDA lines from log file $LOG_FILE ...\n"
apachectl stop
mv $LOG_FILE $LOG_FILE.1
sed -e '/NULL.IDA/d' $LOG_FILE.1 > $LOG_FILE
apachectl start
printf "Done!\n"


延伸: 统计邮件使用者的邮件使用率

由于使用了qmail-scanner,所以找不到什么好的统计软件来统计正常使用的邮件数量(尽管病毒邮件数量统计可以用 QSS 2.0.2 进行统计 ,查看我的病毒邮件统计数) 。下面试着从qmail-scanner-queue.pl 产生的 log 文件 /var/spool/qmailscan/qmail-queue.log 中读出相应的邮件地址以便进行统计。

Log Sample:

Tue, 18 May 2004 21:21:56 CST:28780: g_e_h: return-path is "sender@domain1.com", recips is "receiver@domain2.com"

脚本  下载最新脚本 mailuser_stat.sh v 0.3 Edited at 06/11/2004 0:16

#!/bin/sh
# mailuser_stat.sh v 0.2 Kreny 05/19/2004 17:39

DOMAIN='@yourdomain'
SENDER_FILE='/home/user/public_html/log/log/mail_sender.txt'
RECEIVER_FILE='/home/user/public_html/log/log/mail_receiver.txt'
VIRUSE_FILE='/home/user/public_html/log/log/mail_viruse.txt'
LOG_FILE='/var/spool/qmailscan/qmail-queue.log'
QUARANTINE_FILE='/var/spool/qmailscan/quarantine.log'
TODAY=`date`

printf "Statistics for mail sender\n"
awk 'NR==1 {print "Statistic since "$1,$2,$3,$4,$5}' $LOG_FILE > $SENDER_FILE
echo "" >> $SENDER_FILE
awk '/return-path/&&$10~/'$DOMAIN'/ {print gensub(/\"(.*)@.*\",/,"\\1",1,$10)}' $LOG_FILE | sort | uniq -c | sort -nr >> $SENDER_FILE
printf "\n Last Update: $TODAY" >> $SENDER_FILE
printf "Done! \n"

printf "Statistics for mail receiver\n"
awk 'NR==1 {print "Statistic since "$1,$2,$3,$4,$5}' $LOG_FILE > $RECEIVER_FILE
echo "" >> $RECEIVER_FILE
awk '/return-path/&&$13~/'$DOMAIN'/ {print gensub(/\"(.*)'$DOMAIN'\"/,"\\1",1,$13)}' $LOG_FILE | sort | uniq -c | sort -nr >> $RECEIVER_FILE
printf "\n Last Update: $TODAY" >> $RECEIVER_FILE
printf "Done! \n"

# Counting the viruse mail receiver
printf "Statistics for viruse mail receiver\n"
awk 'NR==1 {print "Statistic since "$1,$2,$3,$4,$5,$6}' $QUARANTINE_FILE > $VIRUSE_FILE
echo "" >> $VIRUSE_FILE
awk '{print gensub(/(.*)'$DOMAIN'/,"\\1",1,$8)}' $QUARANTINE_FILE | sort | uniq -c | sort -nr >> $VIRUSE_FILE
printf "\n Last Update: $TODAY" >> $VIRUSE_FILE
printf "Done! \n"

如果你想使用 qmail-scanner 的qmail-queue.log来统计各用户每天的邮件发送件数的话,可以添加以下脚本(awk命令行只有一行)。变量$DOMAIN是你自己的服务器名。感谢 admirer 指导。查看源帖

## Send to Non-dalouis user.(Daliy Statistics)

printf "\nSend to Non-$DOMAIN user.(Daliy Statistics)\n" >> $FILE_PATH/$SENDER_FILE
awk '/return-path/&&$10~/'$DOMAIN'/&&$13!~/'$DOMAIN'/ {print ":"$4,$3,$2,$1,":"gensub(/\"(.*)@.*\",/,"\\1",1,$10)}' /var/spool/qmailscan/qmail-queue.log | sort | uniq -c | awk -F: '{print $2":"$1":"$3}' | sort +0r -1 +1rM -2 +2rn -3 +5rn | awk -F: '{array_a[$1 $3]+=$2}END{for(i in array_a){split(i,array_b,"");printf"%s,\t%d\n",i,array_a[i]}}' |sort +0rn -1 +1rM -2 +2rn -3 +5rn |awk -F, '{if($1==date){printf("\t%d\t%s\n",$3,$2)}else{printf("%s\n\t%d\t%s\n",$1,$3,$2);date=$1;array_b=$0}}'>> $FILE_PATH/$SENDER_FILE

# (Monthly Statistics)Send to Non-dalouis user.
printf "\nSend to Non-dalouis user.(Monthly Statistics)\n\n" >> $FILE_PATH/$SENDER_FILE
awk '/return-path/&&$10~/'$DOMAIN'/&&$13!~/'$DOMAIN'/ {print ":"$4,$3,":"gensub(/\"(.*)@.*\",/,"\\1",1,$10)}' /var/spool/qmailscan/qmail-queue.log | sort | uniq -c | awk -F: '{print $2":"$1":"$3}' | sort +0r -1 +1rM -2 +3rn | awk -F: '{array_a[$1", "$3]+=$2}END{for(i in array_a){split(i,array_b,"");printf"%s\t,\t%d\n",i,array_a[i]}}' |sort +0rn -1 +1rM -2 +5rn |awk -F, '{if($1==month){printf("\t%d\t%s\n",$3,$2)}else{printf("%s\n\t%d\t%s\n",$1,$3,$2);month=$1;array_b=$0}}' >> $FILE_PATH/$SENDER_FILE
printf "Done! \n"

其中是把2004 May 21 Fri, 1 sender 用冒号:分为了三组 $1 $2 $3,2004 May 21 Fri, : 1 : sender然后进行split , 然后再利用星期里面自带的逗号,用 printf"%s,%d\n"    添加了一个逗号使之成为2004 May 21 Fri, 1 , sender 最后打印输出。

如果你使用的是vpopmail, 那么这里我写了一个小脚本能够方便地列出所有虚拟域名(即 /home/vpopmail/domains/)下面的用户名(包含转信邮箱)。

# ::::::::::::::
# vpopmail-alias2recipients
# ::::::::::::::
#!/bin/sh
QMAIL=/var/qmail
VPOPMAIL=/home/vpopmail/domains
for i in `ls -l $VPOPMAIL | grep ^d | awk '{print $9}'`
do
cd $VPOPMAIL/$i
ls -l .qmail-*| grep -v .qmail-default | tr -s " " | awk '{print $9}' | awk -F- '{print $2"@localhost"}' | sed -e 's/localhost/'$i'/' | sort -u >> $QMAIL/users/recipients
done
# ::::::::::::::
# vpopmail-users2recipients
# ::::::::::::::
#!/bin/sh
LANG=C
QMAIL=/var/qmail
VPOPMAIL=/home/vpopmail/domains
for i in `ls -l $VPOPMAIL | grep ^d | awk '{print $9}'`
do
cd $VPOPMAIL/$i
ls -l | grep ^d | awk '{print $9"@localhost"}' | sed -e 's/localhost/'$i'/' | sort -u >> $QMAIL/users/recipients
done

脚本下载 vpopmail-alias2recipients vpopmail-users2recipients

运行结果:

    mail_sender.txt |  mail_receiver.txt  |  mail_viruse.txtviruse_smtp.txt | spam_ip.txt | reject_ip.txt | reject_mail.txt

参考文档:

★ C-shell Cookbook Split a string into an array      ★ IMB文库 Shell 编程系列     ★  Sort 命令集  by bigirl    
ChinaUnix.net Shell 精华集锦   ★ Murray Hill, New Jersey Unix Shell 介绍

Valid HTML 4.01!  Built on Saturday, 05/15/2004 14:51


Creative Commons License
本网址建立在 创作公用 协议下。版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及以上“创作共用”声明。

本文链接:http://www.kreny.com/computer/linux/nullida.html
作者: Kreny 给我留言 Email: Send E-mail.
您来自: 本文最后更新:January 20 2008 21:26:01.