Hotfix 160301 - FINAL

[hans2]

Hi all

i've been working on another hotfix that may interest you.

WARNING #1: THIS UPDATE IS NOT YET READY FOR ALL NOVICE USERS! USE AT OWN RISK (for factory reset)
WARNING #2: THESE UPDATES ASUME 1.51SP1 IS INSTALLED ON YOUR SHIELD

all transactions to install and run scripts need to be done from the CLI root.

Download the hotfix: hotfix_160309-FINAL.tgz
Download the change log: hotfix_160309-FINAL.txt
MD5SUM for this hotfix: 85a06650bfe47bf4d0c0bf641c0c35b7

12) 	= DAILY UPDATE SCRIPT FOR IPS AND WF - version 6
	> /sbin/fw_upgrade /etc/init.d/dnsmasq /etc/itus/update_blacklist.sh /etc/itus/write-categories.sh
	- added the ramdisk functionality so that temporary files are kept in memory only.
	- ref: http://itus.accessinnov.com/Update-script-fw-upgrade-td43.html


13) 	= UPDATE TO SP1 
	> /tmp/upgrade_rc_to_sp1.sh
	- updated script to use dropbox as source of updates
	- ref: http://itus.accessinnov.com/Upgrade-to-1-51SP1-td10.html

14) 	= LUCI - LAST UPDATE DISPLAY
	> /.hf_date /usr/lib/lua/luci/view/admin_status/index.htm
	- added hotfix date visiblity to LuCI
	- check Status > Overview > Firmware Version line

15) 	= LUCI - DIAGNOSTICS
	> /usr/lib/lua/luci/view/admin_network/diagnostics.htm
	- change the default diagnostics URL from itusnetworks.com to www.msftncsi.com
	- check Network > Diagnostics

16) 	= CLI - CLEANING OF OBSOLETE FILES
	> /tmp/cleanup.sh /tmp/cleanup_list CHANGED
	- archives files listed in cleanup_list into cleanup_archive.tgz
	- deletes files if the archive is created correctly
	- restarts snort to download new rules
	- run with "sh /tmp/cleanup.sh" 

17)	= BOOT - NTP AND DROPBEAR
	>  /etc/rc.local
	- force a dropbear restart 30 seconds after last bood command
	- restart NTP client after dropbear

18) 	= INIT - NTP CRON
	> /etc/init.d/ntpclient
	- set the cron job to run at midnight instead of every 10 minutes.
	- check system > scheduled tasks

19) 	= OPKG - ARCH
	> /etc/opkg.conf
	- adds the architectures for cn70xx and octeon to the package list.

20) 	= IPS - LOG PROBLEM
	> /etc/snort/snort.conf
	- disabled preproc_rules for preprocessor, decoder and sensitive date
	- ref http://itus.accessinnov.com/Speed-issue-due-to-log-size-too-big-SOLUTION-td189.html

21) 	= LUCI WF - CONTENT FILTERING OPTIONS
	> /usr/lib/lua/luci/model/cbi/e2guardian.lua
	- removed all but Ads, Malicious and Drugs from option list 
	- this is related due to limitations of fw_upgrade script

22) 	= LUCI - UTM MODE DISPLAY
	> /usr/lib/lua/luci/view/admin_status/index.htm /etc/rc.local /.shield_mode /etc/itus/detect_mode.sh
	- runs at startup detect-mode script. This determines router/bridge/gateway mode

[user8446]

Thanks! I'll start doing some testing on my end when I can.

Just thinking, for small hotfix updates could we do that via the backup config? Someone could just use the restore backup and never leave the GUI.

[Me_3594]

CONTENTS DELETED

The author has deleted this message.

[user8446]

Great work as usual!

12 - fw_upgrade - experienced no problems
13 - already on sp1
14 - Shows the "hotfix" in the Firmware Version, should probably be hansfix :)
15 - defaults to www.msftncsi.com now
16 - I already manually deleted as I don't use the webfilter

Suggestions:

In /etc/rc.local I have added:

sleep 30
/etc/init.d/dropbear restart

sleep 10
/usr/sbin/ntpclient -s -p 123 -h 0.us.pool.ntp.org || /etc/init.d/ntpclient restart

SSH lost in reboot and to pull the current time at startup. Then do monthly updates:

/etc/init.d/ntpclient

...snip...
cron_seed() {
        local cronstuff='40 3 2 * *'
        local reset="/etc/init.d/ntpclient restart"


Do you think you should do a cron seed for the IPS log clear until it's figured out why snort isn't using the log size limit?

[hans2]

user8446 wrote

12 - fw_upgrade - experienced no problems
13 - already on sp1
14 - Shows the "hotfix" in the Firmware Version, should probably be hansfix :)
15 - defaults to www.msftncsi.com now
16 - I already manually deleted as I don't use the webfilter

thanks for testing

user8446 wrote

In /etc/rc.local I have added:

sleep 30
/etc/init.d/dropbear restart

sleep 10
/usr/sbin/ntpclient -s -p 123 -h 0.us.pool.ntp.org || /etc/init.d/ntpclient restart

SSH lost in reboot and to pull the current time at startup. Then do monthly updates:

/etc/init.d/ntpclient

...snip...
cron_seed() {
        local cronstuff='40 3 2 * *'
        local reset="/etc/init.d/ntpclient restart"

excellent ideas - i've added them to the hotfix!

user8446 wrote

Do you think you should do a cron seed for the IPS log clear until it's figured out why snort isn't using the log size limit?

we could do that but now it also depends on the Shield mode (bridge/router)

[user8446]

Better yet instead of the logclear the snort config logsize limit fix: http://itus.accessinnov.com/Speed-issue-due-to-log-size-too-big-SOLUTION-td189.html

[hans2]

user8446 wrote

Better yet instead of the logclear the snort config logsize limit fix: http://itus.accessinnov.com/Speed-issue-due-to-log-size-too-big-SOLUTION-td189.html

Agreed - this is now part of BETA4

Changes made vs BETA3:

20) 	= IPS - LOG PROBLEM
	> /etc/snort/snort.conf
	- disabled preproc_rules for preprocessor, decoder and sensitive date
	- ref http://itus.accessinnov.com/Speed-issue-due-to-log-size-too-big-SOLUTION-td189.html

21) 	= LUCI WF - CONTENT FILTERING OPTIONS
	> /usr/lib/lua/luci/model/cbi/e2guardian.lua
	- removed all but Ads and Malicious from option list - due to limitations of fw_upgrade script

21) will then show as



This is temporary until we figure out how to change fw_upgrade to include other areas.

@ALL - this is the latest version of the March release - please check and i will move it to the general area.

[Roadrunnere42]

Hans
What about putting in a hotfix for ipvar Home-net any as it should really be you local ipaddress/24  for use who are in router mode, as mentioned in the forum.


roadrunnere42
 

[Roadrunnere42]

Tried to install hotfix but keep getting these error

root@Shield:/# tar -zxvf hotfix_160301-BETA4.tgz

tar: invalid tar magic

roadrunnere42

[Gnomad]

Hi Hans, Roadrunnere42 -
I'm switched to Router mode, and find that after installing hotfix_160301-BETA4.tgz the Luci Status Overview page shows Firmware Version of "v1.51 SP1 + Hotfix Mar 4" as expected, but Operating Mode now shows "UTM Bridge".  Just an UI bug?  Or is there an issue with this hotfix forcing Bridge mode?  Things seem to be operating okay as far as I can tell.

Spotted your comment regarding snort config's ipvar Home-net Roadrunnere42, any other tips/gotchas to be aware of in Router mode?
Thanks!

[hans2]

Gnomad wrote

I'm switched to Router mode, and find that after installing hotfix_160301-BETA4.tgz the Luci Status Overview page shows Firmware Version of "v1.51 SP1 + Hotfix Mar 4" as expected, but Operating Mode now shows "UTM Bridge".  Just an UI bug?  Or is there an issue with this hotfix forcing Bridge mode?  Things seem to be operating okay as far as I can tell.

Correct, "UTM Bridge" is hard coded into the htm file - I am looking for a way to detect the shield mode via script. WIP.

[Roadrunnere42]

Hans
I remember seeing in one of the  upgrade scripts (could be the last one from Itus) that it checked for which mode it was running, then did the appropriate action, but can't remember which script

roadrunnere42

[hans2]

I will have a look at my RC1 and v1 images - thanks for the tip!

[Roadrunnere42]

Hans

here is want you what to find out which node the shield is in

if  [ `df -h | grep -m1 mmcblk* | awk '{ print sunstr( $0, 6, 14 )  }'` ]: then
DISK_PARTITION=`df -h | grep -m1 mmcblk* | awk '{ print substr( $0, 6, 14 ) }'`
if [ $DISK_PARTITION = mmcblk0p2 ]; then
SHEILD_MODE=Router
elif if [ $DISK_PARTITION = mmcblk0p3 ]; then
SHEILD_MODE=Gateway
elif [ $DISK_PARTITION = mmcblk0p4 ]; then
SHEILD_MODE=Bridge

else
echo "Shield operation error"
fi


roadrunnere42

[hans2]

Roadrunnere42 wrote

Hans

here is want you what to find out which node the shield is in

if  [ `df -h | grep -m1 mmcblk* | awk '{ print sunstr( $0, 6, 14 )  }'` ]: then
DISK_PARTITION=`df -h | grep -m1 mmcblk* | awk '{ print substr( $0, 6, 14 ) }'`
if [ $DISK_PARTITION = mmcblk0p2 ]; then
SHEILD_MODE=Router
elif if [ $DISK_PARTITION = mmcblk0p3 ]; then
SHEILD_MODE=Gateway
elif [ $DISK_PARTITION = mmcblk0p4 ]; then
SHEILD_MODE=Bridge

else
echo "Shield operation error"
fi

roadrunnere42

... [show rest of quote]

Awesome. What I will probably do is save the output in a .mode file and have LuCI pull the contents.

That means an update to /usr/lib/lua/luci/view/admin_status/index.htm

Where should I put this script - in /etc/rc.local so that it runs once during startup?

[Roadrunnere42]

Hans

I'm still learning so best left to your expert judgement .


ps hotfix still did not run on Shield, i can open file on my computer so will just copy across.
 
roadrunnere42

[hans2]

Can you check the MD5SUM of the file - it should be 5f13e013787d7332a344d35362100d4b
It matches my google drive too, this is where i put my files for this forum before upload.

root@Shield:/tmp/d# md5sum ../hotfix_160301-BETA4.tgz
5f13e013787d7332a344d35362100d4b  ../hotfix_160301-BETA4.tgz
root@Shield:/tmp/d# tar -zxvf ../hotfix_160301-BETA4.tgz
tmp/make_hotfix.txt
sbin/fw_upgrade
etc/init.d/dnsmasq
etc/itus/update_blacklist.sh
etc/itus/write-categories.sh
tmp/upgrade_rc_to_sp1.sh
.hf_date
usr/lib/lua/luci/view/admin_status/index.htm
usr/lib/lua/luci/view/admin_network/diagnostics.htm
tmp/cleanup.sh
tmp/cleanup_list
etc/rc.local
etc/init.d/ntpclient
etc/opkg.conf
etc/snort/snort.conf
usr/lib/lua/luci/model/cbi/e2guardian.lua

[Roadrunnere42]

Han

I'm getting

root@Shield:/# md5sum hotfix_160301-BETA4.tgz
4c26561a89807b0348f07f1792756e26  hotfix_160301-BETA4.tgz


roadrunnere42

[hans2]

I have updated the FINAL version of the patch - will focus on bug fixes afterwards.

The last changes are:

21) 	= LUCI WF - CONTENT FILTERING OPTIONS
	> /usr/lib/lua/luci/model/cbi/e2guardian.lua
	- removed all but Ads, Malicious and Drugs from option list 
	- this is related due to limitations of fw_upgrade script

22) 	= LUCI - UTM MODE DISPLAY
	> /usr/lib/lua/luci/view/admin_status/index.htm /etc/rc.local /.shield_mode /etc/itus/detect_mode.sh
	- runs at startup detect-mode script. This determines router/bridge/gateway mode

thanks to Roadrunner42:
1) the fw_upgrade script can now also filter DRUGS related content. See http://itus.accessinnov.com/Update-script-fw-upgrade-td43.html for more details.
2) In LuCI the UTM Mode file will now show UTM Router, Bridge or Gateway. See http://itus.accessinnov.com/Hotfix-160301-BETA-ONLY-td157.html#a308

[CWS]

CONTENTS DELETED

The author has deleted this message.