Notepad ~Angky R~ » 2008 » May

[squid-users] All url_rewriter processes are busy x Too many open files

May 29, 2008 SQUID No Comments

Amos Jeffries
Tue, 01 Apr 2008 21:18:49 -0700

Marcio Augusto Stocco wrote:
Testing Squid/SquidGuard with thousands of users, the cache.log shows
the following messages:

2008/04/01 15:19:16| WARNING: All url_rewriter processes are busy.
2008/04/01 15:19:16| WARNING: up to 2730 pending requests queued
2008/04/01 15:19:16| Consider increasing the number of url_rewriter
processes to at least 3552 in your config file.
2008/04/01 15:19:34| WARNING! Your cache is running out of filedescriptors
2008/04/01 15:19:50| WARNING! Your cache is running out of filedescriptors
2008/04/01 15:19:56| comm_open: socket failure: (24) Too many open files
2008/04/01 15:19:56| comm_open: socket failure: (24) Too many open files
2008/04/01 15:19:56| comm_open: socket failure: (24) Too many open files

The server is a HP DL360G5 (2x Xeon Dual 1.6 GHz, RAM 8 GB, HP Smart
Array - RAID 1).

Is there any way to increase SQUID_MAXFD from 8192 to 65536, so I can
try using the sugested number of url_rewriter processes?
Squid 2.6: --with-maxfd=65536
Squid 3.x: --with-filedescriptors=65536

Be sure your OS can handle a single process with that many FD though. Using these options overrides the automatic build detections AFAIK. 

You can also use ulimit while compiling (I don't know the details).
With SQUID_MAXFD=8192 I got lots of "comm_open: socket failure: (24)
Too many open files" if url_rewriter is set higher than 200 (roughly).

Thanks for any help,
Marcio.
For our info, you say you are handling thousands of users;
and what release of squid is it?
what request/sec load is your squid maxing out at?

Amos

Optimizations SQUID for diskdaemon

May 29, 2008 SQUID No Comments

Awalnya seperti biasa, nemu error semacem ini

2008/05/25 22:44:49| storeDiskdSend: msgsnd: (35) Resource temporarily unavailable
2008/05/25 22:44:49| storeDiskdSend OPEN: (35) Resource temporarily unavailable
2008/05/25 22:44:51| storeDiskdSend: msgsnd: (35) Resource temporarily unavailable
2008/05/25 22:44:51| storeDiskdSend OPEN: (35) Resource temporarily unavailable
2008/05/25 23:45:38| sslReadClient: FD 60: read failure: (60) Operation timed out

di cache.log dan terkadang si SQUID suka mati tiba2 servicenya, cari-cari di forum dan milis squid, katanya harus ada optimize kernel untuk cache_dir yang menggunakan option diskd, dan memang kalo digunakan ufs sih ndak masalah.

Berikut parameter yang harus di tambah di kernelnya :-)

FreeBSD
setting in: in /etc/sysctl.conf

kern.maxfilesperproc=8192

setting in the kernel config file (larger values may be needed for very busy caches):

options MSGMNB=8192 # max # of bytes in a queue
options MSGMNI=40 # number of message queue identifiers
options MSGSEG=512 # number of message segments per queue
options MSGSSZ=64 # size of a message segment
options MSGTQL=2048 # max messages in systemoptions SHMSEG=16

options SHMMNI=32
options SHMMAX=2097152
options SHMALL=4096
options MAXFILES=16384

Linux

Stefan Köpsell reports that if you compile sysctl support into your kernel, then you can change the following values:

  • kernel.msgmnb
  • kernel.msgmni
  • kernel.msgmax

Winfried Truemper reports: The default values should be large enough for most common cases. You can modify the message queue configuration by writing to these files:

  • /proc/sys/kernel/msgmax
  • /proc/sys/kernel/msgmnb
  • /proc/sys/kernel/msgmni

selesai sudah deh, tinggal compile kernel dan restart cache_dir, Alhamdullilah cara ini sementara ampuh menghandle issue ini, ntar itu bugs ato apa yg jelas so far so good setelah melakukan optimize kernel :-)

Ref: http://squid.biz.net.id/faq

Thanks,
~Angky R~

Happy 25th Birthday DNS

May 28, 2008 IT No Comments

I have invitation from DNSstuff to attend a roundtable event celebrating the 25th Anniversary of DNS! The most influential DNS experts in the industry will be represented – including the father of DNS – Paul Mockapetris.

Seated at the DNS expert Roundtable:
- Paul Parisi, CTO of DNSstuff.com – Speaker and Moderator
- Paul Mockapetris, Inventor of DNS
- Cricket Liu, DNS expert & Author
- Paul Vixie, Author & Architect of BIND

This team will discuss why DNS is necessary, what the state of DNS is now and what the future holds.
Click here to Register Today!

Thanks,
~Angky R~

Transparent NAT on Foundry for ServerIron

May 27, 2008 Network No Comments

Istilah pada Title mungkin kurang tepat, tetapi menurut saya ya itu :-)

Ceritanya dengan mengimplentasikan config coretan saya yang “ServerIron and the real servers are on different sub-nets” kurang memuaskan bagi saya. Memang sih tidak ada yang salah dengan config itu, namun ketika config itu di implementasikan untuk Load Balancing DNS ada sedikit issue (seperti yang saya utarakan juga) dan Alhamdullilah issue itu sudah bisa di atasi, dimana source IP client bisa tembus dan tercatat di log oleh server DNS, dimana sebelumnya yang terdeteksi adalah IP dari source nat si Foundry. berikut contoh log yang terdeteksi oleh server DNS

26-May-2008 11:19:05.340 client 10.10.10.1#17689: view network-biznet: query: media.netapp.com IN A +
26-May-2008 11:19:05.340 createfetch: netapp.mediacache.clickability.com A

tetapi setelah fungsi Transparent NAT yang gw maksud itu berjalan mulus, hasil log di server DNS menjadi

27-May-2008 12:19:05.472 client 202.169.33.251#17689: view network-biznet: query: ftp.compaq.com IN A +
27-May-2008 12:19:05.472 createfetch: ftp.compaq.com A

Nah, berikut config lengkapnya ya…..

Current configuration:
!
ver 07.3.05T12
!
!
server port 53
allow-recursive-search
udp
server source-nat-ip 10.10.10.1 255.255.255.0 0.0.0.0
!
!
!
!
!
!
!
!
!
!
!
!
!
server real ns2_0 10.10.10.2
port dns
port dns keepalive
port dns zone “biz.net.id”
!
server real ns2_1 10.10.10.3
port dns
port dns keepalive
port dns zone “biz.net.id”
!
!
server virtual ns2.biz.net.id 141.149.66.2
predictor round-robin
port dns
bind dns ns2_0 dns ns2_1 dns
!
!
!
enable telnet password …..
enable super-user-password …..
hostname JKTBIZ-SLB-DNS1
ip address 141.149.66.10 255.255.255.0
ip nat inside
ip nat inside source static 10.10.10.2 141.149.66.2
ip nat inside source static 10.10.10.3 141.149.66.2
ip default-gateway 141.149.66.1
snmp-server community ….. rw
clock timezone gmt GMT+07
sntp server 218.100.41.254
!
!
!
!
end

Adapun perubahan yang dilakukan adalah
1. Remove fungsi “server source-nat”
2. Rubah nat pool menjadi static nat
3. Remove access-list

Dengan demikian semua berjalan dengan baik….so kalo ada yang kurang ato salah, isi di comment aja ya :-)

Thanks,
~Angky R~

ServerIron and the real servers are on different sub-nets (Foundry)

May 22, 2008 Network No Comments

Sebenarnya 6 bulan yang lalu sempat iseng-iseng bikin Server Load Balancing pake Foundry ServerIronXL, ceritanya berhasil dan nyoba juga implementasi antara IP real server dengan IP Management Foundry/Virtual IP beda sub-net (ya bisa di bilang real dibuat IP Private dan virtual IP dibuat public), mungkin gambaran umunya seperti ini
design

Nah berhubung gw ini pelupa banget…(maklum lah :P ) jadinya ketika mau dipake dan diimplentasikan bener-bener gw jadi kocar-kacir cari documentasi lama, berharap dulu sempat curat coret mengenai ini, tapi hasilnya nihil….trus coba minta bantuan uncle Google tapi masih banyak yang kurang jelas juga….alhasil gw musti rada sedikit kerja keras buat bikin ini karena si “Bos” sudah mulai mengluarkan WARNING :-)

Tapi dengan usahan yang gigih (cieee kaya yg iye aje), doa dan tawekal akhirnya gw bisa juga dan hari ini pun gw langsung bikin documentasinya biar ngak lupa lagi :D …..langsung aje deh, ceritanya gw mau bikin Server Load Balancing pake foundry untuk server DNS yang mana real server dengan Virtual Server berbeda subnet…..berikut cuplikan confignya

*Kondisinya:
Management IP Address: 141.149.66.10
VIP 141.149.66.2
Real Server 1 10.10.10.2
Real Server 2 10.10.10.3
Gw real server 10.10.10.1

Current configuration:
!
ver 07.3.05T12
!
!

server port 53
allow-recursive-search
udp
server source-nat
server source-nat-ip 10.10.10.1 255.255.255.0 0.0.0.0
!
!
!
!
!
!
!
!
!
!
!
!
!
server real dns2_0 10.10.10.2
source-nat
port dns
port dns keepalive
port dns zone “biz.net.id”
!
server real dns2_1 10.10.10.3
source-nat
port dns
port dns keepalive
port dns zone “biz.net.id”
!
!
!
server virtual dns2.biz.net.id 141.149.66.2
predictor round-robin
port dns
bind dns dns2_1 dns dns2_0 dns
!

!
!
enable telnet password …..
enable super-user-password …..
hostname JKTBIZ-SLB-DNS1
ip address 141.149.66.10 255.255.255.0
ip nat inside
ip nat inside source list 1 pool jktns2 overload
ip nat pool jktns2 141.149.66.2 141.149.66.2 netmask 255.255.255.0
ip default-gateway 141.149.66.1
ip filter 100 permit any any
snmp-server community ….. rw
!
!
!
access-list 1 permit 10.10.10.0 0.0.0.255
!
!
end

Nah kira2 itu config yg gw pake, Alhamdullilah berjalan dengan normal dan lancar….so keep smile

Issue: Dengan mengimplentasikan konsep ini ada sedikit issue yang belum terpecahkan yaitu, fungsi ACL pada Server DNS tidak bisa berfungsi karena server DNS (real server) akan mendeteksi packet datangnya dari IP yang di allow yaitu gw dari dirinya sendiri, dalam hal ini IP 10.10.10.1 punya Foundry, sehingga siapapun yang mengakses IP DNS 141.149.66.2 akan dilayani oleh server DNS walaupun fungsi recursion untuk network yang tidak di allow sudah dimatikan :-)

Mungkin next step kalo nemu cara mengatasi issue ini akan langsung saya update lagi tulisan ini….

Thanks,
~Angky R~

« Previous Entries