Video YouTube Tidak Dapat Di Cache Squid

Saya sempat memposting tentang cara memaksa agar squid proxy memaksa agar video di Youtube dapat di cache dengan baik – silakan cek posting dynamic content caching. Akan tetapi sejak awal Maret 2013, sepertinya teknik tersebut sudah tidak dapat digunakan untuk melakukan cache video youtube. Bagaimana hal ini bisa terjadi?

!! Silakan cek update storeurl terbaru di bagian bawah artikel ini !!

Story behind YouTube video cache

Video YouTube memiliki sifat sebagai dynamic content (konten dinamik) walaupun pada faktanya video yang ditampilkan di YouTube dengan resolusi apapun adalah object web yang statik. Dikatakan content dynamic karena setiap video yang ada di YouTube memerlukan beberapa parameter tertentu (video ID, itag, range, dan sebagainya) untuk kemudian di eksekusi oleh server lewat perantaraan server side scripting. Namun juga dapat dikatakan statik karena video yang di host dan di distribusikan oleh CDN adalah object yang sama persis dengan object yang di push dari server asalnya.

yt-fail

Konsep dibelakang dynamic content caching adalah menyimpan (kedalam cache) object web yang di generate secara dinamik oleh server pada saat diakses pertama kali. Pada saat akses selanjutnya, user tidak lagi mengambil object tersebut ke internet, namun cukup mengambilnya dari cache lokal, sehingga bandwidth dapat dihemat, dan konten/object dapat disampaikan dengan kecepatan penuh – full speed.

Setidaknya ada 2 metode duplikasi video YouTube yang di implementasikan dengan proxy squid, pertama menyimpan video kedalam direktori cache squid itu sendiri, dan yang kedua adalah menggunakan perantara aplikasi ketiga – misalnya apache, nginx, dan sebagainya untuk menyimpan video-video tersebut kedalam direktori lain. Kedua metode ini pada umumnya menggunakan fitur storeurl_rewrite_program – fitur pada squid 2.7.

Working on Youtube’s videos caching

Metode yang paling banyak digunakan untuk melakukan caching video Youtube adalah metode yang fenomenal yang dipopulerkan oleh mas Chuddy Fernandez. Eyang Chuddy menggunakan fitur  storeurl untuk menduplikasi video YouTube kedalam cache_dir squid. Silakan baca tutorial beliau disini.

Pada awalnya, setiap video di youtube memiliki video id yang statik. Tidak peduli apakah video tersebut non-range (full video) atau video yang telah dipotong-potong (ranged).  Setiap video atau potongan video tersebut di host oleh server CDN yang tersebar diseluruh dunia, sehingga bisa saja pada ranged video setiap potongan videonya di hosting oleh server CDN yang berbeda.

Ide youtube video caching adalah melakukan caching yang agresif untuk video-video atau potongan video yang sering diakses atau populer.

Pada squid versi 2.7 diperkenalkan konsep storeurl rewrite yang dibahas tadi. storeurl rewrite memungkinkan kita untuk memanipulasi / me-rewrite URL yang diakses oleh klien squid sebelum menyimpan atau mengakses object didalam cache squid.

How it works

Contohnya saya mengakses sebuah video youtube, dan secara otomatis youtube menanayangkan sebuah video dengan URL berikut (ini cuman contoh… bukan URL beneran.. hehehe)

http://r17—sn-npo7en7r.c.youtube.com/videoplayback?id=OkjnGF45&itag=34&ip=10.20.30.254&region=id&signature=98JLJKDBQDAKHJHH553HKK4H432M3NB54J32LN42K455503.2KNAFAF5AF&sver=2&expire=87654321445&key=yt4&ipbits=0&range=1-878676

Sebagian orang mungkin akan bingung melihat URL tersebut – karena cukup panjang dan memiliki banyak parameter (id, itag, region, dsb), namun tahukah anda, sebenarnya untuk mengakses video tersebut kita cukup mendefinisikan beberapa parameter saja.

Untuk video non-range, parameter yang diperlukan hanya id dan itag. Sedangkan untuk video ranged/partial, parameter yang diperlukan hanya id, itag, dan range. Parameter lainnya hanya bersifat tracking, penentuan signature, dan sebagainya.

Quote dari eyang Chuddy :

UPDATE: &range suppose to be partial contents… you may redirect them without “&range=xxx-xxx” to cache the whole content

Sehingga dengan bantuan storeurl rewriter, kita dapat menyederhanakan URL tersebut.

Untuk video non-range :

http://r17—sn-npo7en7r.c.youtube.com/videoplayback?id=OkjnGF45&itag=34

Untuk video range :

http://r17—sn-npo7en7r.c.youtube.com/videoplayback?id=OkjnGF45&itag=34&range=1-878676

Lebih pendek bukan? Selain itu, url pendek seperti ini lebih ‘cache-friendly’ alias lebih “enak” untuk di cache oleh squid. Video yang hit akan tampak seperti ini pada di squid access.log

1362105243.589  17760 10.20.30.254 TCP_HIT/200 156611767 GET http://r17—sn-npo7en7r.c.youtube.com/videoplayback? – NONE/- video/mp4

Teknik seperti ini bertahan sampai awal maret tahun 2013. :)

YouTube cache : March 2013 until today

Mimpi buruk datang pada awal Maret tahun 2013 – Youtube merubah coding mereka! Hal ini membuat para administrator / pengelola squid proxy menjadi galau, dan frustasi, ada yang menjadi linglung, insomnia, susah tidur, lupa cebok, berhalusinasi, ingin coli terus, kehilangan fokus dan bahkan ada yang ingin gantung diri. Apalagi banyak video youtube yang sedang booming, sebut saja X-Factor (Fatiiinn… I love you!! ~~), Indonesia Mencari Bakat, video harlem shake, video bokep, dan banyak lagi.

Video id (atau parameter sejenisnya) yang awalnya bersifat statik dirubah menjadi dinamik alias berubah-ubah. Alhasil, hampir semua video youtube tidak dapat di cache oleh squid. Jika anda menggunakan storeurl yang seperti mirip eyang Chuddy, maka anda mungkin akan menemui video yang dibuka tidak sesuai dengan video yang di inginkan, karena video id yang tertukar, atau hal lainnya.

Contoh kegalauan di Facebook yang lucu dan unik .. hehehe. (ane juga galau kok… sama seperti anda-anda semua, mari kita bergalau riaaa…!!)

galau-youtube

Youtube juga mengimplementasikan url redirection yang semakin mempersulit bagi squid untuk melakukan cache video-video youtube.

Video kegalauan youtube di Facebook :)

Credits to Forum Belajar Bareng – video by Timunx Spectra

 

 

 Kesimpulan

Untuk sementara, sangat sulit untuk melakukan caching video Youtube oleh squid. Belum ada teknik yang di publikasikan yang benar-benar efisien untuk melakukan hal tersebut. :D

Jadi, bagaimana cara mengatasi masalah ini? Jawabannya : silakan menunggu, karena belum ada storeurl yang sip untuk hal ini. hehehehe…

New Update!! New Update!!

Mas ucok_karnadi pada posting dibawah menambahkan storeurl terbaru untuk caching video Youtube. Silakan download di :

https://code.google.com/p/tempat-sampah/source/browse/storeurl.pl

Yang perlu dicatat adalah untuk menggunakan storeurl tersebut anda harus menginstal modul ReadBackwards untuk Perl. Silakan download di : http://search.cpan.org/~uri/File-ReadBackwards-1.05/ReadBackwards.pm

Untuk pemakai Ubuntu, anda juga dapat menginstal modul ini menggunakan apt-get.

nuff said. :)

About Fajar R. 297 Articles
Seseorang yang antusias terhadap dunia internet, IT, networking dan komputer. Tipikal introvert tapi suka berorganisasi dan adventuring ke tempat-tempat baru.

24 Komentar

  1. mas fazar..kq youtube saya gk ke cache yach..padahal udah pake storeurl yg terbaru dan udah install ReadBackwards.saya menggunakan ubuntu server 12 dan squid lusca 14809.makasih

  2. kenapa y maz setelah pake proxy kok ssamrtbilling gak mw kebuka harus di disable dulu tranparent proxynya dan game online gak kebuka, trims

  3. mas, maaf mau nanya lagi.
    begini squid aku udah jalan dan storeurl.pl nya aku pakai dari mas ucok_karnandi yang mas share. pas aku cek pakai lusca -d1 running. nah permasalahan nya kenapa youtube saya jadi error ?
    gimana mas solusinya ?

    • ada acl non range ya? dan jgn kupa ketikkan perintah : touch /var/log/squid/yt.log

      lakukan chown pada file yt.log jika dibutuhkan.

      • mas, kenapa mas yah ketika saya mencoba utk memblokir website menggunakan lusca, youtube menjadi error. tapi kalau settingan utk memblokir dihapus, lusca jalan.
        kenapa itu mas ? apa tdk bisa memblokir situs sambil melakukan cache ?
        kalau blokir –> lusca cache gk jalan
        kalau tdk blokir –> lusca cache jalan

        tolong yah mas.

          • ini squid.conf saya mas :

            #=============================================
            # Port and Transparent
            #=============================================
            http_port 3128 transparent
            server_http11 on
            icp_port 0
            #=============================================
            # Lusca Cache Directory
            #=============================================
            cache_dir aufs /cache 46000 108 256
            cache_replacement_policy heap LFUDA
            memory_replacement_policy heap GDSF
            #=============================================
            # Lusca Log Options
            #=============================================
            emulate_httpd_log on
            logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
            log_fqdn off
            logfile_rotate 14
            debug_options ALL,1
            cache_access_log /var/log/lusca/access.log
            cache_log /var/log/lusca/cache.log
            cache_store_log /var/log/lusca/store.log
            #=============================================
            # DNS and FTP option
            #=============================================
            #I used DNSAMSQ service for fast dns resolving
            #so install by using "apt-get install dnsmasq" first
            dns_nameservers 202.18.6.2 203.130.206.250 8.8.4.4 192.168.4.1 127.0.0.1 221.132.112.8
            ftp_user anonymous@
            ftp_list_width 32
            ftp_passive on
            ftp_sanitycheck on
            #=============================================
            # Access Control List (ACL) Option
            #=============================================
            acl all src 0.0.0.0/0.0.0.0
            acl manager proto cache_object
            acl localhost src 127.0.0.1/255.255.255.255
            acl localnet src 192.168.88.0/24
            acl to_localhost dst 127.0.0.0/8
            acl SSL_ports port 443 563 873 # https snews rsync
            acl Safe_ports port 80 # http
            acl Safe_ports port 20 21 # ftp
            acl Safe_ports port 70 # gopher
            acl Safe_ports port 210 # wais
            acl Safe_ports port 1025-65535 # unregistered ports
            acl Safe_ports port 631 # cups
            acl Safe_ports port 10000 # webmin
            acl Safe_ports port 901 # SWAT
            acl Safe_ports port 280 # http-mgmt
            acl Safe_ports port 488 # gss-http
            acl Safe_ports port 591 # filemaker
            acl Safe_ports port 777 # multiling http
            acl Safe_ports port 873 # rsync
            acl Safe_ports port 110 # POP3
            acl Safe_ports port 25 # SMTP
            acl Safe_ports port 2095 2096 # webmail from cpanel
            acl Safe_ports port 2082 2083 # cpanel
            acl purge method PURGE
            acl CONNECT method CONNECT
            http_access deny manager
            http_access allow SSL_ports
            http_access allow purge localhost
            http_access deny purge
            http_access deny !Safe_ports
            http_access deny CONNECT !SSL_ports
            http_access allow localhost
            http_access allow localnet
            http_access allow all
            http_reply_access allow all
            icp_access allow all
            #=============================================
            # Administrative Parameters
            #=============================================
            # I used Ubuntu Server so user and group is proxy
            cache_effective_user proxy
            cache_effective_group proxy
            cache_mgr methodist-8.sch.id
            visible_hostname methodist-8.sch.id
            unique_hostname methodist-8.sch.id
            #=============================================
            # Accelerator Options
            #=============================================
            acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
            upgrade_http0.9 deny shoutcast
            acl apache rep_header Server ^Apache
            broken_vary_encoding allow apache
            icp_hit_stale on
            query_icmp on
            shutdown_lifetime 10 seconds
            negative_ttl 2 minutes
            pipeline_prefetch on
            high_page_fault_warning 2
            nonhierarchical_direct on
            prefer_direct off
            half_closed_clients off
            quick_abort_min 0 KB
            quick_abort_max 0 KB
            vary_ignore_expire on
            reload_into_ims on
            log_fqdn off
            memory_pools off
            cache_swap_low 98
            cache_swap_high 99
            max_filedescriptors 65536
            fqdncache_size 16384
            ipcache_low 98
            ipcache_high 99
            retry_on_error on
            offline_mode off
            pipeline_prefetch on
            # If you want to hide your proxy machine from being detected at various site use following
            via off
            #=============================================
            # Options Which Affect The Cache Size
            #=============================================
            # If you have 4GB memory in Squid box, we will use formula of 1/3
            # You can adjust it according to your need. IF squid is taking too much of RAM
            # Then decrease it to 128 MB or even less.
            cache_mem 8 MB
            minimum_object_size 0 bytes
            maximum_object_size 100 MB
            maximum_object_size_in_memory 128 KB
            #=============================================
            # SNMP , if you want to generate graphs for -
            # SQUID via MRTG
            #=============================================
            snmp_port 3401
            acl snmppublic snmp_community public
            snmp_access allow snmppublic all
            #=============================================
            # ZPH Option
            #=============================================
            tcp_outgoing_tos 0x30 all
            zph_mode tos
            zph_local 0x30
            zph_parent 0
            zph_option 136
            #=============================================
            # ACL Caching Youtube
            #=============================================
            acl youtube dstdomain .youtube.com
            cache allow youtube
            cache allow all
            quick_abort_min -1 KB
            acl ytrange url_regex ^.*youtube.com/videoplayback.*range=.*$
            http_access deny ytrange
            acl video urlpath_regex \/(get_video|videoplayback\?id|videoplayback.*id)
            acl speedtest urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|txt)\?
            acl angka1 url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
            acl angka2 url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
            acl gambar urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
            acl rapidshare url_regex \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]* ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*) \.doubleclick\.net.*
            acl photobucket url_regex ^http:\/\/[.a-z0-9]*\.photobucket\.com.*\.[a-z]{3}$ quantserve\.com
            acl google url_regex ^http:\/\/[a-z]+[0-9]\.google\.co(m|\.id)
            acl indowebster url_regex ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(rar|zip|flv|wm(a|v)|3gp|mp(4|3)|exe|msi|avi|(mp(e?g|a|e|1|2|3|4))|cab|exe)
            acl store_rewrite_list urlpath_regex \/(get_video|videoplayback\?id|videoplayback.*id)
            acl store_rewrite_list urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|swf)\?
            acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
            acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
            acl store_rewrite_list_path urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
            acl store_rewrite_list_domain_CDN url_regex (khm|mt)[0-9]?.google.com
            acl store_rewrite_list_domain_CDN url_regex photos-[a-z].ak.fbcdn.net
            acl store_rewrite_list_domain_CDN url_regex \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]*
            acl store_rewrite_list_domain_CDN url_regex ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*)
            acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[.a-z0-9]*\.photobucket\.com.*\.[a-z]{3}$
            acl store_rewrite_list_domain_CDN url_regex (khm|mt)[0-9]?.google.co(m|\.id)
            acl store_rewrite_list_domain_CDN url_regex streamate.doublepimp.com.*\.js\? \.doubleclick\.net.* yieldmanager cpxinteractive quantserve\.com
            acl dontrewrite url_regex yimg.com redbot\.org (get_video|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]* \.php\? threadless.*\.jpg\?r=
            acl getmethod method GET
            storeurl_access deny dontrewrite
            storeurl_access deny !getmethod
            storeurl_access allow video
            storeurl_access allow speedtest
            storeurl_access allow gambar
            storeurl_access allow rapidshare
            storeurl_access allow photobucket
            storeurl_access allow indowebster
            storeurl_access allow store_rewrite_list_domain_CDN
            storeurl_access allow store_rewrite_list
            storeurl_access allow store_rewrite_list_domain store_rewrite_list_path
            storeurl_access deny all
            storeurl_rewrite_program /etc/squid/storeurl.pl
            storeurl_rewrite_children 1
            storeurl_rewrite_concurrency 99
            max_stale 10 years
            acl QUERY urlpath_regex -i \.(ini|ui|lst|inf|pak|ver|patch)$
            acl QUERY urlpath_regex -i (dat.asp|afs.dat|notice.swf|patchlist.txt|hackshield|captcha|reset.css|update.ver|notice.html|updates.txt|gamenotice)
            cache deny QUERY
            #=============================================
            # Refresh Pattern Options
            #=============================================
            refresh_pattern -i \.flv$ 10080 80% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
            refresh_pattern -i \.mp3$ 10080 80% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
            refresh_pattern -i \.mp4$ 10080 80% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
            refresh_pattern -i \.swf$ 10080 80% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
            refresh_pattern -i \.gif$ 10080 80% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
            refresh_pattern -i \.jpg$ 10080 80% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
            refresh_pattern -i \.jpeg$ 10080 80% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
            refresh_pattern -i \.exe$ 10080 80% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
            refresh_pattern +\....?$ 11080 100% 10080
            refresh_pattern ^(https?|ftp)\:\/\/+ 11080 100% 10080
            refresh_pattern ^(https?|ftp)\:\/\/+\....?$ 11080 100% 10080
            refresh_pattern (get_video\?|videoplayback\?|videodownload\?) 5259487 99999999% 5259487 override-expire ignore-reload ignore-private negative-ttl=0
            refresh_pattern -i \.(class|css|js|xml)$ 1440 50% 10080 ignore-no-cache ignore-private reload-into-ims override-lastmod
            refresh_pattern -i \.index\.(html|htm|php)$ 0 50% 1
            refresh_pattern -i \.(cgi|asp|shtml|pl|html|htm)$ 0 20% 1
            refresh_pattern -i (get_video\?|videoplayback\?id|videoplayback.*id) 10080 500% 43200 override-expire ignore-no-cache ignore-private ignore-reload
            refresh_pattern imeem.*\.flv 0 0% 0 override-lastmod override-expire
            refresh_pattern ((google.com)|(74.125.135.106)).*\.(html|htm|js|php|swf|xml|txt|jpg|png|gif) 129600 999999% 129600 ignore-reload ignore-no-cache ignore-no-store store-stale
            refresh_pattern \.speedtest/.* 43200 9999% 43200 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-must-revalidate store-stale
            refresh_pattern ^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*) 129600 999999% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims store-stale negative-ttl=0
            refresh_pattern -i \.(sc-|dl-|ex-|mh-|mst|dll)$ 0 20% 0
            refresh_pattern -i (main.exe|notice.html)$ 0 20% 0
            refresh_pattern -i (livescore.com|UpdaterModifier.exe|FreeStyle.exe|FSLauncher.exe) 0 20% 0
            refresh_pattern .speedtest.* 0 60% 10 negative-ttl=0
            refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]* 161280 90% 161280 ignore-reload store-stale
            refresh_pattern ^.*safebrowsing.*google 129600 999999% 129600 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth ignore-must-revalidate negative-ttl=10080 store-stale
            refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk) 129600 999999% 129600 override-expire ignore-reload store-stale ignore-private negative-ttl=10080
            refresh_pattern ytimg\.com.*\.(jpg|png) 129600 999999% 129600 override-expire ignore-reload store-stale
            refresh_pattern (avgate|avira).*\.(idx|gz)$ 1440 90% 1440 ignore-reload ignore-no-cache ignore-no-store store-stale ignore-must-revalidate
            refresh_pattern images\.friendster\.com.*\.(png|gif) 129600 999999% 129600 override-expire ignore-reload store-stale
            refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(png|gif) 129600 999999% 129600 override-expire ignore-reload store-stale
            refresh_pattern garena\.com 129600 999999% 129600 override-expire reload-into-ims store-stale
            refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 999999% 129600 override-expire ignore-reload store-stale
            refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\? 129600 999999% 129600 ignore-no-cache override-expire override-lastmod store-stale
            refresh_pattern .fbcdn.net.*\.(jpg|gif|png) 129600 999999% 129600 ignore-no-cache override-expire ignore-reload store-stale negative-ttl=5
            refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(rar|mov|mkv|cab|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store ignore-private store-stale ignore-auth
            refresh_pattern imeem.*\.flv 0 0% 0 override-lastmod override-expire store-stale
            refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?) 5259487 99999999% 5259487 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims store-stale
            refresh_pattern (get_video\?|videoplayback\?|videodownload\?) 5259487 99999999% 5259487 override-expire ignore-reload
            refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?) 5259487 99999999% 5259487 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims store-stale
            refresh_pattern -i (get_video\?|videoplayback\?id|videoplayback.*id||videodownload\?|\.flv?) 5259487 99999999% 5259487 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims store-stale
            refresh_pattern \.(ico|video-stats) 5259487 99999999% 5259487 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod negative-ttl=10080 store-stale
            refresh_pattern \.etology\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache store-stale
            refresh_pattern galleries\.video(\?|sz) 129600 999999% 129600 override-expire ignore-reload ignore-no-cache store-stale
            refresh_pattern brazzers\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache store-stale
            refresh_pattern \.adtology\? 129600 999999% 129600 override-expire ignore-reload ignore-no-cache store-stale
            refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 129600 20% 129600 ignore-no-cache ignore-no-store ignore-private override-expire ignore-reload ignore-auth store-stale negative-ttl=40320 max-stale=10
            refresh_pattern ^.*safebrowsing.*google 129600 999999% 129600 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth negative-ttl=10080 store-stale
            refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk) 129600 999999% 129600 override-expire ignore-reload ignore-private store-stale negative-ttl=10080
            refresh_pattern ytimg\.com.*\.jpg 129600 999999% 129600 override-expire ignore-reload store-stale
            refresh_pattern images\.friendster\.com.*\.(png|gif) 129600 999999% 129600 override-expire ignore-reload store-stale
            refresh_pattern garena\.com 129600 999999% 129600 override-expire reload-into-ims store-stale
            refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 999999% 129600 override-expire ignore-reload store-stale
            refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\? 129600 999999% 129600 ignore-no-cache override-expire override-lastmod store-stale
            refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 999999% 129600 reload-into-ims override-expire ignore-private store-stale
            refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\. 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire store-stale
            refresh_pattern ^http:\/\/www.onemanga.com.*\/ 129600 999999% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire store-stale
            refresh_pattern ^http://file.pb.gemscool.com/hackshield/.* 720 100% 4320 override-expire override-lastmod reload-into-ims store-stale

            refresh_pattern .speedtest.* 0 60% 10 negative-ttl=0
            refresh_pattern (get_video|videoplayback|videodownload|\.flv).*(begin|start)\=[1-9][0-9]* 0 0% 0
            refresh_pattern imeem.*\.flv 0 0% 0 override-lastmod override-expire
            refresh_pattern ^ftp: 40320 20% 40320 override-expire reload-into-ims store-stale
            refresh_pattern ^gopher: 1440 0% 1440
            refresh_pattern -i (livescore.com|UpdaterModifier.exe|FreeStyle.exe|FSLauncher.exe) 0 20% 0
            refresh_pattern speedtest.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png|swf|txt|js) 0 50% 180 store-stale negative-ttl=0
            #ads
            refresh_pattern ^.*safebrowsing.*google 131400 999999% 525600 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth negative-ttl=10080 store-stale
            refresh_pattern ^.*(streamate.doublepimp.com.*\.js\?|utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 5259487 20% 5259487 ignore-no-cache ignore-no-store ignore-private override-expire ignore-reload ignore-auth store-stale negative-ttl=40320 max-stale=1440
            # banner IIX
            refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 129600 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            refresh_pattern ^http:\/\/img.ads.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            refresh_pattern ^http:\/\/openx.kompas.com.*\/ 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            refresh_pattern -i cdn-[a-z]\.kaskus.\us.*\.(jp(e?g|e|2)|gif|png) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 43200 20% 43200 ignore-no-cache ignore-private override-expire ignore-reload
            #ytimg, google, etc
            refresh_pattern ^.*safebrowsing.*google 43200 999999% 43200 override-expire ignore-reload ignore-no-cache ignore-private
            refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.id) 43200 999999% 43200 override-expire ignore-reload ignore-private
            refresh_pattern ytimg\.com.*\.jpg 43200 999999% 43200 override-expire ignore-reload
            refresh_pattern ^http:\/\/www.onemanga.com.*\/ 43200 999999% 43200 reload-into-ims ignore-no-cache ignore-reload override-expire
            refresh_pattern ^http://v\.okezone\.com/get_video\/([a-zA-Z0-9]) 43200 999999% 43200 override-expire ignore-reload ignore-no-cache ignore-private override-lastmod
            #facebook games
            refresh_pattern ^http:\/\/apps.facebook.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache
            refresh_pattern -i \.zynga.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache
            refresh_pattern -i \.farmville.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache
            refresh_pattern -i \.ninjasaga.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache
            refresh_pattern -i \.mafiawars.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache
            refresh_pattern -i \.crowdstar.com.*\/ 10080 999999% 43200 ignore-reload override-expire ignore-no-cache
            #IIX download
            refresh_pattern -i ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache
            #antivirus
            refresh_pattern avast.com.*\.vpx 40320 50% 525600 store-stale reload-into-ims
            refresh_pattern (avgate|avira).*\.(idx|gz)$ 1440 90% 1440 ignore-reload ignore-no-cache ignore-no-store store-stale
            refresh_pattern kaspersky.*\.avc$ 131400 999999% 525600 ignore-reload store-stale
            refresh_pattern kaspersky 1440 50% 131400 ignore-no-cache store-stale
            refresh_pattern .symantecliveupdate.com.*\.zip 1440 90% 131400 store-stale
            refresh_pattern .update.nai.com/.*\.(gem|zip|mcs) 43800 999999% 43800 ignore-reload store-stale
            refresh_pattern .symantec.com.*\(exe|zip) 43800 999999% 43800 ignore-reload store-stale
            refresh_pattern ^http://file.pb.gemscool.com.*\.zip 131400 999999% 131400 override-expire store-stale
            refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(rar|mov|mkv|cab|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 43200 99999% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store ignore-private store-stale ignore-auth
            refresh_pattern .kaskus.us.*\.(jpg|gif|png) 1440 60% 131400 override-expire store-stale
            #fb
            refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif) 43800 200% 10080 ignore-reload override-expire ignore-no-cache ignore-private reload-into-ims
            refresh_pattern -i ^http:\/\/*\.[a-z]\.fbcdn.net.*\.(jpg|gif|png|swf|mp3|flv|mp4) 43800 200% 10080 ignore-reload override-expire ignore-no-cache ignore-private reload-into-ims
            refresh_pattern -i static\.[a-z]\.fbcdn\.net*\.(jpg|gif|png) 43800 200% 10080 ignore-reload override-expire ignore-no-cache ignore-private reload-into-ims
            refresh_pattern -i ^http:\/\/profile\.[a-z]\.fbcdn.net*\.(jpg|gif|png) 43800 200% 10080 ignore-reload override-expire ignore-no-cache ignore-private reload-into-ims
            refresh_pattern -i ^http:\/\/*\.akamaihd\.(net|com).*\.(jpg|gif|png) 43800 200% 10080 ignore-reload override-expire ignore-no-cache ignore-private reload-into-ims
            #specific sites
            refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]* 131400 90% 525600 ignore-reload store-stale
            refresh_pattern ^http://v\.okezone\.com/get_video\/([a-zA-Z0-9]) 131400 999999% 43200 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod negative-ttl=10080 store-stale
            refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?) 525600 99999999% 525600 override-expire ignore-reload ignore-no-cache ignore-private store-stale negative-ttl=0
            refresh_pattern \.(ico|video-stats) 525600 999999% 525600 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod negative-ttl=10080 store-stale
            refresh_pattern \.etology\? 525600 999999% 525600 override-expire ignore-reload ignore-no-cache store-stale
            refresh_pattern galleries\.video(\?|sz) 525600 999999% 525600 override-expire ignore-reload ignore-no-cache store-stale
            refresh_pattern brazzers\? 525600 999999% 525600 override-expire ignore-reload ignore-no-cache store-stale
            refresh_pattern \.adtology\? 525600 999999% 525600 override-expire ignore-reload ignore-no-cache store-stale
            refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.id) 131400 999999% 525600 override-expire ignore-reload store-stale ignore-private negative-ttl=10080
            refresh_pattern ytimg\.com.*\.(jpg|png) 525600 999999% 525600 override-expire ignore-reload store-stale
            refresh_pattern images\.friendster\.com.*\.(png|gif) 131400 999999% 525600 override-expire ignore-reload store-stale
            refresh_pattern garena\.com 525600 999999% 525600 override-expire reload-into-ims store-stale
            refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 525600 999999% 525600 override-expire ignore-reload store-stale
            refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\? 525600 999999% 525600 ignore-no-cache override-expire override-lastmod store-stale
            refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\. 131400 999999% 525600 ignore-no-cache ignore-no-store ignore-reload override-expire store-stale
            refresh_pattern ^http:\/\/www.onemanga.com.*\/ 525600 999999% 525600 reload-into-ims override-expire store-stale
            refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 131400 999999% 525600 reload-into-ims override-expire ignore-private store-stale
            refresh_pattern \.macromedia.com.*\.(z|exe|cab) 131400 999999% 525600 ignore-reload override-expire store-stale
            #files
            refresh_pattern -i \.(gif|png|jpeg|jpg|bmp|tif|tiff|ico)$ 10080 50% 43200 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private store-stale
            refresh_pattern -i \.(gif|png|jpeg|jpg|bmp|tif|tiff|ico)\? 10080 50% 43200 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private store-stale
            refresh_pattern -i \.(xml|html|htm|js|jsp|txt|css|php|asp)$ 10080 50% 43200 reload-into-ims ignore-reload ignore-no-cache ignore-auth store-stale
            refresh_pattern -i \.(xml|js|jsp|txt|css)$ 10080 50% 43200 reload-into-ims ignore-reload ignore-no-cache ignore-auth store-stale
            refresh_pattern -i \.(xml|js|jsp|txt|css)\? 10080 50% 43200 reload-into-ims ignore-reload ignore-no-cache ignore-auth store-stale
            refresh_pattern -i \.(flv|x-flv|mov|avi|qt|mpg|mpeg|swf|wmv)$ 10080 50% 43200 reload-into-ims ignore-reload ignore-no-cache store-stale
            refresh_pattern -i \.(wav|mp3|mp4|au|mid)$ 10080 50% 43200 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private store-stale
            refresh_pattern -i \.(iso|deb|rpm|zip|tar|tgz|ram|rar|bin|ppt|doc)$ 10080 90% 43200 ignore-no-cache ignore-auth store-stale
            refresh_pattern -i \.(zip|gz|arj|lha|lzh|rho|nzp|iop|pak)$ 10080 100% 43200 ignore-no-cache ignore-auth store-stale
            refresh_pattern -i \.(rar|tgz|tar|exe|bin)$ 10080 100% 43200 ignore-no-cache ignore-auth ignore-reload ignore-no-cache store-stale
            refresh_pattern -i \.(hqx|pdf|rtf|doc|swf)$ 10080 100% 43200 ignore-no-cache ignore-auth store-stale
            refresh_pattern -i \.(inc|cab|ad|txt|dll)$ 10080 100% 43200 ignore-no-cache ignore-auth store-stale
            #general
            refresh_pattern \.(jp(e?g|e|2)|tiff?|bmp|gif|png) 131400 999999% 525600 ignore-no-cache ignore-no-store reload-into-ims override-expire store-stale
            refresh_pattern \.(z(ip|[0-9]{2})|r(ar|[0-9]{2})|jar|bz2|gz|tar|rpm|vpu) 131400 999999% 525600 override-expire ignore-no-cache reload-into-ims
            refresh_pattern \.(mp3|wav|og(g|a)|flac|midi?|rm|aac|wma|mka|ape) 131400 999999% 525600 override-expire reload-into-ims ignore-reload
            refresh_pattern \.(exe|msi|dmg|bin|xpi|iso|swf|mar|psf|cab|mar) 131400 999999% 525600 override-expire reload-into-ims ignore-no-store ignore-no-cache
            refresh_pattern \.(mkv|mpeg|ra?m|avi|mp(g|e|4)|mov|divx|asf|wmv|m\dv|rv|vob|asx|ogm|flv|3gp|on2) 525600 9999999% 525600 ignore-private ignore-no-cache override-expire reload-into-ims
            refresh_pattern ^ftp: 1440 20% 10080
            refresh_pattern -i (cgi-bin) 0 0% 0
            refresh_pattern \.(php|jsp|cgi|asx)\? 0 0% 0
            refresh_pattern . 0 50% 525600 store-stale
            #=============================================
            # ALLOWED ACCESS
            #==============================================
            acl lan src 202.18.6.0/24
            acl terlarang url_regex -i "/etc/squid/terlarang.txt"
            http_access deny terlarang
            http_access allow lan
            http_access allow localhost
            http_access deny all
            http_reply_access allow all
            icp_access allow lan
            icp_access allow localhost
            icp_access deny all
            always_direct deny all

          • youtube error dengan konfigurasi yg demikian. :)
            untuk blokir situs tertentu, periksa kembali bagian

            acl terlarang url_regex -i "/etc/squid/terlarang.txt"

            pastikan isi file /etc/squid/terlarang.txt menggunakan regular expression yg benar. untuk lebih mempermudah (ketimbang pake regex) lebih baik gunakan dstdomain.

            acl terlarang dstdomain "/etc/squid/terlarang.txt"

            isi /etc/squid/terlarang.txt seperti ini (contoh saja):

            .youtube.com
            .tube8.com
            .yahoo.com

  4. storeurl chuddy vs storeurl ucok
    youtube broser: agak lama | lebih cepat
    squid HIT | squid HIT
    |
    youtube IDM: lebih cepat | lambat
    file dari | file dari source
    squid HIT | tidak ambil dari
    | squid HIT

    • storeurl chuddy :
      youtube pake browser: agak lama tp squid HIT
      youtube pake IDM: lebih cepat ambil file dr squid HIT

      storeurl ucok karnadi :
      youtube pake browser: lebih cepat squid HIT
      youtube pake IDM: lebih lambat karena ambil file dr source bukan dr squid HIT

      • sebenernya dari struktur coding nya tidak berbeda jauh, jika masbro punya pengalaman yg berbeda untuk tiap storeurl, maka gunakanlah yg menurut masbro yg terbaik. good luck.

  5. apa beda nya storeurl.pl by masta chuddy fernandez vs storeurl.pl by ucok karnadi
    keunggulan n kekurangan nya?? :grin: :roll: :oops:

  6. ok mas…..tapi ada masalah lagi yang muncul…setelah dicoba..kenapa game point blank jika 10 client yang ol…game pb sering disconnct di semua client dan browsingnya tetap jalan…apakah squid nya hang atau apa???
    saya memakai mikrotik rb750g..memakai queue tree…dari alamat ini “http://www.wirelessrouterproxy.com/2012/08/cara-setting-bypass-hit-squidmangle.html”…
    tolong pencerahannya lagi…. :grin:

    • pointblank menggunakan port yg berbeda dengan browsing. jadi tinggal masbro atur aja priority untuk port pointblank tsb. setting yg dijelaskan disana cukup bagus, tinggal menyesuaikan dengan ditempat masbro. kemungkinan kedua adalah, pada saat user online bersamaan, bandwidth yg tersedia tidak mencukupi, sehingga terjadi putus koneksi atau nge-lag. perlu dicatat bahwa alokasi bandwidth untuk queueing tidak dapat di set mentok dengan bandwidth yg disewa dari isp – misalnya telkom 3 mbps, maka idealnya 3 mbps ini dikurangi 15 – 20% sebagai jatah di queue.

  7. mas fazar terima kasih atas bantuannya…ternyata kesalahan squid lusca saya ada di squid.conf nya…dan skrg proxynya sdh HIT youtube….sudah jalan dengan mikrotik rb750g….saya memakai storeurl yang lama….
    sekali lagi TERIMA KASIH banyak….
    salam..nyong MANADO….

  8. mas mau tanya lagi …squid lusca saya telah running…tapi masalahnya kenapa video youtube yg telah di putar kedua kali sampai 5 kali tapi msh saja tcp miss…padahal saya sdh menggunakan storeurl terbaru dan telah install backward…kenapa bisa bagitu???…tolong bantuannya….

  9. {share} storeurl.pl terbaru by chuddy fernandez
    #!/usr/bin/perl
    # $Rev$
    # by chudy_fernandez@yahoo.com
    # Youtube updates at http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube/Discussion
    $|=1;
    while () {
    @X = split;
    $x = $X[0] . " ";
    $_ = $X[1];
    $u = $X[1];

    #profile.ak.fbcdn.net/hprofile-ak-xxx/xxx.jpg
    if (m/^http:\/\/profile.ak.fbcdn.net\/hprofile-ak-[a-z]{3}[0-9]\/(.*)/) {
    print $x . "http://profile.ak.fbcdn.net/hprofile-ak-cdn/" . $1 . "\n";

    #(xx.sphotos|photos-x).ak.fbcdn.net/hphotos-ak-xxx/
    } elsif (m/^http:\/\/.*?([a-z]{4,}).*?.ak.fbcdn.net\/([a-z]*)-ak-[a-z]{3}[0-9]\/([^&?]*)/) {
    print $x . "http://$1.ak.fbcdn.net/$2-ak-cdn/$3\n";

    #maps.google.com
    } elsif (m/^http:\/\/(khm|mt)[0-9]?(.google.com.*)/) {
    print $x . "http://" . $1 . $2 . "\n";

    #youtube/google video with range
    } elsif ($X[1] =~ /(youtube|google).*(videoplayback|liveplay)\?/){
    @itag = m/[&?](itag=[0-9]*)/;
    @id = m/[&?](id=[^\&]*)/;
    @range = m/[&?](range=[^\&]*)/;
    print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/@id&@itag@range\n";

    } elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
    print $x . "http://STOREURL.www.google-analytics.com/__utm.gif\n";

    #Cache High Latency Ads
    } elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
    $y = $3;$z = $2;
    for ($y) {
    s/pixel;.*/pixel/;
    s/activity;.*/activity/;
    s/(imgad[^&]*).*/\1/;
    s/;ord=[?0-9]*//;
    s/;&timestamp=[0-9]*//;
    s/[&?]correlator=[0-9]*//;
    s/&cookie=[^&]*//;
    s/&ga_hid=[^&]*//;
    s/&ga_vid=[^&]*//;
    s/&ga_sid=[^&]*//;
    # s/&prev_slotnames=[^&]*//
    # s/&u_his=[^&]*//;
    s/&dt=[^&]*//;
    s/&dtd=[^&]*//;
    s/&lmt=[^&]*//;
    s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
    s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
    s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
    s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
    s/[;&?]ord=[?0-9]*//;
    s/[;&]mpvid=[^&;]*//;
    s/&xpc=[^&]*//;
    # yieldmanager
    s/\?clickTag=[^&]*//;
    s/&u=[^&]*//;
    s/&slotname=[^&]*//;
    s/&page_slots=[^&]*//;
    }
    print $x . "http://STOREURL." . $1 . $2 . $y . "\n";

    #cache high latency ads
    } elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
    print $x . "http://STOREURL." . $1 . "/" . $2 . "\n";

    } elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
    print $x . "http://STOREURL." . $1 . "\n";

    #cdn, varialble 1st path
    } elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
    @y = ($1,$2,$4,$5);
    $y[0] =~ s/[a-z0-9]{2,5}/cdn./;
    print $x . "http://STOREURL." . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

    #rapidshare
    } elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
    print $x . "http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "\n";

    } elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
    print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";

    #domain/path/.*/path/filename
    } elsif (($u =~ /fucktube/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?[^\/]*\/[^\/]*)\/(.*)\/([^\/]*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
    @y = ($1,$2,$4,$5,$6);
    $y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
    print $x . "http://STOREURL." . $y[0] . $y[1] . "/" . $y[2] . "/" . $y[3] . "." . $y[4] . "\n";

    #like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
    #} elsif (($u =~ /tube8|pornhub|xvideos|hardsextube/) && (#m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
    # print $x . "http://cdn." . $4 . $6 . "\n";

    } elsif (m/^http:\/\/[^\/]*(tube8|pornhub|xvideos|hardsextube)[^\/]*\/(.*)\/([^\/]*)\.([^\/.?&_;]{2,4}).*$/) {
    print $x . "http://cdn.$1/$3.$4\n";

    #for yimg.com video
    } elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
    print $x . "http://cdn.yimg.com//" . $3 . "\n";

    #for yimg.com doubled
    } elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
    print $x . "http://cdn.yimg.com/" . $3 . "\n";

    #for yimg.com with &sig=
    } elsif (m/^http:\/\/([^\.]*)\.yimg\.com\/(.*)/) {
    @y = ($1,$2);
    $y[0] =~ s/[a-z]+([0-9]+)?/cdn/;
    $y[1] =~ s/&sig=.*//;
    print $x . "http://STOREURL." . $y[0] . ".yimg.com/" . $y[1] . "\n";

    #youjizz. We use only domain and filename
    } elsif (($u =~ /media[0-9]{1,5}\.youjizz/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?\.[^\/]*)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
    @y = ($1,$2,$4,$5);
    $y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
    print $x . "http://STOREURL." . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";

    #general purpose for cdn servers. add above your specific servers.
    } elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
    print $x . "http://squid-cdn-url//" . $2 . "." . $3 . "\n";

    # spicific extention
    # } elsif (m/^http:\/\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|swf).*?/) {
    # @y = ($1,$2);
    # $y[0] =~ s/((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
    # print $x . "http://" . $y[0] . "." . $y[1] . "\n";

    #generic http://variable.domain.com/path/filename."ex", "ext" or "exte"
    #http://cdn1-28.projectplaylist.com
    #http://s1sdlod041.bcst.cdn.s1s.yimg.com
    } elsif (m/^http:\/\/([^\/]*?)(\.[^\.\-]*?\.[^?&]*)\/([^.]*)\.([\w\d]{2,4})[?&]?.*/) {
    @y = ($1,$2,$3,$4);
    $y[0] =~ s/([a-z][0-9][a-z]dlod[\d]{3})|((cache|cdn)[-\d]*)|([a-zA-A]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
    print $x . "http://STOREURL.$y[0]$y[1]/$y[2].$y[3]\n";

    # all that ends with ;
    } elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
    print $x . "http://STOREURL." . $1 . "/" . $2 . "\n";

    } else {
    print $x . $_ . "\n";
    }
    }

Leave a Reply

Alamat email Anda tidak akan dipublikasikan.


*