Caching Dynamic Content dan Youtube Menggunakan Squid

ok deh mas fazar, untuk saat ini saya bypass dolo, jadi youtube tidak lewat proxy… entar kalo sudah ada yang sip, dipost ya mas storeurl.pl nya mas… ditungguin mas, nasib net saya ditangan anda :)

tidak ada cara lain untuk saat ini ya mas fazar untuk cache youtube??..

kalo saya pakai storeurl.pl ini :

#!/usr/bin/perl
# $Rev$
# by chudy_fernandez@yahoo.com
# Updates at http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube/Discussion
$|=1;
while (<>) {
@X = split;
# $X[1] =~ s/&sig=.*//;
$x = $X[0] . ” “;
$_ = $X[1];
$u = $X[1];

# Speedtest
#if (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
# print $x . “http://www.speedtest.net.SQUIDINTERNAL/speedtest/” . $2 . “\n”;

# Mediafire
#} els
if (m/^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*)/) {
print $x . “http://www.mediafire.com.SQUIDINTERNAL/” . $1 .”/” . $2 . “\n”;

# Fileserve
} elsif (m/^http:\/\/fs\w*\.fileserve\.com\/file\/(\w*)\/[\w-]*\.\/(.*)/) {
print $x . “http://www.fileserve.com.SQUIDINTERNAL/” . $1 . “./” . $2 . “\n”;

# Filesonic
} elsif (m/^http:\/\/s[0-9]*\.filesonic\.com\/download\/([0-9]*)\/(.*)/) {
print $x . “http://www.filesonic.com.SQUIDINTERNAL/” . $1 . “\n”;

# 4shared
} elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/download\/(.*)\/(.*\..*)\?.*/) {
print $x . “http://www.4shared.com.SQUIDINTERNAL/download/$2\/$3\n”;

# 4shared preview
} elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/img\/(\d*)\/\w*\/dlink__2Fdownload_2F(\w*)_3Ftsid_3D[\w-]*\/preview\.mp3\?sId=\w*/) {
print $x . “http://www.4shared.com.SQUIDINTERNAL/$2\n”;

# Photos-X.ak.fbcdn.net where X a-z
} elsif (m/^http:\/\/photos-[a-z](\.ak\.fbcdn\.net)(\/.*\/)(.*\.jpg)/) {
print $x . “http://photos” . $1 . “/” . $2 . $3 . “\n”;

# YX.sphotos.ak.fbcdn.net where X 1-9, Y a-z
} elsif (m/^http:\/\/[a-z][0-9]\.sphotos\.ak\.fbcdn\.net\/(.*)\/(.*)/) {
print $x . “http://photos.ak.fbcdn.net/” . $1 .”/”. $2 . “\n”;

# Maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
print $x . “http://” . $1 . $2 . “\n”;

# Compatibility for old cached get_video?video_id
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
$z = $2; $z =~ s/video_id=/get_video?video_id=/;
print $x . “http://video-srv.youtube.com.SQUIDINTERNAL/” . $z . “\n”;

# Youtube fix
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/videoplayback\?(.*)/) {
$p_str = $2;
$tag = “”;
$alg = “”;
$id = “”;
$range = “”;
if ($p_str =~ m/(itag=[0-9]*)/){$tag = “&”.$1}
if ($p_str =~ m/(algorithm=[a-z]*\-[a-z]*)/){$alg = “&”.$1}
if ($p_str =~ m/(id=[a-zA-Z0-9]*)/){$id = “&”.$1}
if ($p_str =~ m/(range=[0-9\-]*)/){$range = “&”.$1; $range =~ s/-//; $range =~ s/range=//; }
print $x . “http://video-srv.youtube.com.SQUIDINTERNAL/” . $tag . “&” . $alg . “&” . $id . “&” . $range . “\n”;

} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
print $x . “http://www.google-analytics.com/__utm.gif\n”;

# Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
print $x . “http://” . $1 . $2 . $y . “\n”;

# Cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
print $x . “http://” . $1 . “/” . $2 . “\n”;

# spicific servers starts here….
} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
print $x . “http://” . $1 . “\n”;

# cdn, varialble 1st path
} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . “http://” . $y[0] . $y[1] . “/” . $y[2] . “.” . $y[3] . “\n”;

# Rapidshare
} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
print $x . “http://cdn.” . $3 . “/SQUIDINTERNAL/” . $5 . “\n”;

} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . “http://” . $1 . “/SQUIDINTERNAL/” . $3 . “\n”;

# like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . “http://cdn.” . $4 . $6 . “\n”;
#…spicific servers end here.

# Photos-X.ak.fbcdn.net where X a-z
} elsif (m/^http:\/\/photos-[a-z].ak.fbcdn.net\/(.*)/) {
print $x . “http://photos.ak.fbcdn.net/” . $1 . “\n”;

# For yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
print $x . “http://cdn.yimg.com//” . $3 . “\n”;

# For yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
print $x . “http://cdn.yimg.com/” . $3 . “\n”;

# For yimg.com with &sig=
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+[0-9]+/cdn/;
$y[1] =~ s/&sig=.*//;
print $x . “http://” . $y[0] . “.yimg.com/” . $y[1] . “\n”;

# Youjizz. We use only domain and filename
} elsif (($u =~ /media[0-9]{2,5}\.youjizz/) && (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})((\?|\%).*)?$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
print $x . “http://” . $y[0] . $y[1] . “/” . $y[2] . “.” . $y[3] . “\n”;

# General purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
print $x . “http://squid-cdn-url//” . $2 . “.” . $3 . “\n”;

# Generic http://variable.domain.com/path/filename.”ex” “ext” or “exte” with or withour “? or %”
} elsif (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\.([^\/\?\&]{2,4})((\?|\%).*)?$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
print $x . “http://” . $y[0] . $y[1] . “/” . $y[2] . “.” . $y[3] . “\n”;

# generic http://variable.domain.com/…
} elsif (m/^http:\/\/(([A-Za-z]+[0-9-]+)*?|.*cdn.*|.*cache.*)\.(.*?)\.(.*?)\/(.*)$/) {
print $x . “http://cdn.” . $3 . “.” . $4 . “/” . $5 . “\n”;

# spicific extention that ends with ?
} elsif (m/^http:\/\/(.*?)\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|on2)(.*)/) {
print $x . “http://” . $1 . “/” . $2 . “.” . $3 . “\n”;

# all that ends with ;
} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
print $x . “http://” . $1 . “/” . $2 . “\n”;

} else {
print $x . $_ . “sucks\n”;
}
}

bisa tercache tapi videonya aneh2 mas fazar, contohnya saya buka lagu armada hargai aq yang sudah tercache.. entar yang muncul mala someone like… truss aq coba dikompi lain untuk buka lagu numb linkinpark, truss keluarnya juga someone like…. jadi stresss deh

mas fazar… lusca FMi sama lusca head, bagus yang mana ya mas?, sekarang saya pakai lusca FMI dan buat cache youtube dengan storeurl.pl yang mas pakai, sama sekali tidak mau tercache youtubenya…

cache youtube range masih work gak mas fazar?

byk kasus buka video yg berbeda tapi yg hit video itu2 aja

mas fazar…makasih tutsnya…lusca saya udah jalan n sudah berhsil cache video nya…

saya skrg punya maslah baru mas…udah oprak-oprek-oprik ga jadi2

saya punya topologi :

internet (2 line)–>mikrotik (router+LB)–>ubuntu+lusca (bridge)–>switch–>client

|

AP

saya nambah access point di switch dan sudah seting hotspot dari mikrotik n ip binding buat ip lokal biar ip lokal terbypass..koq client dhcp buat hotspot smuanya juga terbypass…ga ada yg kena login pagenya mikrotik mas…kira2 solusinya gmana mas???

tolong dibantu mas…mkasih..

wah thanks mas, ternyata salah di gateway.. tapi agak aneh mas, ketika saya buka situs detik tanggal dan artikel di situs tidak terupdate (‘:cry:’) . masih sama seperti hari berikutnya. kira kira salah dimananya ya mas.. termiakasih (‘:grin:’)

Mass fajar saya mau tanya, saya udah kongfigurasi Lusca_HEAD_r14809.
Proxy udah berhasil nge cache video youtube dan situs situs lain.. tapi kenapa kalo di tes pake PC lain malah narik ulang dari situs yang di tuju ya.. gk ambil dari proxynya dulu..

Metode tes:

#Proxy Server

eth0: 10.20.30.55 (public/inet)

eth1: 192.168.10.99 (client)#Client PC

1ip 192.168.10.88

gateway 10.20.30.55

#Client PC2

ip 192.168.10.77

gateway 10.20.30.55

#partisi cache di proxyada 5 partisi cache, masing masing partisi 25Gb
Pas tes di PC 1 Inet saya loss di mikrotik biar load video Utube nya cepet.. udah selesai, saya limit ip 10.20.30.55 di mikrotik jd 16Kbps. Cache and cookies dibrowser saya bersihin.. Alhasil Video berhasil ke cache, begitu juga situs lain.. :-)

Test di PC2 dengan metode seperti di atas. buka situs yg udah di buka di PC1 alhasil gk ke cache sama sekali..
ada salah dimana ya kira kira mass fazar.
Terimakasih sebelumnya

Semuanya polos masbro fazar…tidak pakai mikrotik.

Masbro fazar, saya pernah pake SQUID bukan lusca kalo download file yg sudah ter-cache speednya sekitar 6 MB/sec tapi kok di SQUID LUSCA download file yg sudah ter-cache speed nya cuma sekitar 3 MB/sec? apa ada conf yg perlu ditambah?

Makasih masbro fazar…setelah dicermati ternyata storeurl saya ada sedikit kesalahan, sekarang sudah ok dan berjalan dengan lancar.