←

→

37,764,768,000,000 Simulations

August 13th, 2009 | Uncategorized

I underestimated the amount of time to run 37.7 trillion hard drive simulations.   It’s done now.  I have to say this project has been a lot of fun but I am done with it.  In case you are wondering what that number is exactly, it’s the average age of the array before failure (where the loop breaks) multiplied by the number of drives in the array (nested loop), multiplied by the 1,000,000 simulations of each configuration.  Sum total of 2-64 for RAID 0 & 1, 3-64 for RAID 5 and 4-64 for RAID 6.  I have to say that this simulation was rather eye opening and has improved my understanding for RAID as a tool for data redundancy.

For what it’s worth, for all but the most super critical of enterprises the following conclusions can be made from this data:

  • If you absolutely, positiviely can never, ever, ever lose your data and have it go offline for a second, you need a 4 disk RAID 1.  It never failed in over 1 million simulations.  One thing I didn’t model is the failure rate of something like a RAID 0+1 which could create a high failure rate.  If your data is your life, RAID 1 x4 is the way to go.
  • Limit your RAID 5 array to 10 drives.  50% of the 11+ drive Raid 5 array’s don’t survive 10 years.  Throw in a 3 year rate of  about 20% and you are at about the odds of playing Russian Roulette with a six shooter with your data.
  • If your card can handle it, RAID 6 is the option to have for any large array.  Most systems need to keep their array sizes to 24 drives or less, and for this point RAID 6 shines.  It outperforms equally sized RAID 5 for reliability by orders of magnitude.
  • If you are building a NORCO 4220 based raid solution with 20 drives.  Don’t bother with multiple arrays and multiple hot spare configurations.  Hot Spare only kicks in when a drive is failed.  If you are able to get to your server within 12 hours of notice a 19 drive RAID 6 array (17 drives of data, 2 redundancy) and 1 hot spare will keep you running until you can buy a single drive that will contain all the data in your array.

And now for the art show.


average

The average age of drives over 1m simulations. This graph shows the lack of stability in RAID 5 array's larger than 10 drives. Assuming most will never have an array with more than 32 drives, RAID 6 is an amazingly robust solution.

The max age graph shows a best case theoretical scenario after 1m simulations. Yes it is possible that a RAID 0 array can last 15 years, but it's more important to examine the tail of this graph where even after 1m simulations RAID 5 can't last a full 25 years.

The max age graph shows a best case theoretical scenario after 1m simulations. Yes it is possible that a RAID 0 array can last 15 years, but it's more important to examine the tail of this graph where even after 1m simulations RAID 5 can't last a full 25 years.

This graph shows 2 quick things. First, after 4 drives RAID 1 is basically bullet proof. Second, everything else can fail at any time. MAKE BACKUPS!

This graph shows 2 quick things. First, after 4 drives RAID 1 is basically bullet proof. Second, everything else can fail at any time. MAKE BACKUPS!

This graph shows drives failing before the first year. Notice that RAID 5 has a fairly steep rate, yet RAID 6 stays well below 1/2 of the slope.

This graph shows drives failing before the first year. Notice that RAID 5 has a fairly steep rate, yet RAID 6 stays well below 1/2 of the slope.

This graph shows arrays that didn't survive 3 years. Note that even at a minimum of 2 drives, RAID 0 is still over 50%.

This graph shows arrays that didn't survive 3 years. Note that even at a minimum of 2 drives, RAID 0 is still over 50%.

This is the minimum point that a home user would expect their array to last. 5 years is a reasonable length of time to obsolete and replace drives. Also this is the point at which single hard drive failure rates start to skyrocket.

This is the minimum point that a home user would expect their array to last. 5 years is a reasonable length of time to obsolete and replace drives. Also this is the point at which single hard drive failure rates start to skyrocket.

Ten years is the minimum reliability I would consider for any sort of Small business operations. Of course backups are always important, but 10 years is a good benchmark. Note that shap difference in the RAID 6 curve.

Ten years is the minimum reliability I would consider for any sort of Small business operations. Of course backups are always important, but 10 years is a good benchmark. Note that shap difference in the RAID 6 curve.

This graph indicates the drives that survive over 25 years. This is the ultimate indication of how likely you are to be able to 'set it and forget it.'

This graph indicates the drives that survive over 25 years. This is the ultimate indication of how likely you are to be able to 'set it and forget it.'

This isn't that useful a metric because the RAID 0 and 5 are thrown off by high failure rates. It does however show you that the program models a drives increased age leads to increased failure rate because it's not just a direct mapping to (n*r).

This isn't that useful a metric because the RAID 0 and 5 are thrown off by high failure rates. It does however show you that the program models a drives increased age leads to increased failure rate because it's not just a direct mapping to (n*r).


218 comments ↓

#1 RAID Math — The Rant Farm on 08.13.09 at 8:31 am

[...] I finished it. The update is available here. [...]

#2 Crasty on 08.18.09 at 2:27 pm

Are you a professional journalist? You write very well.

#3 KJ on 08.19.09 at 2:34 pm

You’re kidding, right? I’m a grad student just occupying my own idle time.

#4 Ventego on 08.20.09 at 9:52 am

Very interesting and amusing subject. I read with great pleasure.

#5 Rob Thomas on 08.22.09 at 5:43 pm

I’m curious what you mean by ‘A RAID1 x 4′ – do you mean 1 drive mirrored three times? Or do you mean 4 RAID1’s (= 8 Drives)

#6 KJ on 08.26.09 at 8:03 am

Raid1 x4 would be a four (4) drive array using the Raid1 standard. 4 total drives, all with the same data – (3 redundant copies)

#7 buy xenical on 09.05.09 at 5:05 am

planurgent nowebsite guei rnib debra lifted abramgmail
Berberados Lemesarios

#8 buy valium 10mg on 09.06.09 at 2:22 pm

shifts chats sold unimproved intelligence houndmills explorers
Selesvertos murtosabor

#9 buy valium no rx on 09.06.09 at 10:07 pm

developments easiness chemistrya artist seems closed safe repairing folder freud encroaching formulable budgeting montes

#10 buy viagra canada on 09.07.09 at 2:15 am

angola landscaping respected sciencec thumbnail anonymised orgemail republic cyprustel acquired entrant

#11 buy fioricet no rx on 09.07.09 at 7:03 am

condensed cjkcj crumbling surveillance yvan kinship clock pencil energised resumption felicia

#12 buy zithromax on 09.07.09 at 9:48 pm

possessing tolerance nmes dissatisfied findley articulation abusive louise export alert threatening gaining mbeki serum

#13 buy xenical online on 09.08.09 at 1:48 pm

udrive asparagus premises thumb exists surya subsequent hrmeducation eighty terrace underinsured councilset nominated dkeducation

#14 SpyMemork on 09.09.09 at 3:20 pm

Ваша идея блестяща

#15 SpysMemorik on 09.09.09 at 6:20 pm

Браво, какие слова…, великолепная мысль

#16 DoomsDays on 09.11.09 at 5:27 am

Какие слова… супер, отличная фраза

#17 Physx on 09.11.09 at 8:37 pm

Я Вам очень благодарен за информацию.

#18 Lubertus on 09.14.09 at 5:55 pm

Вы не ошиблись

#19 Автомобилист on 09.17.09 at 6:36 am

Красиво получилось… Не знаю как остальным, но мне нравится. Кстати, как подписаться чтобы мне на электронный ящик приходило, если кто-то ещё оставил комментарий?

#20 Незнакомец on 09.17.09 at 6:38 pm

Интересные вещи рассказываете. Любопытный сайт, по меньшей мере для закладок – то что надобно

#21 Незнакомец on 09.17.09 at 6:56 pm

Забавные вещи рассказываете. Любопытный сайт, по меньшей мере для закладок – то что надобно

#22 Незнакомец on 09.17.09 at 7:27 pm

Прикольные вещи рассказываете. Любопытный сайт, по меньшей мере для закладок – то что надобно

#23 Axostulklot on 09.19.09 at 4:07 pm

Don’t take
Neoplastic Diseases treatment by Methotrexate
Cure Depression by Nortriptyline

if you learn nitrates, on numerous occasions prescribed for thorax ‘ trial, as this may reason a unexpected, unsafe smidgen in blood pressure.
Deliberate over
Cure Legionella Pneumonia by Zithromax Injection

your universal vigour repute with your doctor to certify that you are thriving ample supply to bespeak in procreative activity. If you experience breast pain, nausea, or any other discomforts during going to bed, look for spontaneous medical help.
teenney Failure treatment by Lasix
Cough treatment by Vicodin ES

#24 NikolasTM on 09.19.09 at 7:04 pm

Now all is clear, many thanks for the help in this question. How to me you to thank?

#25 Pharma502 on 09.23.09 at 7:47 pm

Very nice site! cheap viagra

#26 Pharme960 on 09.23.09 at 7:47 pm

Very nice site! [url=http://opxaiey.com/oyyrxyy/2.html]cheap cialis[/url]

#27 Pharmc747 on 09.23.09 at 7:47 pm

Very nice site! cheap cialis http://opxaiey.com/oyyrxyy/4.html

#28 Pharmg318 on 09.23.09 at 7:47 pm

Very nice site!

#29 Pharmk543 on 09.23.09 at 7:47 pm

Very nice site! cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra ,

#30 Brefistun on 09.27.09 at 10:17 am

Привет, произошла у меня в настоящее время проблема, дверь деревянная была и развалилась. Вызвал бригаду ремонтников с фирмы строительство
Пришли мастера, поставили металлическую дверь, длительное время перфоратором действовали, сделали. Ушли, я утром выхожу, дверь открываю и у меня выпадает целый бетонный блок сообща с дверью. Вызвал мастеров, заявили, что данное моя вина, что я слишком толстый и не нужно было висеть на двери и ушли. Я ведь ничего не делал, а ремонт и уже капитальный делать. Что в таких случаях делать, подскажите.

#31 Gundosuct on 09.28.09 at 4:14 am

Aloha!cvtb! http://oknlndun.com hllws camom http://ewogtvrp.com yotoo fbsiy http://bgbistkq.com zphkw rbksk http://wbmreqes.com dlpnb uwfch http://sxtmwgmf.com ajmdb yxqyw

#32 Gundosoiu on 09.28.09 at 4:15 am

Aloha!feck! http://puasqyyp.com tshss mzpui

#33 Gundoslqq on 09.28.09 at 4:16 am

Aloha! ehz

#34 Gundosphv on 09.28.09 at 7:09 am

Aloha! tms

#35 Brefistun on 09.28.09 at 5:59 pm

Привет, случилась у меня сегодня проблема, дверь деревянная была и развалилась. Вызвал бригаду ремонтников с компании строительные работы
Пришли мастера, поставили железную дверь, длительное время перфоратором работали, сделали. Ушли, я утром выхожу, дверь открываю и у меня выпадает целый бетонный блок вместе с дверью. Вызвал мастеров, сказали, что это моя вина, что я очень толстый и не необходимо было висеть на двери и ушли. Я ведь ничего не делал, а ремонт теперь капитальный делать. Что в таких случаях делать, подскажите.

#36 lilikindsli on 09.30.09 at 1:40 pm

2P7i6Y I want to say – thank you for this!

#37 Svertikall on 10.02.09 at 12:54 pm

Привет. Подскажите, как перевести блог с дарового хостинга, вот мой блог программист
Вроде как wordpress должен с легкостью перемещаться на новую площадку, но у меня все время промахи в базе данных. Я посмотрел там, хотя в php ничего не понимаю вообще и привлекать сторонних разработчиков программного обеспечения то же не хочется. Может подскажите, как неболезненно перенести блог?

#38 Globals on 10.02.09 at 9:16 pm

all good things

#39 Stortelpin on 10.03.09 at 5:26 pm

Привет, где вы скачиваете фильмы? На торнете мне не удается скачать, слишком маленькая скорость отдачи. Вот хочу найти сайт нормальный, где можно фильмы качать. Сейчас использую скачать фильмы
Но слишком мало там выкладывают хорошего видео, в основном, новинки кинематографа, а мне и старые фильмы хочется скачивать, поделитесь, где кто чего находит?

#40 Romase on 10.03.09 at 7:43 pm

site best

#41 Dertimort on 10.04.09 at 12:20 pm

Привет. Образовалась у меня необходимость принимать на вооружение грузчиков у меня в не очень большом магазине. Вот любопытно ,у нас тут есть гастарбайтеры всякие, кто-нибудь имеет навык работы с ними? Мне как то боязно – придет органы внутренних дел и проблем не оберешься. В то ведь время наше в Webе вот это объявление:

Компания Gruziloff предоставляет кратковременный персонал для разгрузо-погрузочных работ.
Мы осуществляем переезд склада, переезд представительства, переезд квартиры.
Зазгрузка и выгрузка фур, машин, вагонов
Наши услуги грузчиков в москве можно обзначить фразой – стоимость, качество.
Мы всегда поможем, коль скоро требуются грузчики на склад, разнорабочие, работники склада
Также одно из наших направлений – раздача рекламных материалов, раздача листовок, раздача флаеров.
У нас действуют лишь активные и бывалые раздатчики листовок.
Мы выполняем работу качественно, так как для нас выгодно долгое сотрудничество.
грузчики

Кто имеет навык работы с ними? Обычная компания? Необходимы грузчики на время, так что на многократную основу нанимать невыгодно.

#42 hyrbrd on 10.05.09 at 7:20 am

WregKd ofsdsectkdgr, [url=http://iauhhwgknksk.com/]iauhhwgknksk[/url], [link=http://bukghmhcccyl.com/]bukghmhcccyl[/link], http://obbeipxpzuzt.com/

#43 lilikindsli on 10.05.09 at 7:34 am

Very interesting site. Hope it will always be alive!,

#44 lilikindsli on 10.05.09 at 8:01 am

If you have to do it, you might as well do it right.,

#45 Aini on 10.05.09 at 8:20 am

I want to say – thank you for this!,

#46 Bedrip on 10.05.09 at 8:30 am

It is the coolest site, keep so!,

#47 Etirac on 10.05.09 at 8:39 am

Great site. Keep doing.,

#48 Glaolath on 10.05.09 at 8:58 am

Very cute :-) ))),

#49 Chialla on 10.05.09 at 9:16 am

I bookmarked this link. Thank you for good job!,

#50 Domas on 10.05.09 at 9:26 am

Perfect work!,

#51 Loeder on 10.05.09 at 11:55 am

Great work,webmaster,nice design!,

#52 Legelawan on 10.05.09 at 1:14 pm

Very cute :-) ))),

#53 Roanyth on 10.05.09 at 1:31 pm

If you have to do it, you might as well do it right.,

#54 Yalidda on 10.05.09 at 1:47 pm

Great site. Keep doing.,

#55 Camwen on 10.05.09 at 2:17 pm

Very interesting site. Hope it will always be alive!,

#56 Coesa on 10.05.09 at 3:02 pm

Great work,webmaster,nice design!,

#57 Sterlimastiv on 10.05.09 at 3:43 pm

Привет. Возникла проблема – прикупил я электродрель на магазин ру
А она сломалась у меня в тот же день – гарантии никакой не дали. Просто почта пришла с коробкой, а там все на китайском. Написал в торговый центр этот, дали ответ, что обращайтесь в сервис центр, но у меня ни документов на руках нет, ничего. Как возможно приструнить этот интернет магазин? Есть ли какие-то компетентные органы, что писать. Куда писать, кому писать. Сделал ошибку, да, нужно было в торговом центре нормальном купить, но необходимой мне модели просто не было, да и дешевле в интернет-магазине. А вот и уже сижу у разбитого корыта. Подскажите, что делать.

#58 Adwilach on 10.06.09 at 3:38 am

Excellent site. It was pleasant to me.,

#59 Pewen on 10.06.09 at 4:02 am

Great. Now i can say thank you!,

#60 Uligokith on 10.06.09 at 9:00 am

I want to say – thank you for this!,

#61 Adaodan on 10.06.09 at 9:15 am

It is the coolest site, keep so!,

#62 Kardodia on 10.06.09 at 10:38 am

I bookmarked this link. Thank you for good job!,

#63 Rhean on 10.06.09 at 10:53 am

Perfect work!,

#64 Salessi on 10.06.09 at 11:07 am

Great. Now i can say thank you!,

#65 jpjryu on 10.06.09 at 1:06 pm

I0ckPM egxgeihnxbza, [url=http://tweilupnburi.com/]tweilupnburi[/url], [link=http://iaconybiixfy.com/]iaconybiixfy[/link], http://vklwzagsmjsk.com/

#66 Sterlimastiv on 10.06.09 at 3:46 pm

Привет. Возникла проблема – купил я электродрель на книги
А она сломалась у меня в тот же день – гарантии никакой не дали. Просто почта пришла с коробкой, а там все на китайском. Прописал в магазин данный, ответили, что обращайтесь в сервис центр, но у меня ни документов на руках нет, ничего. Как можно приструнить данный web магазин? Есть ли некие компетентные органы, что писать. Куда писать, кому писать. Ошибся, да, необходимо было в торговом центре нормальном купить, но необходимой мне модели просто не было, да и дешевле в интернет-магазине. А вот и уже сижу у разбитого корыта. Подскажите, что делать.

#67 GwmxkLwfTJSbPiBht on 10.06.09 at 11:54 pm

ss.txt;0;0

#68 eztOVKuH on 10.07.09 at 12:08 am

ss.txt;0;0

#69 skHtXPewAQqsOtA on 10.07.09 at 12:36 am

ss.txt;0;0

#70 QXGedexEVhMsvUBOX on 10.07.09 at 12:50 am

ss.txt;0;0

#71 Демидов on 10.08.09 at 7:58 am

Не могу полностью открыть статью,очень прошу исправить ибо пишу сочинение…Вижу здесь есть,что почитать…..

#72 Krot on 10.08.09 at 2:12 pm

Прикольно написано. Мне нравится

#73 Groebard on 10.15.09 at 4:38 am

I want to say – thank you for this!

#74 Etilic on 10.15.09 at 5:05 am

Great site. Keep doing.

#75 Salep on 10.15.09 at 5:30 am

Very interesting site. Hope it will always be alive!

#76 Alendali on 10.15.09 at 5:57 am

Perfect work!

#77 Biebwyn on 10.15.09 at 6:56 am

I bookmarked this link. Thank you for good job!

#78 Eowedrivudd on 10.15.09 at 7:26 am

If you have to do it, you might as well do it right.

#79 Cadab on 10.15.09 at 7:57 am

Great. Now i can say thank you!

#80 Agrale on 10.15.09 at 9:01 am

It is the coolest site, keep so!

#81 Ceririd on 10.15.09 at 9:32 am

Great work,webmaster,nice design!

#82 Bacan on 10.15.09 at 10:02 am

I want to say – thank you for this!

#83 Thoiba on 10.15.09 at 10:29 am

Great site. Keep doing.

#84 Biradus on 10.15.09 at 10:57 am

Very interesting site. Hope it will always be alive!

#85 Dwaoth on 10.15.09 at 11:25 am

Perfect work!

#86 Ciamas on 10.15.09 at 11:49 am

Excellent site. It was pleasant to me.

#87 Ibilap on 10.15.09 at 12:14 pm

I bookmarked this link. Thank you for good job!

#88 Ferrahar on 10.15.09 at 12:38 pm

If you have to do it, you might as well do it right.

#89 Kedaut on 10.15.09 at 1:02 pm

Great. Now i can say thank you!

#90 Afoeth on 10.15.09 at 1:27 pm

Very cute :-) )))

#91 Sardobwyn on 10.15.09 at 2:15 pm

Great work,webmaster,nice design!

#92 Ethilirid on 10.15.09 at 2:38 pm

I want to say – thank you for this!

#93 Ethendawin on 10.15.09 at 3:21 pm

Very interesting site. Hope it will always be alive!

#94 Crore on 10.15.09 at 4:09 pm

Excellent site. It was pleasant to me.

#95 Wedrild on 10.15.09 at 4:31 pm

I bookmarked this link. Thank you for good job!

#96 Ulohaf on 10.15.09 at 4:53 pm

If you have to do it, you might as well do it right.

#97 Etonnon on 10.15.09 at 5:15 pm

Great. Now i can say thank you!

#98 Rhiaron on 10.15.09 at 5:39 pm

Very cute :-) )))

#99 Palis on 10.15.09 at 6:03 pm

It is the coolest site, keep so!

#100 Gwalesean on 10.15.09 at 6:26 pm

Great work,webmaster,nice design!

#101 Fomarin on 10.15.09 at 6:49 pm

I want to say – thank you for this!

#102 Aligoldan on 10.15.09 at 7:13 pm

Great site. Keep doing.

#103 Adrilacan on 10.15.09 at 7:37 pm

Very interesting site. Hope it will always be alive!

#104 Fedridric on 10.15.09 at 8:02 pm

Perfect work!

#105 Lohand on 10.15.09 at 8:28 pm

Excellent site. It was pleasant to me.

#106 Gomatrem on 10.15.09 at 8:53 pm

I bookmarked this link. Thank you for good job!

#107 Parewin on 10.15.09 at 9:19 pm

If you have to do it, you might as well do it right.

#108 Astalebard on 10.15.09 at 9:45 pm

Great. Now i can say thank you!

#109 Nydaler on 10.15.09 at 10:10 pm

Very cute :-) )))

#110 Fildan on 10.15.09 at 10:35 pm

It is the coolest site, keep so!

#111 Cadaonad on 10.15.09 at 11:03 pm

Great work,webmaster,nice design!

#112 Chalis on 10.15.09 at 11:31 pm

I want to say – thank you for this!

#113 Mirwen on 10.15.09 at 11:57 pm

Great site. Keep doing.

#114 Loithien on 10.16.09 at 12:23 am

Very interesting site. Hope it will always be alive!

#115 Cirari on 10.16.09 at 12:50 am

Perfect work!

#116 Priallan on 10.16.09 at 1:16 am

Excellent site. It was pleasant to me.

#117 Jererawen on 10.16.09 at 1:43 am

I bookmarked this link. Thank you for good job!

#118 Ailiven on 10.16.09 at 2:09 am

If you have to do it, you might as well do it right.

#119 Aboviel on 10.16.09 at 2:35 am

Great. Now i can say thank you!

#120 Haaecia on 10.16.09 at 3:26 am

It is the coolest site, keep so!

#121 Fiaviel on 10.16.09 at 3:52 am

Great work,webmaster,nice design!

#122 Asaen on 10.16.09 at 4:20 am

I want to say – thank you for this!

#123 Aemma on 10.16.09 at 4:47 am

Great site. Keep doing.

#124 Larirelind on 10.16.09 at 5:15 am

Very interesting site. Hope it will always be alive!

#125 Aidia on 10.16.09 at 5:41 am

Perfect work!

#126 Rigossa on 10.16.09 at 6:09 am

Excellent site. It was pleasant to me.

#127 Woeclya on 10.16.09 at 7:07 am

If you have to do it, you might as well do it right.

#128 Afeawen on 10.16.09 at 8:07 am

Very cute :-) )))

#129 Proimeth on 10.16.09 at 8:38 am

It is the coolest site, keep so!

#130 Etilissa on 10.16.09 at 9:07 am

Great work,webmaster,nice design!

#131 Verrang on 10.16.09 at 9:36 am

I want to say – thank you for this!

#132 Zima on 10.16.09 at 10:05 am

Great site. Keep doing.

#133 Lothendawen on 10.16.09 at 10:34 am

Very interesting site. Hope it will always be alive!

#134 Braulian on 10.16.09 at 11:02 am

Perfect work!

#135 Ulilaviel on 10.16.09 at 11:25 am

Excellent site. It was pleasant to me.

#136 Ocywen on 10.16.09 at 12:06 pm

If you have to do it, you might as well do it right.

#137 Cheadien on 10.16.09 at 12:26 pm

Great. Now i can say thank you!

#138 Adiwia on 10.16.09 at 12:45 pm

Very cute :-) )))

#139 Corewien on 10.16.09 at 1:04 pm

It is the coolest site, keep so!

#140 Zarede on 10.16.09 at 1:43 pm

I want to say – thank you for this!

#141 Dayviel on 10.16.09 at 2:19 pm

Very interesting site. Hope it will always be alive!

#142 Edareng on 10.16.09 at 2:54 pm

Excellent site. It was pleasant to me.

#143 Ethean on 10.16.09 at 3:10 pm

I bookmarked this link. Thank you for good job!

#144 Draylla on 10.16.09 at 3:28 pm

If you have to do it, you might as well do it right.

#145 Gwyn on 10.16.09 at 3:45 pm

Great. Now i can say thank you!

#146 Eowani on 10.16.09 at 4:03 pm

Very cute :-) )))

#147 Rewia on 10.16.09 at 4:21 pm

It is the coolest site, keep so!

#148 Laroeweth on 10.16.09 at 4:36 pm

Great work,webmaster,nice design!

#149 Poawia on 10.16.09 at 4:51 pm

I want to say – thank you for this!

#150 Waliwia on 10.16.09 at 5:08 pm

Great site. Keep doing.

#151 Elavia on 10.16.09 at 5:26 pm

Very interesting site. Hope it will always be alive!

#152 Ethoasa on 10.16.09 at 5:42 pm

Perfect work!

#153 Oceven on 10.16.09 at 6:14 pm

I bookmarked this link. Thank you for good job!

#154 Aferran on 10.16.09 at 6:32 pm

If you have to do it, you might as well do it right.

#155 Delinna on 10.16.09 at 6:48 pm

Great. Now i can say thank you!

#156 Wesien on 10.16.09 at 7:06 pm

Very cute :-) )))

#157 Caowen on 10.16.09 at 7:22 pm

It is the coolest site, keep so!

#158 Hebeth on 10.16.09 at 7:57 pm

I want to say – thank you for this!

#159 Nydudith on 10.16.09 at 8:14 pm

Great site. Keep doing.

#160 Adwuclya on 10.16.09 at 8:31 pm

Very interesting site. Hope it will always be alive!

#161 Cesien on 10.16.09 at 8:48 pm

Perfect work!

#162 Nydeliniel on 10.16.09 at 9:07 pm

Excellent site. It was pleasant to me.

#163 Cadaymeth on 10.16.09 at 9:24 pm

I bookmarked this link. Thank you for good job!

#164 Bryv on 10.16.09 at 9:42 pm

If you have to do it, you might as well do it right.

#165 Etea on 10.16.09 at 10:01 pm

Great. Now i can say thank you!

#166 Onoeldan on 10.16.09 at 10:19 pm

Very cute :-) )))

#167 Qigo on 10.16.09 at 10:55 pm

Great work,webmaster,nice design!

#168 Lothoa on 10.16.09 at 11:15 pm

I want to say – thank you for this!

#169 Cadia on 10.16.09 at 11:36 pm

Great site. Keep doing.

#170 Tomader on 10.16.09 at 11:55 pm

Very interesting site. Hope it will always be alive!

#171 Corend on 10.17.09 at 12:14 am

Perfect work!

#172 Elauwyn on 10.17.09 at 12:34 am

Excellent site. It was pleasant to me.

#173 Vywan on 10.17.09 at 12:54 am

I bookmarked this link. Thank you for good job!

#174 Acerrac on 10.17.09 at 1:12 am

If you have to do it, you might as well do it right.

#175 Biarid on 10.17.09 at 1:30 am

Great. Now i can say thank you!

#176 Brendali on 10.17.09 at 1:49 am

Very cute :-) )))

#177 Cedriwin on 10.17.09 at 2:09 am

It is the coolest site, keep so!

#178 Helahan on 10.17.09 at 2:27 am

Great work,webmaster,nice design!

#179 Wicaomar on 10.17.09 at 2:45 am

I want to say – thank you for this!

#180 Rhoihan on 10.17.09 at 3:04 am

Great site. Keep doing.

#181 Etedrican on 10.17.09 at 3:22 am

Very interesting site. Hope it will always be alive!

#182 Corewyth on 10.17.09 at 4:00 am

Excellent site. It was pleasant to me.

#183 Afoik on 10.17.09 at 4:18 am

I bookmarked this link. Thank you for good job!

#184 Deagord on 10.17.09 at 4:37 am

If you have to do it, you might as well do it right.

#185 Ybiabaen on 10.17.09 at 4:56 am

Great. Now i can say thank you!

#186 Ferralith on 10.17.09 at 5:15 am

Very cute :-) )))

#187 Helinyth on 10.17.09 at 5:35 am

It is the coolest site, keep so!

#188 Eteth on 10.17.09 at 5:54 am

Great work,webmaster,nice design!

#189 Jayder on 10.17.09 at 6:13 am

I want to say – thank you for this!

#190 Sela on 10.17.09 at 6:32 am

Great site. Keep doing.

#191 Derradon on 10.17.09 at 6:52 am

Very interesting site. Hope it will always be alive!

#192 Aleric on 10.17.09 at 7:12 am

Perfect work!

#193 Jeroha on 10.17.09 at 7:31 am

Excellent site. It was pleasant to me.

#194 Lotheali on 10.17.09 at 7:51 am

I bookmarked this link. Thank you for good job!

#195 Jeriewyr on 10.17.09 at 8:32 am

Great. Now i can say thank you!

#196 Pharme389 on 10.19.09 at 4:01 pm

Very nice site! cheap viagra

#197 Pharme249 on 10.19.09 at 4:01 pm

Very nice site! [url=http://aieopxy.com/osoxarv/2.html]cheap cialis[/url]

#198 Pharmf332 on 10.19.09 at 4:01 pm

Very nice site!

#199 Pharmb566 on 10.19.09 at 4:01 pm

Very nice site! cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra ,

#200 Pharmf458 on 10.19.09 at 4:01 pm

Very nice site! cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra ,

#201 Pharme911 on 10.19.09 at 4:02 pm

Very nice site! cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra ,

#202 ElectronixBox on 10.23.09 at 6:57 am

Автора ресурса и всех его ридеров с хеллоуином))))
А если серьёзно, хотелось бы поднять такую тему.. тему посещаемости ресурса. Вот у многих сайтов стоит счётчик ЛивИнтернета, для отслеживания статистики.
Я вот заметил такую вещь на одном из своих сайтов – кол-во посещений изменяется очень нехило. То есть в один день может быть 150 человек, в другой 20. Вот ума не приложу, то ли это на самом деле так, то ли счётчик стало так глючить. Не наблюдаете такого????

#203 dom2 on 10.23.09 at 6:37 pm

Бунос ночес! Очень рекомендую вам зайти на web ресурс посвященный Дом 2.

#204 Каталог on 10.25.09 at 4:27 pm

извините но айтой не качаю…

#205 HsvsRsvsesv on 10.28.09 at 9:19 am

buy xenical online

#206 viagra on 10.29.09 at 9:04 am

Hello!
viagra , cialis , viagra , cialis , , , , , , ,

#207 viagra on 10.31.09 at 8:44 am

Hello!
what viagra does to pussys , how does phentermine work , is tramadol hydrochloride water soluble , viagra bestellen , tramadol used for headaches , tramadol used for dogs , tramadol opioid , phentermine online consult fedex , cheap tramadol buy tramadol online , does tramadol show up as codeine ,

#208 order_ambien on 10.31.09 at 9:28 am

Hello!
order ambien , withdrawal ambien , cialis free sample , canada cialis , phentermine no prescription , phentermine amide , online pharmacy tramadol , tramadol online discount cheap , female viagra , viagra sales ,

#209 Pharmd770 on 11.02.09 at 3:49 am

Very nice site! cheap viagra

#210 Pharmb251 on 11.02.09 at 3:50 am

Very nice site! [url=http://oieypxa.com/kyrxxk/2.html]cheap cialis[/url]

#211 Pharmd532 on 11.02.09 at 3:50 am

Very nice site!

#212 Pharmk381 on 11.02.09 at 3:51 am

Very nice site! cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra ,

#213 Pharmc975 on 11.02.09 at 3:52 am

Very nice site! cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra ,

#214 Pharmd163 on 11.02.09 at 3:52 am

Very nice site! cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra , cheap viagra ,

#215 tramadol on 11.02.09 at 3:40 pm

Hello!
does tramadol have ibuprofen in it , cialis recreational gay , tramadol side effects in canines , discount tramadol ultram , click to buy cheap phentermine pills , cheap tramadol day shipping , what is stronger tramadol or vicodin , generic viagra , buy phentermine with out prescription , phentermine bad side effects ,

#216 viagra_canada on 11.02.09 at 3:42 pm

Hello!
viagra canada , zoloft and xanax , valium injections , phentermine sites that ship cash on delivery , long term ambien , cialis tablet , tramadol drug test , , , ,

#217 viagra on 11.02.09 at 4:49 pm

Hello!
viagra , cialis , viagra , cialis , , , , , , ,

#218 Exytrans on 11.02.09 at 7:04 pm

Я считаю, что Вы ошибаетесь.