A bit of history:

When LymeNet first started in 1993, the World Wide Web had not yet been invented. Today everyone associates the Web with the Internet--one in the same. In fact, the total number of hosts on the Internet at that time was only about 25,000, and the only way for Internet users to obtain information on Lyme disease was to "telnet" in to our system and search our text database.

As Internet technology improved, we moved one step up to "Gopher", which was the text-only precursor to the World Wide Web that everyone uses today. Our systems ran on two small servers that were obtained from generous contributors at Sun Microsystems, and our connection to the Internet was via a 28K business dial-up line.

A few years later in 1995 we were running a web site which averaged about 300 ``hits'' per day. There weren't really any search engines in operation, so you just had to know our address ``www.lymenet.org'' to find our site. We started experimenting with Lotus Domino software to hold our informational database, and thus the beginnings of the ``LymeNet Flash'' discussion board. The ability for our users to communicate with each other and share their experiences proved to be the single most important feature we offered.

By year 2000, we had upgraded our servers with a one-time grant and upgraded our software with the ``Ultimate Bulletin Board'' forum software, and we averaged about 25,000 hits per day. This larger number of users required us to upgrade our Internet connection to a 384K business DSL line at a cost of several hundred dollars per month.

As of July, 2005, the statistics show that we averaged almost 200,000 hits per day, and that is on the same 384K business DSL line and old servers we had since before the year 2000! Over the years we pulled several tricks like implementing data compression, optimizing the layout of our web site, utilizing some outside services to handle some of our network traffic, etc. But we've been running the DSL line at near 100% capacity all day long.

There have been several suggestions about using third-party hosting providers, consumer-grade Internet service, etc. to save money. On the surface they seem cheaper, but the fact is that if you calculate the amount of traffic we pass through the system, the amount of data we store, and the cost of backups and mandatory server maintenance charges, it becomes about as expensive as running the systems in-house as we have for the past several years.

On a personal note, Marc Gabriel and I have spent countless hours over the past 15 years from original conception to what we have today. With a system that today supports several different Lyme-related web sites (with over 100,000 registered users on the LymeNet Flash forums alone), there is quite a bit of hands-on maintenance, late nights, and personal expense and sacrifice required to keep things going.

Why would we do such a thing for so long without any compensation? It is because we believe in what we are doing. I truly believe that the amount of benefit that so many people receive from the site justifies the back-breaking work of keeping everything up and running.

There have been mixed feelings about the proposal to charge money to support the bulletin boards. I think it's important that you know where the money is going, and where it is not going. We run as a non-profit organization, and we as system administrators never see a penny. One of the biggest expenses we have is for the Internet connection. Then comes ordinary administrative expenses (i.e. postage, etc.) to support the non-Internet endeavors of the Lyme Disease Network. Much of the expense of maintaining the equipment comes out of our own pockets.

We are very thankful for the many generous and loyal contributors over the years--they are the only reason we have made it this far. But as you can see from our growth numbers above, changing times require us to step back and find a way to cope what that growth. Unfortunately we just haven't been able to maintain enough ongoing funding from donations to pay for operations. By charging a nominal fee, we hope to generate a steady stream of income to pay for our Internet connection and equipment maintenance.

There are several outcomes which the LymeNet community can look forward to as a result of this funding effort:

1. We will upgrade our Internet connection to better reflect the daily demand for the site.

2. We will upgrade our servers to provide faster, more reliable and redundant service.

3. In the short term, we will upgrade our UBB forum software which should provide much better performance for things like the search feature which has been overloaded for so long.

4. Over time, we plan to upgrade to a more modern, feature-rich forum software which will allow our users to collaborate more effectively. Depending on demand, we may offer the capability to syndicate our forums through the use of RSS, and we may even consider allowing users to have a journal or ``blog'' on the system.

5. Reduction in ``trolls'' and other unproductive activity.

Despite all of this technology, our number one concern is to be faithful to our charter--to provide information and support to the community of people who have been affected by Lyme disease. We are always listening to you, whether you have 6 posts like I do, or 12,000 posts like some other long-time members do. We appreciate your loyalty and generosity over the years, and will continue to serve the LymeNet community as long as we are able.

Regards,

And even more, I want to thank you and all the others who keep Lymenet going from the bottom of my heart.

Lymenet has indeed been a lifeline to me and, I'm sure, to countless other Lymies.

There's a top of the line host out there that offers your own server with 350GB of data transfer per month for around $200 per month. That's insanely competitive and I know that's cheaper than what you guys have to go through. It would also make your lives easier too.

OK here's my stats discussion. A meaningful measure of traffic is really how much bandwidth or data transfer you have per day. Unique visits or IP's is another good one, but 'hits' is a stat I don't know why anyone uses. It's impressive because it's a high number and everyone assumes that everytime someone visits the site it counts as a hit. So people confuse hits with visits. For those that want to know, a hit is counted everytime you refresh a page but it also counts every image on that page as additional hits. So every smiley face, lyme logo, etc. is counted. So you could have 25 images on a page, so that would be 26 hits everytime you open that page. So one visitor could create hundreds of hits in a brief visit.

If a website has a lot of tiny images, they could generate a lot of hits with relatively low data transfer. Or you might have a website with few large imagaes that create much less hits but much higher data transfer or bandwidth used. So hits is difficult to make any sense of in my opinion. My personal website gets about 23,000 hits per day. Sounds huge, but it's only actually 4,000 visitors. I don't have a lot of images, but if I did, I could easily have 75,000 or so hits per day with just 4,000 visitors.

I'd be really interested to know how much data transfer Lymenet has per month. The UBB is bandwidth inefficient like crazy, and it would be interesting to see how much a PHPBB board by itself would save money/bandwidth. Probably 40% or more.

I'd like to echo Janet's response.

You have given me a lifeline to recovery and I thank everyone at lymenet for their effort and time required to continue this site.

Sincerely,

Lymester
CT

quote:

---

Originally posted by 24bit:
Thomas,

There's a top of the line host out there that offers your own server with 350GB of data transfer per month for around $200 per month. That's insanely competitive and I know that's cheaper than what you guys have to go through.

---

24bit-

I appreciate your input and your interest in helping LymeNet.

I am aware of several off-site hosting providers and the associated costs and we have discussed internally at great length the pros and cons of using such a service to host LymeNet. However, we found that if you look at the actual costs (i.e. look beyond the "sticker price" and total up the entire cost of hosting, including disk storage, web and database application hosting, E-mail, backups and off-site storage, total and peak network bandwidth, and advanced firewall/IDS to protect against the frequent hack attempts we receive) they wind up being the same or far greater than what we currently pay to host the system ourselves. And the big kicker is that over the long term we realize substantial savings because we own almost all of our equipment and generally use our personal (non-LymeNet) funds to make improvements and fix any problems that occur. If we had used a hosting provider for the $200 per month you quote, we would have had to pay $12,000 for hosting over the past 5 years alone. And it turns out that a more realistic price for what we have in place today winds up being more like $400-500 per month.

As far as alternative forum software, we do have plans in place to migrate from UBB to phpBB in the next several weeks, but it is not a trivial task. There is no properly-working direct conversion utility to migrate from UBB to phpBB, and so there is a substantial amount of scripting and manual effort involved in converting the 6800 users and tens of thousands of forum posts over. I have already done this once on a trial basis, and the automated portion of the conversion alone took nearly 20 hours on a fairly fast multi-CPU server, not to mention the hours of manual effort to fix the many "glitches" left behind by the conversion scripts that exist to date. The reason we are moving to phpBB is to extend available features, both on the user side as well as on the back-end, as well as performance and reliability. We will not save on bandwidth at all; in fact, it will increase our traffic quite a bit and we will have to spend some time to mitigate this by carefully designing templates to optimize bandwidth while providing a user experience which is as similar to the current UBB forums as possible...as many of our users are very "sensitive" to change, as you may have gathered ;-)

I'm also aware of the more effective traffic measurements you started to touch on, and we have kept detailed statistics on several metrics including "hits", whole-page loads, partial/cached page loads, unique daily visitors, referrals, search engine bandwidth, amount of traffic generated by certain pages and types of files, page load time, etc. We do use that information on an ongoing basis to tune performance where possible. The reason I quoted "hits" is to illustrate in laymans terms the combination of raw volume of traffic going over our network as well as interest in the site, and that is a metric which we have collected since our web server went live over 10 years ago. Regardless of the usefulness of "hits" as a metric in the real world, it does clearly show the order of magnatude of growth versus the relatively little amount of money we have spent on improvements of our equipment and communication lines over the years.