• Home
• About
• Contact

Subscribe

Upcoming Book

Ads

Introducing Waegis

This would be one of the announcements that I make public on a regular basis and every a few months!

During the past three months, I've been working on a big project and an enterprise platform/solution that I'm going to introduce now. More than all its side-effects I enjoyed the development process from the early stages to the current point and would love the rest as well.

Although I started the development in the recent three months but the idea and the mind work were with me for almost three years from the university days and I thought much about it in this while.

History

It's almost three years that I'm blogging on this site. Before Graffiti CMS, my site was powered by various versions of Telligent Community Server from the first Betas. In the first a few months of blogging on this site, I got some great friends on the Community Server community like Jayson Knight and Dave Burke. At the same time, our blogs were attacked by lots of spammers and actually Community Server got affected by them. At first days, it wasn't so hard to delete a few spams but I had some previous experiences from online web spammers from a Persian blog and could expect a worse thing in the near future.

At that time, we had some conversations with Jayson, Dave and others about the problem and my suggestion was to have a centralized web service for our sites and blogs to help the spam filtering. That idea was very simple but after a short while Jayson first introduced us to Akismet by Automattic. After a while, many of our blogs and sites used this service to protect their content against spammers and some of my friends like Dave became a solid fan of CAPTHCA controls.

But that idea was still in my mind even though someone else had done it. I was studying Applied Mathematics and there were lots of courses and stuff related to this field that could help filter spams. The birth of Akismet stopped me from getting serious for the implementation at the moment but this field was interesting for me to follow so studied more about it.

In the past three years I've been thinking about the idea and finally got interested to implement it since last year. During this while I also did some works around spam filtering that you may be aware of them. Unfortunately my busy schedule at service didn't give me the time to work on it but three months ago and when we were going to get closer to our long new year holidays, I finally got my feet wet and started the development!

Last month and in a blog post (that was more like an article than a blog post) I outlined many points about web spammers and discussed about different aspects of the work that they're doing. Reading that post may give you a better understanding before following this post as well.

This time I had some new goals and reasons in my mind. One of them was that I wanted to train my skills in mathematics, statistics and software architecture and development in a very high level. The other reason was the fact that I've been in a progress on the community and day after day have played a more important role on it. I found that all .NET blogging engines and sites are heavily dependent to Akismet which is a PHP service and this doesn't seem very good. Moreover, .NET couldn't find its place on the web between enterprise sites and solutions so I decided to try to fill this gap in a field like this. A few days ago I wrote a post related to this as well and hope that I can do more in this field.

In general, some sites are known as strategic sites and such a site/service is one of them because the technology isn't easy to achieve due to its bold prerequisites like good algorithms and high level of best performance code.

Introduction

After this long introduction, let me introduce Waegis. Waegis, which stands for Web + Aegis, is simply an enterprise online platform, service and solution for site spamming and acts as a centralized service to filter site, blog, forum, contact form or Wiki spam. The primary goal is obvious and is being an aegis for the web!

Here I would say that first time Phil Haack introduced me to the Aegis English word when we were working on a project and I tried to work on this word to find a good, meaningful and short domain name.

I have written a detailed official proposal for Waegis and will publish it in the near future and after getting closer to the final launch. But for now let me give some perspectives about it to let you understand what is it and what is it supposed to do.

Goals

Generally, I have some main goals of building Waegis:

• Help the world: Spamming is wasting many resources from the human. From a technical perspective, it uses the server resources for sites, makes them dirty, and has a negative effect on the content. From another view, it wastes a lot of time from site owners, webmasters and bloggers to manage their sites. If you use CAPTCHA controls to save your time then you're wasting time from your visitors to force them enter something to validate themselves. There is no doubt that the success of such a service is very good for the world. Just suppose that it can save 15-30 minutes from owners and visitors of a site. Many of these guys are technical people so their time worth it!
• A training for myself: I love to solve complex problems, build enterprise stuff and play with the latest technologies. Moreover, my military service has made me busy. So I decided to implement this idea to train myself and keep my mind up. In fact, it was a great training and I've been loving this work. Regardless of anything else, I'm satisfied with what I got from the project. I learned many things and applied many of the concepts that I knew in an enterprise platform and a professional level. By nature, this was a good problem to solve and I liked it.
• Making money: Of course, my mind and my time cost much but at first glance this site needs money resources to have an enterprise hosting and uptime.
• Minimum fault ratio: In fact, the success of such a service depends on the number of users and sites that it hosts and also its fault ratio. Here fault ratio is defined as false-positive and false-negative and the less values for both ratios is better.
• Targeting Web 3.0: As long as we move on and web grows, spammers bother us with their works and we need to evolve with them. So Waegis tries to target web 3.0 to make it cleaner from spams.
• Simplicity: simplicity is what I always follow in my works and this time I tried to bring simplicity to this platform.

Technical Details

Waegis is built on top of a special architecture which has SOA in its core and is powered by the latest bits of .NET technology from Microsoft. I used .NET 3.5 and C# 3.0 to write everything and also applied ASP.NET 3.5, ASP.NET AJAX and ASP.NET AJAX Control Toolkit to write the site and Windows Communication Foundation 3.5 to write the service-side of things. Waegis uses SQL Server 2005 and many of its new features for the database backbone. I also used Windows Workflow Foundation 3.5 at some point but after some performance testing, I decided to move on without it.

Waegis is actually a platform, an enterprise platform that is easy to redistribute so I can make it available as a standalone product as well. The problem (spamming) and the nature of it (which evolves quickly) and the fact that at present I'm the only guy behind the site with a busy schedule and program, required me to invest many things to automate everything and write a platform that lets me react against the changes from spammers quickly. So Waegis is a platform with many features that are already there but not used and are built per my prediction of the future of spamming. Moreover, Waegis has some main extensibility points that let me develop it quickly.

Both service and site are optimized to work with latest server technologies from Microsoft and 64 bit platforms.

Technical Features

I know that there are many things to write in the next sections and I don't want to bother you with long descriptions so I shorten this part and postpone it to the future but in a nutshell:

• Waegis is an enterprise extensible platform.
• Simplicity is the first thing that you'll notice if I release the source code after my death (Microsoft and open source?! Are you kidding?!).
• There is a professional integrated security mechanism to defend it from many attacks from hackers and spammers.
• It is built on top of some self-learning methods and tries to reduce the need to a human administrator as much as it can.
• There is a professional RESTful API built on top of WCF 3.5. I tired to build this API with the same structure as similar existing services.
• Performance and speed have been two main concerns from the early stages of development because I was aware of the complexity of the spam filtering methods and algorithms (when applied in good details). Believe it or not but I rewrote %50 of the code to gain the best performance.
• Automated administration was another goal and feature to help keeping up the service with less human effort and best quality.
• Monitoring and diagnostics were two other concerns that are here to prevent anything bad to happen!

Commercial Usage

Like similar services, while Waegis will offer a free service for small sites and blogs, it also will offer some commercial plans for bigger sites with more features. The goal is to make money both to provide a good hosting solution and keep the progress of the site and also to help provide money for my addictions!

At first glance, my will is to be able to make the hosting %100 stable. A professional hosting costs more and I'd like to make money to provide a %100 trustable hosting solution.

Competition

I have no doubt that everyone has one question in mind about the competition with Akismet. Well, you're right to some extent but let me describe my notes at this first stage to clarify things.

First of all, I want you to understand that spam filtering has had its big market for a longer time even before we find that there is Web 1.0, Web 2.0 or Web 3.0. Both spammers and spam filters have been evolving during the years and have had their good market. So I don't think there is a limited space to work on this field especially because there are only two services in this field (Akismet and Defensio). Defensio is powered by Ruby on Rails but I didn't hear good feedbacks from this site so don't consider it at the moment.

Moreover, I love competition and competing with others in a fair manner. Competing with a huge successful company like Automattic with all the bright guys at this company like Matt isn't easy for an individual for me but is possible. You would wonder that how I could get in this field to compete with such big names!

Believe it or not, in my mind, Akismet is a friend not a competitor and I have respect for all the guys who have built this service. No one can understand what I'm saying until he or she writes such a service. Here you don't work with a regular data-in/data-out system. What you want to write is a very critical software that is one of the most professional instances of data transaction mechanisms and also needs a deep knowledge in some fields like mathematics, statistics and artificial intelligence. Here you need to invest something to make your methods convergent and get the result. So I think that Waegis is on the same track as Akismet and similar services. Such services are there to help the web more than making the money! The effort that is put in these services for development, maintenance and hosting is much more than normal sites and services.

However, with no doubt the success of Waegis can challenge the success of Akismet to some extent. We (.NET community) are trying to improve things and improve our position in the software world and as I stated above, this has been one of my main goals of building this platform as well. After that Telligent challenged WordPress with Graffiti CMS, now Waegis may challenge Akismet and I hope that it can!

What's Next?

Waegis has hit the first Alpha version and is now being tested. There are some minor parts of the platform that I'm developing in parallel with this Alpha test. I'm also feeding the system with lots of real world data which is a very important point and a key point in the success of such a service.

On the other hand, hosting is a very critical aspect of an online service like Waegis. So I'm planning and working to provide a professional hosting solution with powerful hardware and software. Most likely I'll do a direct hosting in a good data center to have control on everything and reduce monthly costs. I'm also going to be in touch with a company to get sponsorship.

Right now I hope that I can launch a public Beta in the next 45 days (even sooner, if possible). At the start point, I want to target .NET community and .NET powered blogging engines including Community Server, Graffiti, BlogEngine.NET, Subtext and DasBlog so hopefully Waegis comes with a .NET library with an open license as well as add-ons and extensions for all these blogging engines. Then the next step is to get outside the .NET world and target other engines and tools as well. Right now I have developed the library and integrations for Community Server 2008, Community Server 2007.1, Graffiti CMS and BlogEngine.NET and unfortunately there wasn't an extensibility option to let me integrate it with Subtext and DasBlog so it requires an effort form their team members to help me.

In the latest stages of the project and for business side of things and service hosting, my dear friend, Mehrdad Ebrahimi, has helped me a lot so I'd thank him here.

How to Help?

The question that may arise for my readers is why I'm announcing this site at this stage? Well, there are two reasons:

• This public announcement puts me under a pressure to launch the site on time.
• Waegis should get an initial data and I need to feed it. Currently I have reached to a good point in this progress but I want to do much more than this. Having a good initial database can help the site in competition as well. I don't have a popular blogging engine with many users to use it in order to feed the system.

So here I want to invite everyone to help me feed the system. I'm looking for active bloggers who have a blog powered by Community Server (2008 or 2007.1), Graffiti or BlogEngine.NET who are interested to help me and meet these conditions:

• Have access to the server to deploy add-ons and extensions.
• Get a reasonable number of content items (comments, trackbacks, pingbacks or forum posts) on the blog or site.
• Currently get spam. The number doesn't matter but I need to have a good combination of legitimate content and spam.
• Having a trustable spam filtering mechanism would be excellent because it can help us automate the feeding progress without doing a manual work and worrying about any bad effect on the site or blog.

Note that your content and language doesn't matter at all. I already have fed Waegis with six spoken languages with different content categories. I would be satisfied with 10-20 blogs/sites for Alpha 1 and 40-50 blogs/sites for Alpha 2. A blog or site that gets more spams would be better, of course. And same is for a blog or site that doesn't use some solutions like CAPTCHA but none of these can stop you from participating.

Moreover, if you have a blog database that you think that it may help my work, then I'd appreciate your help.

If you're interested to help and can help, please drop me a line and we'll be in touch quickly. I'd prefer some real names and well-known bloggers who are identified to the web to prevent any cheating. Please keep your participation interest private and do not leave comments about it since there may result to some attacks to your sites to beat the system. To appreciate any help, I'll give out a commercial account to these Alpha helpers/testers.

Before the end and to be clear, I want to let everyone knows that neither this is a %100 community work nor a %100 commercial work. Commercial goals are bold so consider this in your support. I try to appreciate your help with something more than a simple "Thanks"!

I'll write more about Waegis in the upcoming weeks!

17 Comments : 05.12.08

Feedbacks

Leave a Comment

Name

Mail

Website

Comment