ISOOSI Research Engine & Web Directory

ISOOSI™ (pronounced "I Soo See") Research Engine is a hybrid search engine / web directory. It is the product of a fun, yet challenging project turned adventure pursued by a collective of programmers, data wonks and internet geeks that has spanned several years.

Our crawler continually traverses the web automatically discovering new links. After several cleaning operations (checking the links are live (3 strike policy), no malware detected or home page redirects, etc.) we add the links and associated meta information to our searchable "ISODEX" index. Currently, only ~20% of the links crawled end up in the index due to our QA cleaning processes.

The system then tries to determine the most relevant category in the web directory portion ISOOSI for each individual link within the ISODEX based on a set of criteria. For each potential category match, our algorithm calculates a relevancy score and only those scoring 95% or higher are sent to our editorial team for categorizing. The rest stay in our search index but do not get categorized in the web directory.
 

ISODEX Contains ISOOSI Web Directory


On an ongoing basis our editorial team manually reviews the categorized links and makes them featured when we are certain it is a quality website and the content adheres to our featured listing policies. Our editors check a number of critical onsite and content factors before making a link featured. This includes checking content on any linked social channels. We want to ensure that all featured listings are quality resources; web pages of value that our website visitors would like to visit.

Obviously our crawler finds new links much faster than we can categorize them; and we can categorize them much faster than our editorial team can review them. For this reason our searchable ISODEX grows significantly faster than our web directory does. This means the searchable ISODEX is populated with much fresher links and current information augmenting the search results of links specifically listed in the web directory.

The ISOOSI Research Engine currently contains over 10.8 million links and is growing every day. Contained within it is ISOOSI Web Directory - one of the largest categorized directories of English language web pages we know of online.

Statistics (as of August 1st 2013)

10.8 Million links in total in the ISODEX, which includes;
4.2 Million categorized (English Language) links in the ISOOSI Web Directory
850k human reviewed categorized web directory links (English Language)


ISOOSI is more than just that though. It is a research tool to perform numerous types of searches all from one place. The tabbed "ISOSEARCH" feature that is available throughout the website integrates search functionality via APIs from a number of different websites such as Google, Bing, Twitter, Wikipedia, Facebook, Amazon, etc. The additional search functions integrated were chosen based on what we felt would be helpful to users and our members that list their businesses. It certainly is not exhaustive by any stretch and we will hopefully be adding more as the site develops. If you want additional search functionality added or current search functionality improved, we greatly appreciate the feedback.

We want the ISOOSI website to be a useful research tool. However we are under no pretense that it could ever match the size and scale of many of the large indexed search systems currently on the web.

We hope though that by having a hybrid search engine / directory system and complimenting that with search results from many other great sources on the web, we can help some people find the information they want in an easy, organized way.

It started life as a home-grown project and has been carried along for many years by a handful of folks over many long days, long nights and far too many weekends :) It is and probably always will be a "work in progress".... continually evolving, adapting and improving.

We hope you find it useful :)
 

History

It makes us chuckle to think back, but the ISOOSI Research Engine started life as a searchable MS Access 95 database with just a few form designs as a User Interface. Prior to that the data was just in the form of a list of BBS systems and Usenet groups.

Primarily all the initial links were science related (math / physics / materials science) and focused on the post-graduate research work performed by Carlos Fernandes (our company founder), at the Synchrotron Radiation Source (SRS)UMIST and associate research teams at CERN.

As the web developed, more and more general links were added to the database. Web work didn’t pay very well back in the mid 90's (at least not for us) so we moonlighted from regular 9-5 jobs for a while and did what we could to improve the system when we could. We built a very crude but successful crawler and added that into the system activated by a MS Access macro.

Web clients had their websites added into the database and for good measure we added all the links of their competitors and other useful resources in their industries. We also used our intelligent relevancy algorithm in the search functions for many other databases including patient databases for hospitals and pharmacies.

The database stayed as an offline project until 2003 when Carlos moved ISOOSI from the Cayman Islands and formed ISOOSI, LLC - a small boutique search engine marketing and design agency in Naples, Florida.

In 2004, we moved the ISODEX data from MS Access to MS SQL. Also in 2004, Erik Stafford, who had been a close friend of Carlos in the Cayman Islands, joined ISOOSI, LLC. as Creative Director and Managing Partner. As our team grew we moved in the direction of .NET development.

In April 2006, Jason Farrington joined ISOOSI, LLC. as Director of Web Services. Under his and Carlos's guidance the ISODEX system was improved and a custom .NET platform was built around it.

ISOOSI continued to work on the research engine over many years. We integrated our intelligent relevancy algorithms into numerous other applications and websites including several Chambers’ of Commerce (Naples, Fort Myers and Bonita Springs) and other member based organizations such as the YMCA and local builder associations.

After many years of working on the project on weekends and nights, ISOOSI Web Directory, Inc. was finally formed in 2011. Soon after we began to integtrate our search functionality and platform within the intranet systems of a number of large corporations and businesses.

Carlos Fernandes remains President of ISOOSI, LLC. and is the President of ISOOSI Web Directory, Inc. Jason Farrington is the VP and Erik Stafford is the Creative Director.

Carlos works with several high-profile digital agencies throughout the world. Since 2005 Carlos has also been the Director of Search Engine Marketing for Launch Dynamic Media - a leading digital agency in Reading, Pennsylvania.

Carlos is also retained by several Fortune 100 companies to lead their online marketing strategies.
 

Why did it take So Long?

Up until 2012 there were no employees working specifically on the research engine full time. All development work was performed outside of regular business and it was always more as a "pet" project. Anyone who has been involved in research or development knows - doing such projects takes time.... even if you work on it every night and every weekend.... which we did.

And because of this it takes the support of families too... and we had that in spades.

It also takes money unfortunately :)
 

Why Build it?

There are many great search engines and web directories already. We agree. There are websites with a wealth of information; websites like Wikipedia, WebMD, Mashable, Techcrunch and countless others. There are unbelievably powerful and comples search engines like Google and Bing that deliver answers to questions and information you seek in the blink of an eye. And there are new search engines now like DuckDuckGo that are doing a lot of really cool things that help users perform research on the web. 

However, if our website can help a small business owner get additional exposure for their products and services, then in our opinion it is of value. If our research engine can help even a few users find the information they want to research, that is of value. 

This was always an idea we wanted to pursue....  trying to help people find information or links that they may find useful now or need at a later date.

This involved crawling the web and trying to categorize the links to help those who still wanted to traverse vertically and horizontally through a web directory to find RELEVANT and RELATED information.

On search engines it is often hard to find content on low power / authority websites, primarily becuase they rely on link signals. 

A website owner may have poured their life into writing valuable content on a particular subject. If that owner didn't know anything about SEO or did not get any links into their website, chances are their website would probably never come up in a search engine result.

Low power or low authority does not mean low quality. As in this case...it just meant the owner was more bothered about writing about his or her passion than learning about SEO. A good web directory allows for that content rich resource to get that much more exposure from their traffic.

Web directories ARE still of value today. In this example, if the owner of the low power website submitted their website to just one web directory that was well indexed by the major search engines, it is more likely than not their website too would soon get crawled by the Search Engine robots. This would help.

Unless the search engines can make every business owner SEO savvy, make them use their own ADD URL links (that tend never to work), sign them up to their webmaster tools, or push them over to half-baked services from domain registrars that claim to get you all listed all over page 1 of the SERPs - then good web directories should still have a place in the overall Intenet ecosystem.

BUT, to be of real benefit to those listing their businesses in them, in our opinion they must also be WELL USED... and not just by webmasters and SEOs.

A good web directory is one that adds value. Informative, organized, regularly maintained, trimmed and updated. If  it can also help your website get noticed by the search engines.... GREAT.

But what about the other 99% of the pie? What about some real visitors? Some leads?

Far too often, the only traffic a web directory receives is by people wanting to add their link - for link building purposes. The business owner, webmaster or SEO who visits the directory to submit their link would probably never return to that website again after they know it is listed.

How about people using web directories to search for information? What ever happened to that. Well, you can only search for informaiton so many times and get ZERO results before you get tired and move on.

We were frustrated at visiting web directories and doing searches for things and just not getting any results. It seemed to us that web directories as such were? JUST being used for link equity and nothing more. We were also frustrated about paying certain companies back in the day $300 annually for a web directory listing - simply for it "passing link equity" - but getting no additional traffic from the site.

We still feel that there is a valid use for a web directory, but they can only pay real dividends for the businesses that list in them, if real visitors use them and it is not just webmaster or SEO traffic.

If the directory has fresh results to compliment those categorized results, an intelligent search algorithm that returns more relevant results and other useful information than most directories, then there is an increased chance that the directory website would get used for more than just submitting a website.

The addition of other search functions via APIs of search engines and information sources was also central to achieving the goal of providing a comprehensive online research tool. That benefits the businesses that get a featured listing in the directory too. 

The more people that use the website, the more chance that the website that was just submitted by the small business owner that didnt have a SEO budget gets some real added value; visitors.   
 

Current Technology

Our research engine currently uses MS SQL and custom ASP.NET code built from the ground up.
 

Future Development

We are currently performing a port of the ISODEX to SOLR / Lucene and we hope to have that running in a development environment by December 2013 for evaluation.

We are also experimenting with Node.js and continue to try to work on ways to improve our system and make performance better.

Our search relevancy and categorization algorithms are continually improved and continuous testing and development work is regularly performed on these functions.
 

Frequently Asked Questions

Several FAQs about the website and our web directory are answered here.