How can the web serve be aware that the data is ready by the backend machine
I am setting up a searching engine to provide info searching service. The the system composition is as follows.
(1) Client. The user input query keywords in a text field of a Web Page.
Client <------------------> Web Server<-------------> PC Cluster<-------------------------> Database Server
(running tomcat,Socket client) (searching programs C++, Socket server)
(2) Web Server. Searching programs (In C++) run at the PC Cluster. Hence, the keywords have to be sent to the PC cluster. On the other hand, the search result has to be returned to the Web Server so that it can produce a web page for the user based on the search result. More importantly, the search result web page should be generated as soon as the result is ready.Currently, In the back-end server, Java SERVLETs are used to receive the keywords submitted by the user. As soon as the server receives the keywords, it forward them to a PC cluster via a SOCKET link(a SOCKET client runs on the Web Server). The PC cluster takes care of executing searching operations and generating searching results (finding out relevant documents).
(3) PC Cluster. A set of C++ programs, in which MySQL C APIs are used to access the database in Database Server, are used to execute the searching algorithm. The C++ programs do some processing works on the keywords. After that, the C++ programs execute searching operations involving some database accesses. Hence, the C++ programs can be not simply replaces by some MySQL user-defined procedures.
(4) Database Server. a MySQL database.
A socket server runs on the PC cluster to receive the keywords forwarded by the Web Server. Once the keywords are received, PC cluster executes searching algorithm to generate searching result. Then the PC cluster returns the result to the Web Server by writing the content of the results as records (Two fields: DocumentTitle and Similarity)into a Database (DbResult). Finally, the Web Server generate a web page based on the records and present the web page to the user's client.
Since doing searching on the PC cluster takes time, the servelet on the Web Sever has to keep checking with the database DbResult to see if the result is ready SOON after it sends the keywords to the PC cluster via the SOCKET channel. If the result is ready, it begins to produce the web page (a list of Doc Title and corresponding similarity score) based on the records.
In this scheme, the servlet has to keep checking if the result is ready, consuming valuable CPU time. When there are many visitors, the web server wastes a lot of time on this meaninglesss operation.
I wonder if there is better methods/architectures to implement this system, and if there is some ways to avoid the keeping checking database operation.
Last edited by cy163; 12-28-2008 at 12:23 AM.
There are a few different methods of implementing this (that I can think of)
1. You can miss out the database tier, and have tomcat JDBC to a stored proc on the database.
2. When you make the socket call, you can block on it and have the database tier, either send the results back over it, or send a message which would then cause you to read the db results.
3. Use a async protocol such as JMS to communicate with the database server.
Hope this helps
Before you criticize someone, you should walk a mile in their shoes. That way, when you criticize them, and if they get mad, you are a mile away and you have their shoes ;-)
By Sunil Menon in forum .NET
Last Post: 08-30-2002, 02:52 PM
By Constance J. Petersen in forum .NET
Last Post: 08-28-2002, 11:06 PM
By Tim Frost in forum xml.announcements
Last Post: 04-02-2001, 11:53 AM
By Desmond Cassidy in forum authorevents.kurata
Last Post: 04-19-2000, 04:54 AM
Top DevX Stories
Easy Web Services with SQL Server 2005 HTTP Endpoints
JavaOne 2005: Java Platform Roadmap Focuses on Ease of Development, Sun Focuses on the "Free" in F.O.S.S.
Wed Yourself to UML with the Power of Associations
Microsoft to Add AJAX Capabilities to ASP.NET
IBM's Cloudscape Versus MySQL