InformationWeek. com News, analysis and research for business technology professionals, plus peertopeer knowledge sharing. Engage with our community. The C10K problem Help save the best Linux news source on the web subscribe to Linux Weekly News Its time for web servers to handle ten thousand clients. Modules-based+Structure.jpg' alt='Solaris Internals Core Kernel Components Pdf' title='Solaris Internals Core Kernel Components Pdf' />Search List Subjects Authors Bodies must pick a list first Set Page Width BSD aic7xxx appscriptchanges appscriptdev bsdiannounce bsdiusers. The Unix system is composed of several components that were originally packaged together. By including the development environment, libraries, documents and the. Firebird 1. 5. 6 Release Notes. CORE 1. 04. 2fixed by V. Horsun1. 5. 4 Transaction numbers could overflow the 3. Kernelbased Virtual Machine KVM is a virtualization infrastructure for the Linux kernel that turns it into a hypervisor. It was merged into the Linux kernel. For your security, if youre on a public computer and have finished using your Red Hat services, please be sure to log out. Log Out. Fix back ported from Firebird 2. Bug CORE 1. 05. 1fixed by V. Horsun1. 5. 4 Possible stack corruption in the deferred work handler. Fix back ported from Firebird 2. Bug CORE 9. 66fixed by P. Beach, D. Yemanov. Socket binding for events exhibited a bug where 1. Fix back ported from Firebird 2. Bugs CORE 1. 57, CORE 3. P. Beach, A. Peshkoff1. In previous versions of Superserver, if the server could not establish a socket. Connected clients would. Now, under these conditions, the client will hang until the socket call times out by. Failed to. establish a secondary connection for event processing and will continue to serve user. Fix back ported from Firebird 2. Bug CORE 9. 11fixed by V. Horsun1. 5. 4 Access violations segfaults could occur in the Classic server when clients woke up after. Not registeredfixed by P. Beach, D. Sibiryakov 1. When the buffer overflowed in isql, the error was not detected. Fix back ported from Firebird 2. Not registeredfixed by P. Beach, D. Sibiryakov 1. Isql had some problems handling quoted strings. Fix back ported from Firebird 2. Not registeredfixed by C. Valderrama1. 5. 4 On Windows, GSTAT would print unknown Windows NT error instead of. Win. 32 error message. Fix back ported from Firebird 2. Bug CORE 9. 73fixed by A. Harrison 1. 5. 4 A condition was causing BUGCHECK1. Fix back ported from Firebird 2. Bug 1. 17. 51. 57fixed by V. Horsun1. 5. 3 An error in the thread scheduler would cause the server to lock up if an IO error occurred. Not registeredfixed by A. Peshkoff. The iscuseraddmodifydelete functions worked wrongly under Administrator account on. Win. 32. 1. 5. 3 At some point during Inter. Base development, the intention was to make the Win. Superuser privileges on Unix were emulated for Administrator on Win. Super. Userroot on UnixLinux. It worked, up to a. However, if the Win. Administrator user tried to call these functions through a SYSDBA login. The code for a root style login has been disabled in the Win. Now any Windows user, including Administrator, must supply the SYSDBA user name and password. Not registeredfixed by A. Peshkoff1. 5. 3 Denial of Service vulnerability an extra long database file name could crash the server. Endemic. security bug. One of many overflow vulnerabilities fixed in the Firebird 2 code, this fix has been. Not registeredfixed by A. Peshkoff1. 5. 3 The server would crash during some DDL operations. Fix back ported from Firebird 2 HEAD. Not registeredfixed by C. Valderrama1. 5. 3 In some cases, a BLOB filter declaration would cause the server to crash. Fix back ported from. Firebird 2 HEAD. Bug 7. Foxit Reader Serial Key more. A. Peshkov1. 5. 3 A locally exploitable stack overflow vulnerability was detected and fixed. Not registeredfixed by D. Yemanov1. 5. 3 The server could crash while performing a metadata scan for a complex table. Fix back ported from. Firebird 2 HEAD. Not registeredfixed by D. Yemanov1. 5. 3 Database corruption could occur due to allowing certain pre trigger actions, such as deleting a. BEFORE UPDATE trigger. Fix back ported from Firebird 2 HEAD. Not registeredfixed by V. Horsun1. 5. 3 CPU load would rise to 1. Fix back ported from Firebird 2 HEAD. Bug 1. 07. 68. 58fixed by V. Horsun1. 5. 3 A source of possible corruption was exhibiting in the Classic server as a page type exception. Identified, fixed and back ported from. Firebird 2 HEAD. Not registeredfixed by V. Horsun1. 5. 3 When the gfix service code tried to reattach to a database that had become unavailable, the server. An endless loop would occur due to the inability of the service to interact with the end user. Fixed and back ported from. Firebird 2 HEAD. Not registeredfixed by D. Yemanov1. 5. 3 Character set andor collation specified for local variables in PSQL would get lost, potentially. Fix backported from Firebird 2 HEAD. Not registeredfixed by D. Yemanov1. 5. 3 If any DDL operations were active during a database shutdown, the server would crash. Fix back ported. Firebird 2 HEAD. Bug 1. D. Yemanov1. Subqueries in in VIEWs were returning character data with the wrong character set. Fix back ported. Firebird 2 HEAD. Not registeredfixed by C. Valderrama1. 5. 3 There was an issue with quoted identifiers in the ISQL command SHOW GENERATORS in Dialect 3. Not registeredfixed by A. Harrison1. 5. 3 No more than 3. GPRE. Not registeredfixed by A. Peshkoff1. 5. 3 Wrong permissions were installed for the QLI help database. Bug 1. 04. 59. 70fixed by D. Yemanov1. 5. 2 An old legacy bug that has continued to bug us is that, when a client had some events registered. CPU time until the parent port client. API routine reported on its failure. This bug affected all FB versions more or less, depending on the. Dummy. Packet. Interval configuration option and only TCPIP connections. Further work has been done to rectify the problem in v. It now appears to be solved. Bug 5. 44. 13. 2fixed by C. Valderrama1. 5. 2 UDF with NULL input parameter, problem fixed. Bug 7. 28. 83. 9fixed by C. Valderrama1. 5. 2 Left join would defeat a UDF by mangling a null descriptor. Fixed. Not registeredfixed by D. Yemanov1. 5. 2 Error trying to delete from a naturally updatable view containing computed expressions. If you had. a view like this. DELETE operation. This is fixed in FB 1. Not registeredfixed by A. Peshkoff1. 5. 2 Numeric data types represented by floating point variables were being processed incorrectly in. EXECUTE STATEMENT with dialect 1 databases numerics were being scaled incorrectly. Now fixed. Not registeredfixed by V. Horsun. If DISTINCT was used in an aggregate function and the record set being processed aggregated was. This memory was not returned until disconnect. This routine would eat 1. MB on FB 1. 5. 1 and previous. CREATE PROCEDURE MEMLEAK. DECLARE I INT 1. DECLARE T INT. DECLARE C INT. WHILE I 0 DO. SELECT RDBINDEXTYPE. COUNTDISTINCT RDBRELATIONNAME. FROM RDBINDICES. WHERE 01. GROUP BY 1. INTO T, C. I I 1. It is fixed in v. Not registeredfixed by D. Yemanov1. 5. 2 The server leaked resources when an exception was thrown from a selectable stored procedure. The. procedural request wasnt freed properly and caused errors like too many concurrent executions of. CREATE PROCEDURE P INP INTEGER. RETURNS OUTP INTEGER. OUTP INP 0. UPDATE T SET ID 1. WHERE SELECT OUTP FROM P1 1. The leaking request blocks were returned on disconnect. Now fixed. Not registeredfixed by P. Reeves1. 5. 2 The server log was polluted with SIGPIPE errors when running Super. Server on UNIX. The legacy. Inter. Base code was logging sigpipe errors for SS running on ix. Unfortunately sigpipe errors come in. In extreme. cases this led to filling up the entire partition. Logging of SIGPIPE errors has thus been disabled. Not registeredfixed by V. Horsun1. 5. 2 1. CPU usage was exhibited by the cachewriter thread in some rare cases reported by Adrianos. Santos Fernandes. To reproduce in v. Prompt 1. CREATE DATABASE test. CREATE TABLE T N INTEGER. B test. fdb test. C test. fbk test. Prompt 2. gbak B test. The server would consume 9. CPU until the isql prompt 2 was disconnected. The bug didnt occur. GARBAGECOLLECT in the last command. Not registeredfixed by A. Karyakin, D. Yemanov1. A possible source of server crash was discovered in the opconnect handler. When a TCPIP packet lacking user information was received on the server port, the server could crash. Because it was the first packet opconnect in the client server protocol, it exposed the server to any. Do. S attack. Anyone could kill the server with just one TCP packet. Now fixed. Not registeredfixed by D. Yemanov1. 5. 2 The server could crash with complex queries where lots of streams were used in a sortmerge. A. complex union with many aggregations and merge joins could crash the server because of a streams buffer. Although the current limit is 2. The C1. 0K problemHelp save the best Linux news source on the web subscribe to Linux Weekly News. Its time for web servers to handle ten thousand clients simultaneously. After all, the web is a big place now. And computers are big, too. You can buy a 1. 00. MHz machine. with 2 gigabytes of RAM and an 1. Mbitsec Ethernet card for 1. Lets see at 2. KHz, 1. Kbytes, and 5. Kbitssec per client. It shouldnt take any more horsepower than that to take four kilobytes. That works out to 0. Those. 1. 00client licensing fees some operating systems charge are starting to. So hardware is no longer the bottleneck. In 1. 99. 9 one of the busiest ftp sites, cdrom. Gigabit Ethernet pipe. As of 2. 00. 1, that same speed is now. ISPs. who expect it to become increasingly popular with large business customers. And the thin client model of computing appears to be coming back in. Internet, serving. With that in mind, here are a few notes on how to configure operating. The discussion. centers around Unix like operating systems, as thats my personal area. Windows is also covered a bit. See Nick Blacks execellent Fast UNIX Servers. In October 2. 00. Felix von Leitner put together an excellent web page. One of his observations is that the 2. Linux kernel really does beat the 2. OS developers food for thought for some time. See also the Slashdot. Felixs results. If you havent read it already, go out and get a copy of. Unix Network Programming Networking Apis Sockets and Xti Volume 1. W. Richard Stevens. It describes many of the IO. It even talks about the thundering herd problem. And while youre at it, go read Jeff Darcys notes on high performance server design. Another book which might be more helpful for those. Building Scalable Web Sites by Cal Henderson. Prepackaged libraries are available that abstract some of the techniques presented below. ACE, a heavyweight C IO framework. IO strategies. and many other useful things. In particular, his Reactor is an OO way of doing nonblocking IO, and. Proactor is an OO way of doing asynchronous IO. ASIO is an C IO framework. Boost library. Its like ACE updated for. C. IO framework by Niels Provos. It supports kqueue and select. Its level triggered only, I think. Niels has. a nice graph of time to handle one event. It shows kqueue and sysepoll. My own attempts at lightweight frameworks sadly, not kept up to date. Poller is a lightweight C. IO framework that implements a level triggered readiness API using whatever underlying. API you want poll, select, devpoll, kqueue, or sigio. Its useful for benchmarks that compare. APIs. This document links to. Poller subclasses below to illustrate how each of the readiness APIs. C IO framework that was my second try. Poller. Its lgpl so its easier to use in commercial apps and. C so its easier to use in non C apps. It was used in some commercial. Matt Welsh wrote a paper. April 2. 00. 0 about how to balance the use of worker thread and. The paper describes part of his Sandstorm IO framework. Cory Nelsons Scale IO library for Windows. Designers of networking software have many options. Here are a few. Whether and how to issue multiple IO calls from a single thread. Dont use blockingsynchronous calls throughout, and possibly use multiple threads or processes to achieve concurrency. Use nonblocking calls e. ONONBLOCK to start IO. OK to start the next IO on that channel. Generally only usable with network IO, not disk IO. Use asynchronous calls e. IO, and completion notification e. IO finishes. Good for both network and disk IO. How to control the code servicing each client. Unix approach, used since 1. OS level thread handles many clients each client is controlled by. GNU state threads, classic Java with green threads. OS level thread for each client e. Java with native threads. OS level thread for each active client e. Tomcat with apache front end NT completion ports thread pools. Whether to use standard OS services, or put some code into the. Vx. D. The following five combinations seem to be popular. Serve many clients with each thread, and use nonblocking IO and level triggered readiness notification. Serve many clients with each thread, and use nonblocking IO and readiness change notification. Serve many clients with each server thread, and use asynchronous IOserve one client with each server thread, and use blocking IOBuild the server code into the kernel. This is the traditional favorite. With this scheme, the kernel tells you whether a file descriptor is ready. The name level triggered comes from computer hardware. Jonathon Lemon introduced the terms in his. BSDCON 2. 00. 0 paper on kqueue. Note its particularly important to remember that readiness notification from the. Thats why its important to use nonblocking mode when using. An important bottleneck in this method is that read or sendfile. Same thing goes for memory mapped disk files. The first time a server needs disk IO, its process blocks. This is what asynchronous IO is for, but on systems that lack AIO. IO can also get around this. One approach is to use memory mapped files. IO is needed, ask a worker to do the IO. Jef Poskanzer mentions that. Pai, Druschel, and Zwaenepoels 1. Flash web server uses this trick they gave a talk at. Usenix 9. 9 on it. It looks like mincore is available in BSD derived Unixes. Solaris, but is not part. Single Unix Specification. Its available as part of Linux as of kernel 2. Chuck Lever. in November 2. Vivek Pei et al reported. Flash web server. One bottleneck they found was. Another was the fact that sendfile blocks on disk access. EWOULDBLOCK. when the disk page its fetching is not yet in core. Not sure how you tell the user the page is now resident. The end result of their optimizations is a Spec. Web. 99 score of about 8. GHZ1. GB Free. BSD box, which is better than anything on. There are several ways for a single thread to tell which of a set of nonblocking sockets are ready for IO. Unfortunately, select is limited to FDSETSIZE handles. This limit is compiled in to the standard library and user programs. Some versions of the C library let you raise this limit at user app compile time. There is no hardcoded limit to the number of file descriptors poll can handle. Some OSs e. g. Solaris 8 speed up poll et al by use of techniques like poll hinting. Niels Provos for Linux in 1. This is the recommended poll replacement for Solaris. The idea behind devpoll is to take advantage of the fact that often. With devpoll, you get an open handle to devpoll, and. OS just once what files youre interested in by writing to that handle. It appeared quietly in Solaris 7. Sun. at 7. 50 clients, this has 1. Various implementations of devpoll were tried on Linux, but. Linux is not recommended. Caution the example is for Linux devpoll, might not work right on Solaris. This is the recommended poll replacement for Free. BSD and, soon, Net. BSD. See below. kqueue can specify either edge triggering or level triggering. Readiness change notification or edge triggered readiness notification. It then assumes you. EWOULDBLOCK error on a send, recv, or accept call, or a send or recv transfers. When you use readiness change notification, you must be prepared for spurious. This is the opposite of level triggered readiness notification. Its a bit less forgiving of programming mistakes, since. Nevertheless, I have found that edge triggered readiness notification. Open. SSL easier, so its worth trying. Banga, Mogul, Drusha 9. There are several APIs which let the application retrieve file descriptor became ready notifications. This is the recommended edge triggered poll replacement for Free. BSD and, soon, Net. BSD. Free. BSD 4. Net. BSD current as of Oct 2. See also Jonathan Lemons page. BSDCon 2. 00. 0 paper on kqueue. Like devpoll, you allocate a listening object, but rather than opening the file devpoll, you. To change the events you are listening for, or to get the.