|
View:
New views
20 Messages
—
Rating Filter:
Alert me
|
| < Prev | 1 - 2 - 3 - 4 | Next > |
|
|
Httpd 3.0 or something elseSo, after several conversations at Apachecon and on the list, we still have
no real "vision" of how we want to move ahead with httpd "3.0." Or, if we do, it's not communicated very well. Some have suggested we just create a new server project. Others want to keep hacking away at the current code base. Thoughts? -- Brian Akins |
|
|
Re: Httpd 3.0 or something elseLet's get 2.4 out. And then let's rip it to shreds and drop
buckets/brigades and fold in serf. On Nov 4, 2009, at 1:26 PM, Akins, Brian wrote: > So, after several conversations at Apachecon and on the list, we > still have > no real "vision" of how we want to move ahead with httpd "3.0." Or, > if we > do, it's not communicated very well. > > Some have suggested we just create a new server project. Others want > to keep > hacking away at the current code base. > > Thoughts? > > -- > Brian Akins > |
|
|
Re: Httpd 3.0 or something elseI'm with Jim,
Head for 2.4 first. IIRC there was some talk about moving to a 'd' project, since httpd now does ftp (mod_ftp), echo, pop3,... and some other protocols. I don't remember much from it though. I did like the idea back then but thats about the only thing I remember from that. Maybe we could also poll the user base? I know there was a restart on the debate about the current conf and lua/perl/whatever not so long ago so maybe these are all things to look into again for a 3.0? Just my .2 cents /me off to studying windows 2008 server -_- ~Jorge On Wed, Nov 4, 2009 at 8:30 PM, Jim Jagielski <jim@...> wrote: > Let's get 2.4 out. And then let's rip it to shreds and drop > buckets/brigades and fold in serf. > > On Nov 4, 2009, at 1:26 PM, Akins, Brian wrote: > >> So, after several conversations at Apachecon and on the list, we still >> have >> no real "vision" of how we want to move ahead with httpd "3.0." Or, if we >> do, it's not communicated very well. >> >> Some have suggested we just create a new server project. Others want to >> keep >> hacking away at the current code base. >> >> Thoughts? >> >> -- >> Brian Akins >> > > |
|
|
Re: Httpd 3.0 or something elseOn Wed, 4 Nov 2009, Jim Jagielski wrote:
> Let's get 2.4 out. And then let's rip it to shreds and drop > buckets/brigades and fold in serf. I agree, finishing off 2.4 first seems reasonable. I also agree with Brian that plans/visions for future versions would be nice, especially for us "lesser hackers" that aren't able to attend the Cons ... > > On Nov 4, 2009, at 1:26 PM, Akins, Brian wrote: > >> So, after several conversations at Apachecon and on the list, we still have >> no real "vision" of how we want to move ahead with httpd "3.0." Or, if we >> do, it's not communicated very well. >> >> Some have suggested we just create a new server project. Others want to >> keep >> hacking away at the current code base. >> >> Thoughts? >> >> -- >> Brian Akins > /Nikke -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@... --------------------------------------------------------------------------- Hey you, watch me pull a tribble out of my hat! =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= |
|
|
Re: Httpd 3.0 or something elseJim Jagielski wrote:
> Let's get 2.4 out. And then let's rip it to shreds and drop > buckets/brigades and fold in serf. I think we should decide on exactly what problem we're trying to solve, before we start thinking about how it is to be solved. I'm keen to teach httpd v3.0 to work asynchronously throughout - still maintaining the prefork behaviour as a sensible default[1], but being asynchronous and non blocking throughout. [1] The fact that dodgy module code can leak, crash and be otherwise unsociable, and yet the server remains functional, is one of the key reasons why httpd still endures. Regards, Graham -- |
|
|
Re: Httpd 3.0 or something elseOn Thu, 2009-11-05 at 13:38 +0200, Graham Leggett wrote:
> I'm keen to teach httpd v3.0 to work asynchronously throughout - still > maintaining the prefork behaviour as a sensible default[1], but being > asynchronous and non blocking throughout. > > [1] The fact that dodgy module code can leak, crash and be otherwise > unsociable, and yet the server remains functional, is one of the key > reasons why httpd still endures. +1 That the concept is not outdated, we just need to look at Google's Chrome. -- Bojan |
|
|
Re: Httpd 3.0 or something elseHow about support of openmp?
Regards, Jie |
|
|
Re: Httpd 3.0 or something elseOn 05/11/09 12:38, Graham Leggett wrote:
> Jim Jagielski wrote: > >> Let's get 2.4 out. And then let's rip it to shreds and drop >> buckets/brigades and fold in serf. > > I think we should decide on exactly what problem we're trying to solve, > before we start thinking about how it is to be solved. > +1 I'd like we remove the entire forwarding proxy stuff for example. There are also few other things that simply doesn't fit inside 'that web server thing' thought. Others might simply have different ideas. So IMHO we should define what we wanna do first. Regards -- ^TM |
|
|
Re: Httpd 3.0 or something else2009/11/5 Graham Leggett <minfrin@...>:
> Jim Jagielski wrote: > >> Let's get 2.4 out. And then let's rip it to shreds and drop >> buckets/brigades and fold in serf. > > I think we should decide on exactly what problem we're trying to solve, > before we start thinking about how it is to be solved. > > I'm keen to teach httpd v3.0 to work asynchronously throughout - still > maintaining the prefork behaviour as a sensible default[1], but being > asynchronous and non blocking throughout. > > [1] The fact that dodgy module code can leak, crash and be otherwise > unsociable, and yet the server remains functional, is one of the key > reasons why httpd still endures. Sorry, long post but it was inevitable that I was going to air all this at some point. Now seems a good as time as any. I'd like to see a more radical architecture change, one that recognises that it isn't just about serving static files any more and provides much better builtin support for safe hosting of content generating web applications constructed using alternate languages. Before anyone jumps to the conclusion that I want to start seeing even more heavy weight applications being run direct in the Apache server child processes that accept initial requests, know that I don't want that and that I actually want to promote a model which is the opposite and which would encourage people not to do that. As first step, like Jim I would like to see the current Apache server child processes (workers) being asynchronous. In addition to that though, I would like to see as part of core Apache, and running in parent process, a means for spawning and monitoring of distinct processes outside of the set of worker processes. There is currently support in APR and in part in Apache for 'other' processes via 'apr_proc_other_child_???()' functions, but this is quite basic and you still need to a large degree need to roll your own management routines around that for (re)spawning etc. As a result, you see modules such as mod_cgid, mod_fastcgi, mod_fcgid, mod_wsgi all having their own process management code for managing either their daemon processes and/or manager process. Technically one could implement this as a distinct module called mod_procd which had an API which could be utilised by other modules and stop duplication of all this stuff, but perhaps needs to go a step further than that as far as being integrated into core. This is because at present any 'other' processes are dealt with rather harshly on graceful restarts because they are still simply killed off after a few seconds if they don't shutdown. Being able to extend graceful restart semantics into other processes may be worthwhile for some applications. The next thing want to see is for the whole FASTCGI type ecosystem be revisited and for a better version of this concept for hosting web applications in disparate languages be developed which modernises it and brings it in as a core feature of Apache. The intent here being to simplify the task for implementers as well as those wish to deploy applications. An important part of this would be to switch away from the interface being a socket protocol. Instead, let the web server control both halves of the communication channel between Apache worker process and the application daemon process. What would replace the socket protocol as interface would be C API and instead of the application having to implement the socket protocol as foreign process, specific language support would provided as a way of a dynamically loaded plugin. That plugin would then use embedding to access support for a particular language and just execute code in the file that the enclosing code of the web server system told it to execute. By way of example, imagine languages such as Python, Perl or Ruby which in turn now have simplified web server interfaces in the form of WSGI, PSGI and RACK, or even PHP. In the Apache configuration one would simply say that a specific file extension is implemented by a specific named language plugin. One would also indicate that a separate manager process should be started up for managing processes for handling any requests for that language. Only after that separate manager process had been spawned be it by just straight fork or preferably fork/exec would the specific language plugin be loaded. This eliminates the problems caused by complex language modules being preloaded into Apache parent process and causing conflicts with other languages. The existing mod_php module is a good example for causing lots of problems because of it dragging in libraries which aren't multithread safe. That manager process would then spawn its own language specific worker processes as configured for handling actual requests. When the main asynchronous Apache worker processes receive a request and determines that target resource file is related to specific language, it determines then how to connect to those language specific worker processes and proxies the request to them for handling. On the language worker process side the web server part of the code in that process receives the proxied request and then calls into the plugin code to have the request handle against the target file. Because most language solutions for web applications aren't asynchronous, these language specific worker processes would still use traditional threading techniques, or could even be single threaded where language or extension modules for that language aren't thread safe such as is case for PHP. In the bigger scheme of things what we would have is a set of front end Apache worker processes which are asynchronous and which handle static file requests, but where request relates to resources which needed to be implemented by a specific complex language would be proxied internally to other processes managed internally within the sphere of the web server. The language specific worker processes could be single threaded, multithreaded, or could also still provide their own asynchronous API. The important thing is we aren't loading support for these languages into the Apache parent process of the main Apache worker processes. The separation isn't as great as for FASTCGI and is still quite tightly integrated with the web server code effectively controlling both ends of the communication channel used when proxying as well as all the process management. The actual socket protocol used where the proxying occurs is at this point not important as it is a private protocol within the web server and is not intended to be exposed publicly. In other words, the protocol is only used to communicate with the web servers own process on the same server. It is not intended to be used to communicate with process on other servers such as with FASTCGI. For the latter then traditional HTTP proxying techniques can be used. By the protocol being private then it can be changed and updated as needed based on the requirements of the web server and you aren't beholden to some external community and get stuck with a protocol that never gets updated and which over time just becomes a poor solution for the problem needing to be solved, such as in some respects has occurred with FASTCGI where there has never been updates to FASTCGI to make it more modern and usable. The protocol might be packet based like with FASTCGI or AJP, but also might be more like HTTP or something simplified like WAKA. Use of packet based protocols wouldn't be strictly necessary as probably want to avoid trying to multiplex multiple requests over a single proxied channel. Important thing though is to be able to handle end to end 100-continue as necessary, something which FASTCGI cant currently do. Use of distinct manager processes for each language also has other benefits. First is that you could have multiple instances which are configured differently. In other words, multiple process groups to which requests for that language could be delegated. These could be configured differently in respect of number of processes they create for handling requests or number of threads in those process, whether worker processes are precreated, created only on demand, how often they are recycled, killed off when idle or what language modules are preloaded into the manager process before the language specific worker processes are forked. Secondly, process groups could also drop privileges down to different users rather than Apache user, making it a much simpler process to run different applications or different users codes as different users, thereby avoiding the whole mess that is suexec. A third benefit is that the Apache configuration files themselves would really only have details of how to map URLs to those managed processes for each language. The configuration for each language or process group could be distinct. This would allow configuration for process groups to be changed and a process group restarted without having to restart the whole of Apache. As to the language specific worker processes, because the web server code would control the main loop in that that as well you option up better ability to control those processes. This includes better management of process recycling when set number of requests reached, when processes are idle or when memory usage bloats out. It also opens up option for instrumenting code with hooks for collecting statistics about how efficiently processes are handling requests and whether they are getting overloaded and whether the number of processes/threads in a process group needs to be tuned further to cope with load or even to cut back on processes/threads if under utilised. Finally we may even get a chance to improve how error logging is done for such hosted language applications. The current FASTCGI method of proxying error messages back via the request channel has its problems including fact that technically there is no error channel during process startup or between requests. If looking at error logging maybe can come up with better system for handling error logging across a large number of virtual hosts. The current limitations on needing to use VirtualHost or other complex systems for separate live error logs for lots of virtual hosts can be a pain and for VirtualHost doesn't scale for large number of virtual hosts and isn't particular dynamic because of large cost of restarting Apache to add new hosts. So, solve dynamic configuration of virtual hosts where can have separate application error logging and you are definitely on a winner. Anyway, hope some can see an inkling of what I am suggesting. I have left out many details based on my opinions and previous thinking about this and certainly would be much easier to describe this on a white board where can draw pictures. I guess overall I just want to see if we can come up with a more modern web server that is better for complex dynamic web application hosting as well as static file serving. I don't want to see us just go asynchronous, becoming just another static file serving web server like nginx, lighttpd or cherokee and ignore the problem of dynamic web applications and punt that on to a less than capable FASTCGI eco system. Thoughts? Graham |
|
|
Re: Httpd 3.0 or something elseOn 11/5/09 4:30 PM, "Graham Dumpleton" <graham.dumpleton@...> wrote:
> Thoughts? Still digesting, but generally +1 to the entire post. -- Brian Akins |
|
|
|
|
|
Re: Httpd 3.0 or something elseOn 11/5/09 6:08 PM, "Greg Stein" <gstein@...> wrote:
> I think a lot of your discussion can be easily passed off to Apache Thrift. > Let it handle all the message passing to external procceses, and its provided > multi-language support. Or just use HTTP everywhere. IPC is then just HTTP proxy requests. -- Brian Akins |
|
|
|
|
|
Re: Httpd 3.0 or something elseOn Nov 6, 2009, at 8:11 AM, Akins, Brian wrote: > On 11/5/09 6:08 PM, "Greg Stein" <gstein@...> wrote: > >> I think a lot of your discussion can be easily passed off to Apache >> Thrift. >> Let it handle all the message passing to external procceses, and >> its provided >> multi-language support. > > Or just use HTTP everywhere. IPC is then just HTTP proxy requests. +1... |
|
|
Re: Httpd 3.0 or something elseOn Nov 5, 2009, at 4:17 PM, Mladen Turk wrote: > On 05/11/09 12:38, Graham Leggett wrote: >> Jim Jagielski wrote: >> >>> Let's get 2.4 out. And then let's rip it to shreds and drop >>> buckets/brigades and fold in serf. >> >> I think we should decide on exactly what problem we're trying to >> solve, >> before we start thinking about how it is to be solved. >> > > +1 > > I'd like we remove the entire forwarding proxy stuff > for example. There are also few other things that simply > doesn't fit inside 'that web server thing' thought. > Others might simply have different ideas. > > So IMHO we should define what we wanna do first. So we have mod_forward_proxy and mod_reverse_proxy? Interesting take. Would make some sense to make mod_proxy and top-level "framework" and forward/reverse as submodules. |
|
|
Re: Httpd 3.0 or something else2009/11/7 Greg Stein <gstein@...>:
> That's fine, though I think Thrift will give a cleaner model to perform and > handle the requests. > > Mostly I just wanted to point out an alternative to what I read as a lot of > complexity being considered, yet a problem that has already been solved. How does Thrift make it easier? It looks even more complex and scary. The target audience I have in mind is a dumb user who wants to run their PHP, Python, Ruby, Perl web application on a shared hosting system provided by a web provider, as well as the equally dumb hosting companies who want to provide the service. With what I am suggesting, although the implementation is doing a lot and could be regarded as complex internally, because everything is being handled from end to end then as far as the user is concerned they dump their one file in the right place and everything will just work. For the web hosting company, again because everything is provided as a complete package the configuration and setup can be made to be really easy, done one time and not every time. So, as far as I can tell so far, Thrift is for the power user who runs and manages their own site on to which they are deploying their own highly customised application as well as all the infrastructure to support it. I don't see anything in the way of high level interface that allows one to drop a PHP, WSGI, PSGI or RACK application file into a directory and things work automatically. There would therefore be still a lot of work to possibly even make Thrift usable in the context I am talking about and rather than a custom solution which is clean and simple to use you again possibly end up having to deal with trying to configure multiple packages to try and make them work together and all the problems that result from it. As to using HTTP everywhere, that isn't the problem trying to be solved. It is providing a simple means to setup and manage the process management and configuration in one package as well as providing a dead simple deployment mechanism for the dumb user. Graham > On Nov 6, 2009 8:12 AM, "Akins, Brian" <Brian.Akins@...> wrote: > > On 11/5/09 6:08 PM, "Greg Stein" <gstein@...> wrote: > I think a lot > of your discussion can b... > > Or just use HTTP everywhere. IPC is then just HTTP proxy requests. > > -- > Brian Akins > > |
|
|
Re: Httpd 3.0 or something elseOn 06/11/09 20:07, Jim Jagielski wrote:
> >> >> I'd like we remove the entire forwarding proxy stuff >> for example. > So we have mod_forward_proxy and mod_reverse_proxy? Interesting > take. Would make some sense to make mod_proxy and top-level "framework" > and forward/reverse as submodules. > I'd like that we clear the current dependency and API mess. E.g common code depends on balancer (which was a dirty hack I did so we can setup the workers) This would obviously require some decent shared memory code that would allow dynamic config instead stealing the space from the scoreboard. Think that shared memory rewrite was one of the major topics inside 'Amsterdam' discussion few years back. Regards -- ^TM |
|
|
Re: Httpd 3.0 or something elseOn Wed, Nov 4, 2009 at 10:26 AM, Akins, Brian <Brian.Akins@...> wrote:
> So, after several conversations at Apachecon and on the list, we still have > no real "vision" of how we want to move ahead with httpd "3.0." Or, if we > do, it's not communicated very well. > > Some have suggested we just create a new server project. Others want to keep > hacking away at the current code base. > > Thoughts? I see no reason to call what we have been working on anything other than 2.4. A 3.0, a fundamental architectural shift, would be interesting to discuss, I am not sure there is a ton of value in it, though, to be honest. |
|
|
Re: Httpd 3.0 or something elseOn 11/9/09 12:32 AM, "Brian McCallister" <brianm@...> wrote:
> A 3.0, a fundamental architectural shift, would be interesting to > discuss, I am not sure there is a ton of value in it, though, to be > honest. So I should continue to investigate nginx? ;) FWIW, nginx delivers on its performance promises, but is a horrible hairball of code (my opinion). We (httpd-dev type folks) could do much better - if we just would. (Easy for the guy with no time to say, I know...) -- Brian Akins |
|
|
Re: Httpd 3.0 or something elseAkins, Brian wrote:
> FWIW, nginx delivers on its performance promises, but is a horrible hairball > of code (my opinion). We (httpd-dev type folks) could do much better - if > we just would. (Easy for the guy with no time to say, I know...) I think it is entirely reasonable for the httpd v3.0 codebase to do this as a goal: - Be asynchronous throughout; while - Supporting prefork as httpd does now; and - Allow variable levels of event-driven-ness in between. This gives us the option of prefork reliability, and event driven speed, as required by the admin. Regards, Graham -- |
| < Prev | 1 - 2 - 3 - 4 | Next > |
| Free embeddable forum powered by Nabble | Forum Help |